Best practice using DMA to read from UART and write to SD?

[Lars Beiderbecke]

Posted on June 17, 2018 at 12:09

I'm reading data from a UART with DMA and want to write that data to an SD card (on a F722). Since the data is much longer than my buffer, I chose circular buffer for the DMA.

The problem now is that I need to save the receive buffer to SD occasionally, but how do I know when I need to do this?

Can someone recommend the best approach here, using Cube HAL:

• Should I chop my Receive_DMA() into smaller chunks of buffer size, and receive them individually?  (I may loose data.)
• Should I count the number of received bytes (how?) and save the buffer if some threshold is reached?
• Can I enable direct UART-to-SDMMC DMA?  (I don't even know what that means: Does the data go into an open file, and how can I remove the length header of the data first?)

I'd really appreciate some ideas. The Cube examples are way too simple.

#dma

[Tilen MAJERLE · Accepted solution]

Posted on June 17, 2018 at 12:22

Hello,

if you are using DMA in circular mode to receive data over UART, you will not loose your data, if your RX buffer can handle all the bytes received by UART by the time you are saving another part of RX buffer. This depends on UART baudrate and time of saving to SDCARD.

When to save data to card? I would handle 3 different events:

• Half-transfer complete interrupt, means that first part of buffer is full, now let's save it to card
• Transfer complete interrupt, means second part of buffer is full, let's save second part to buffer
• Check some time difference (maybe IDLE line interrupt? Depends on your UART stream) between last sdcard save and current time, then check if there are any data in RX buffer which were not yet processed (like cache) and save it.

You may count number of bytes received by checking stream/channel NDTR register.

Best regards,

Tilen

[Lars Beiderbecke]

Posted on June 17, 2018 at 12:28

Thanks, Tilen, that is very helpful!

Just a small follow-up question:  What is the HAL way of checking NDTR (assuming there is one)?

Also, how do I check NDTR? In a while loop or in an interrupt? (Does a while loop make sense, because it uses again CPU cycles?)

[Tilen MAJERLE]

Posted on June 17, 2018 at 13:42

Hello Lars,

there are different way how to do it, most of them depends on project complexity. Specifically, if you are using F722 MCU, I would assume that UART2SDCARD is not the only task you have to handle, thus I would go with operating system for sure. MCU has enough memory to handle and CPU is smart enough to do the job using RTOS. ST provides FreeRTOS port as part of STM32Cube software.

Example, how i would do it is like this

• Create new task (thread), dedicated only for reading UART and saving data to SDCARD
  • FatFS (if you are using it) supports RTOS mode, so you are safe to use it. Please read FatFS documentation what implementation is mandatory for RTOS mode
• Use single message queue to writenotifications from interrupts for DMA HT or TC interrupts
• Use LL drivers to configure UART and DMA for RX mode and enable HT and TC interrupts
• Wait for new event on message queue or timeout value, when you decide to check for new data

Pseudo code example below:

void
uart_to_sdcard_thread(void) {
 last_dma_position = 0;
 queue = create_message_queue(10); //Create message queue for 10 elements
 
 while (1) {
 //Wait for new queue or timeout
 //If there is timeout, you know that neither TC or HT events happened
 //so manually check if there is anything to read
 message_queue_wait_msg(queue, MAX_TIMEOUT_VALUE);
 if (timeout_happened_on_read) {
 //There was no message in queue but it was timeout
 }
 //Remember: NDTR is downcounter and holds number of 'remaining' elements to transfer
 new_position = LL_DMA_GetDataCounter(...)
 //Do some math, and save to card
 }
}
//DMA interrupt handler
void
dma_irq_handler(void) {
 if (tc_irq) {
 message_queue_write(queue, tc);
 }
 if (ht_irq) {
 message_queue_write(queue, ht);
 }
}
�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?�?

If you need example for message queues on STM32F7, follow this example:STM32Cube_FW_F7_V1.0\Projects\STM32756G_EVAL\Applications\FreeRTOS\FreeRTOS_Queues

Some readings, which may help you to start:

https://community.st.com/thread/42689-efficiently-use-dma-with-uart-rx-on-stm32

[Tesla DeLorean]

Posted on June 17, 2018 at 14:08

/**

* @brief Returns the number of remaining data units in the current DMAy Streamx transfer. * @param __HANDLE__ DMA handle * * @retval The number of remaining data units in the current DMA Stream transfer. */ ♯ define __HAL_DMA_GET_COUNTER(__HANDLE__) ((__HANDLE__)->Instance->NDTR)

If you configure the USART-DMA in a circular fashion, with HT and TC DMA interrupts triggering, you'll know when each half of the ping-pong buffer is available.

Step back from HAL, and go read the Reference Manual, understand the mechanics afforded by the hardware, then decide if the HAL allows you to implement that cleanly/effectively.

The SDMMC and FATFS code really should be in it's own thread, or as the foreground task. I would suggest writes of at least 8KB, which in a continual streaming model would use a 16 KB DMA FIFO.

The alternative is to have a much smaller USART-DMA arrangement, and use a larger file buffer, periodically flushing that it fills or an inactivity timer expires.

>>Can I enable direct UART-to-SDMMC DMA?

No plumbing like that exists, the hardware is simple, complexity is handled in software. You need to manage the file system and the block level protocol used for the SDMMC

>>

The Cube examples are way too simple.

They are designed to be illustrative, not substitute for commercial development efforts. If we want better examples we need to fund technical support properly.

I posted an ADC+DMA+SDIO recorder (F4 SPL) several years back.

[Lars Beiderbecke]

Posted on June 17, 2018 at 15:04

Thanks, I didn't know that there are predefined interrupts for half and full, as they're not listing in CubeMX.

Regarding examples, I'm just saying that they should be somewhat exhaustive of regular use. For example, the GPIO example uses just PinToggle, but doesn't read a value. I view examples as part of the documentation, and would assume ST is interested in good documentation in order to attract both commercial users and hobbyists like myself.

[T J]

Posted on June 18, 2018 at 21:42

The easiest way forward is to understand the packet size of the SDCard.

lets say you like to use a 2k packet to the SDCard.

then make the DMA circular buffer double that.

use the half interrupt to send a packet to the SDCard, and the Full interrupt to send the next...

[Lars Beiderbecke]

Posted on June 18, 2018 at 21:32

Hello Tilen,

Thanks for the link, those approaches are interesting.

I'm somewhat wary of the IDLE detection, though. In my case, the UART is connected to an ESP8266. Even if the ESP receives data at full speed, couldn't its data processing and handling of packages not delay maybe just a single byte sent to the UART, thus triggering the IDLE detection?

[Tilen MAJERLE]

Posted on June 18, 2018 at 21:44

Do you need to save actual +IPD data content from ESP8266 device? Then you need somekind of parser which runs on STM32 and then you can decide how to proceed.

https://github.com/MaJerle/ESP_AT_Lib

 

Anyway, I agree that IDLE line will fire immediately after there is no available data on UART for single byte time.

I would suggest you to go with library above to parse ESP data and to use single thread (NETCONN API) to read and write data to your card.

[MSlei.8]

Hi clive1,

Could you please share a link to the "ADC+DMA+SDIO recorder (F4 SPL)" project example you mentioned?

Thanks in advance!