Efficient way to process USART-received data and flush RX buffer ?

[marcosartore]

I have a transmitter over USART and an STM32H7 receiving data. The transmitter always writes N bytes at a slow repetition rate, i.e. every 1 second or more.

Once N data are received by the STM32H7, for example using interrupt, the function HAL_UART_RxCpltCallback() is executed. There I can check the received buffer and parse it. So far, so good.

However, if an extra junk byte is received, the "modulo N" transaction is lost forever, because the next time HAL_UART_RxCpltCallback() will be fired, the RX buffer will be (at least) one-byte shifted and my parsing will fail.

Is there a way of flushing the receive buffer if an inconsistency is detected, thus waiting for the next, hopefully clean, transmission ?

The only solution I've figured out so far is to get alerted when every one single byte is received (calling HAL_UART_Receive_IT(&huart2, rx_buf, 1)), in order to be able to locate my message start byte and process the following bytes accordingly.

Is there any smarter and less time consuming way ?

Thanks a lot in advance,

Marco

[Guenael Cadier]

Hello @marcosartore​ 

Could you check if in your stm32h7xx_hal_uart.h, you have the below definition ? (you should)

#define __HAL_UART_SEND_REQ(__HANDLE__, __REQ__) ((__HANDLE__)->Instance->RQR |= (uint16_t)(__REQ__))

If yes, you could flush RX data (current data in RDR and/or in shift register) by calling :

 __HAL_UART_SEND_REQ(huart, UART_RXDATA_FLUSH_REQUEST);

Hope this helps.

Regards

[marcosartore]

Hello Guenael,

many thanks for your hint. I confirm that the indicated #define is present in all the stm32h7xx_hal_uart.h files I have under STM32CubeIDE/workspace_1.6.1/ sub folders: uart_rx_dma, uart_rx_int and uart_rx_int_2.

Unfortunately, after the execution of the indicated flushing command inside HAL_UART_RxCpltCallback() , this function gets no longer called at all (despite it ends with a renewed call to HAL_UART_Receive_IT(&huart2, rx_buf, N).

Am I missing anything ?

Thanks !

[Guenael Cadier]

Hello @marcosartore​ 

My understanding is that when HAL_UART_RxCpltCallback() is executed (i.e. after you receive your N bytes), the extra junk byte is not yet received, so the flush has no real effect. But I don't understand why reception is no more working at all ... (The rx fifo flush, clears the already received data without need to read them fromTDR, but does not prevent further reception).

Anyway, I'm not sure flush feature might be of any help in your case. Could you rather adapt your parsing to be able to handle extra junk byte between valid N bytes pattern ?

Regards

Guenael

[marcosartore]

Many thanks for further answering !

Perhaps I am in the wrong direction. What I wish to achieve is a secure reception of an N-bytes command ALSO in the case an extra junk byte appears on the scene.

In this latter case, yes, the parsing routine can understand that it was a bad command and forget about it, waiting for a good one.

The problem is that in my picture if an extra byte was fired, i.e. (N+1) bytes arrived with a junk byte somewhere in between, when the HAL_UART_RxCpltCallback() is executed, the RX buffer will immediately contain one byte. After the next command sent, the RX buffer will contain: the extra byte at position [0] plus (N-1) correct bytes. But the parser will never recognize this and the following buffers as valid, because we will always have a 1-byte shift forever.

By the way, I also figure out that a problem arises if one of the N bytes gets lost during transmission. In this case the command will not be parsed until the next one is sent, which indeed will fill the next RX buffer with (N-1) data, and so on.

As I repeat, probably my picture is wrong, nor I am forced to follow exactly this procedure. If you know about a robust protocol to handle a simple N-byte command reception, I would be happy to implement it.

So far, firing HAL_UART_RxCpltCallback() at every single byte received is the most secure way I've found.

Marco

[Tilen MAJERLE]

@marcosartore​ perhaps you should receive any character from outside device and then do parsing in main loop? With ring-buffer?

Or by using DMA in circular mode and then process any received byte: https://github.com/MaJerle/stm32-usart-uart-dma-rx-tx

[Paul1]

Can you modify the protocol? Is the data binary?

• If binary protocol consider using "DLE stuffing". Use reserved characters for Packet start, packet end, and stuffing. You don't have to use a DLE character. Personally I use [ ] ^ as easy to see those in a terminal program. Also allows triggering CRC check upon ] which simplifies packet processing. Stuffing consists of replacing any of the 3 reserved characters with ^+modified char, i.e. [ in packet becomes ^ [+0x20 which is "^{" if using 0x20 as the modifier. On receive of ^ you simply subtract 0x20 from next char.
• if text protocol set to process Rx on CR or LF instead of a fixed length.
• Both methods support variable and fixed length packets/commands, so you don't have to send the length but can count data between start/end chars.

Paul

[marcosartore]

Thanks to Tilen and Paul for answering with great suggestions.

Tilen, the full-featured DMA approach is a great piece of code, thanks for sharing it. The approach of polling in a thread new-data arrival, provided that DMA takes care of storing them with no-cpu effort, looks a good compromise solution.

Paul, the idea to manage start/stop reserved chars that way easy up processing of binary data and allows also those 'special' chars to be part of the message.

Marco