STM32L4: how to read from UART without knowing message length?

[Sven Köhler]

Posted on March 16, 2018 at 00:30

I am trying to use USART2 of my Nucleo L476RG board. It is connected to the UART of the ST-LINK on the board. 

I'm trying to implement a simple line-based protocol (every message is terminated by \r or \n). For starters, I try to implement a simple 'echo server' which sends every received line back to the sender. 

I get the feeling that the HAL is completely unsuitable here. One would expect, that the HAL_UART_Receive function (receive without callbacks) would return the number of bytes received. That is not the case. When using 

HAL_UART_Receive_IT

, one would expect that the HAL_UART_RxCpltCallback callback would receive some information about how many bytes were received. That is not the case.

In fact, 

HAL_UART_Receive returns only after the specified number of bytes have been received. In fact, 

HAL_UART_Receive_IT only calls the callback, after the specified number of bytes have been received.

After my simple echo server is complete, my next step would be to implement a simple software FIFO buffer. Any data received via 

USART2

would go into that buffer. Whatever is received via the UART is transferred into the software buffer until the buffer is full.Any operating system has such a buffer, as the hardware buffers are usually not big enough. Again, the HAL UART interface doesn't seem to allow to implement that

.

I googled and it seems that STM microcontrollers implement some sort of 'idle detection'. When no data is received for some time, the U(S)ART interrupt is triggered even if there's just a single byte in the hardware buffer.

Some people claimed that HAL simply does neither enable nor support that idle detection. If that was true, HAL would be useless for any sort of generic buffering (as described above) and it would be useless for any protocol with variable-length messages (such as line-based protocols). To be honest, I can't believe it. Some Modem from the 1980s supports a line-based protocol via RS232, but my HAL-based STM32 application in 2018 can't.

Are those people correct? Am I missing some API here, that would enable this idle detection? I find it mentioned nowhere in the HAL code. Also, the API does not even seem to be prepared. No information about how many bytes have been received can be returned by any of the receive functions. To do so, is a pretty standard way of designing any sort of receive/read function.

Reading byte by byte in the main loop is not an option. Reading byte by byte via interrupts would cause way to many interrupts (if it would work at all).

#usart-interrupt #uart-interrupt #stm32l4-hal #rs232

[Tesla DeLorean]

Posted on March 16, 2018 at 01:52

>>

Reading byte by byte via interrupts would cause way to many interrupts (if it would work at all).

Not sure you understand how the IRQ handler works then because each byte does generate an IRQ, and when you call into the HAL enough it calls your callback in interrupt context, so it better get done within one byte time. The L4+ parts have a FIFO on the USART.

The simplified approach

https://community.st.com/s/feed/0D50X00009XkVxKSAV

Nine circles approach

https://community.st.com/s/question/0D50X00009XkWtfSAF/implementing-interrupt-driven-stream-uart-rx-handling-with-stm32cubemx-drivers

[Jack Peacock]

Posted on March 16, 2018 at 02:48

I don;t use the HAL but the 'L4 does have hardware support for what you want to do.  In the USART section of the reference manual look at the section on Modbus.  What you describe is the the Modbus ASCII protocol, where you set up an interrupt for the address matching field for the trailing frame suffix character (\n for Modbus ASCII).  This gives you the mark for the end of a message frame and by implication the start of the next.  This supports using DMA where the RX channel is terminated when the interrupt occurs.

If you use multi-drop you'll need some kind of filter at the USART character RX to determine if the leadin for the next frame is addressed to your node.  If not you throw away the packet (i.e. DMA the entire packet to one location, non-increment memory) until the next end of frame mark occurs.

If you have the memory you can actually buffer multiple frames, so that the CPU handles a completed frame while DMA receives the next incoming.  Using this technique you can get to some very high data rates depending on your physical transport.  I've used this reliably at 460Kbaud over half duplex RS-485 (but don't run over 115Kbaud if you use RS-232 due to slew rates).  DMA offloads 99% of your RX overhead, and if you're clever about using CCM, SRAM1 and SRAM2 the DMA transfer is free of bus conflicts, effectively a true parallel operation with the CPU and DMA running separate transfer paths through the bus matrix switch.

You can use idle detection for binary transfers, where there are no framing characters.  This is much more efficient in terms of transfer rates since there is no hex conversion needed for binary data, effectively you get twice the bandwidth at the same data rate.  The Modbus section in the manual covers this too (section on Modbus/RTU).  You do have to insure a minimum gap between frames, and maximum (but less than frame) gap time between characters.  Don't count on a PC being able to do this, especially if it's going through a USB to serial adapter.  For STM32 to STM32 you can guarantee gap timing by using DMA to send packets.  Idle detection in older STM32's had a problem with the timing gap but this is fixed in the L4 series.  You can program the gap time before idle triggers.

As always if you use high data rates (>115Kbaud) you have to watch out for mismatched baud rates, especially with large packet sizes.  Asynchronous serial really needs a gap between characters to reset the async bit timing window, which is what makes async serial tolerant of slight mismatches in baud rates.  If you jam a lot of bits together without spacing you run the risk of start bits not lining up.  That's one of the reasons multiple stop bit setting are there, to add a delay between characters so the receiver bit clock can reset.

Jack Peacock

[Sven Köhler]

Posted on March 17, 2018 at 02:06

Thank you for your clarification. I am surprised. A rate of 115.200 baud, that would mean 14400 interrupts per second. Running at 20MHz, that would mean at most 1388 cycles per interrupt.

This begs for some very simple software buffer (like your first example provides), so that the data can be processed by the main loop later.

The simplicity of the code you linked compared to the complexity of the HAL-based code is just amazing. I would have to call 

HAL_UART_Receive_IT with a length of 1 byte over and over again - actually, also from the interrupt routine itself.(HAL stops calling any callbacks after the requested number of bytes has been received, so I have to re-initiate the transfer again and again). That is just way too much hassle for such a simple thing. And maybe I would exceed the 1388 cycles limit, but I don't want to push it.

[Tesla DeLorean]

Posted on March 17, 2018 at 02:48

>> 14400

It will be below 11520, but yes if I were running at 20 MHz I'd be using a circular DMA buffer as a FIFO. Done that on the L0 I'm running in that realm, 24-27 MHz where the flash tops out for zero wait state.

[LMI2]

Posted on March 17, 2018 at 14:15

These look interesting

The simplified approach

https://community.st.com/0D50X00009XkVxKSAV

Nine circles approach

https://community.st.com/0D50X00009XkWtfSAF

The first is made by you, Clive One. Do you know if it works in other CPUs too, like 407 or 746?

[Tesla DeLorean]

Posted on March 17, 2018 at 14:37

The L4 has a different UART construction than the F4, the former having flags in the ISR rather than SR, and having TDR/RDR rather than DR, clearing error states is also different. As I recall the F7 is like the L4. The L4+ add a FIFO into the mix.

The concept isn't hard to port across platforms.

I think I've shown similar code for F2/F3/F4 platforms before, the L4 was interesting as the recovery from overrun/noise errors is more critical when you have a streaming input you have been ignoring for a while.

[henry.dick]

Posted on March 17, 2018 at 18:09

you will need to design a 'protocol' to signal to the receiver the end of your messages. '\n' or '\r', or even '0' - which I often use.

if there is truly no way of knowing the length of the massage (ie it could be from 1 char to infinite number of chars), use a circular buffer -> you run the risk of losing data receiver earlier.

[Sven Köhler]

Posted on March 18, 2018 at 00:01

Maybe I wasn't clear, but the message size is unknown in the sense that it is determined by the something inside the byte-stream that is being received. To the best of my knowledge, any protocol that uses \r, \n or even '0' can be called a protocol with unknown message size, as the message size unknown a priori (it is unknown before the data has been received).