What could be causing the STM32L152 UART Rx interrupt to intermittently not fire when data arrives and interrupts are correctly configured?

[BCogh.2]

Hello all,

First off, I apologize that I am not able to provide code as everything I have been able to do reproduce this problem is proprietary. Given that, I am looking for advance debugging techniques for diagnosing this intermittent failure of the UART interrupt to fire.

The overall problem is that we noted the device would appear to "drop" tens of bytes at a time which appears to be caused by the UART Rx interrupt not firing even though a tap on the UART line shows the data arriving.

Overview of the device and setup in question:

I am working with a legacy device that runs an STM32L152 running Keil RTX. All logic levels are 3.3V, UART is clocked at 16MHz, system clock 32MHz, and the baud rate is 115200. The UART Rx interrupt has the highest priority available to the RTX implementation and the UART interrupt is configured to trigger on errors and received data.

When data is received on the UART Rx line it is read from the data register, placed in a circular buffer, then a signal is sent to a high priority RTOS task to parse the data. If an error flag is detected the interrupt flag is cleared via a read of the flag, then a read of the data register.

What has been tried thus far:

This set up works perfectly 99% of the time, but we notice large chunks of bytes being dropped. Using Keil's event viewer, we can see that even when the RTOS is in idle mode (i.e. it is not in handler mode), the UART Rx interrupt will sometimes not fire.

The NVIC does not appear to be servicing any other interrupt and the only flags set in the UART status register are the TXE and TC from the previously sent frame. Additionally, tracking the state of the configuration registers, we've confirmed that the enabled interrupts configuration is not changing.

We have confirmed that the data arrives on the Rx line of the device via an FTDI UART/USB tap we have placed on the board trace. We have verified that the issue persists regardless of the RTOS tasks running, if an error (framing, overflow, etc.) occurs, or the baud rate (we have observed the behavior with a baud rate of 9600). The amount of data being transferred via UART when this happens is usually a few tens of bytes.

It is not a buffer overflow as we have greatly expanded the size of the circular buffer and ensured that code checking for a buffer overflow does not return true in these situations.

We have also observed this behavior in all of the device we have produced, not just the one I am developing with, and it appears isolated to USART2 even though UARTs 1-4 are used.

I have found nothing in the RTX documentation that I thought could be causing this and am thoroughly out of ideas. In leu of any concrete solution, I've simply had to make our "we've missed something so retry last command" patch more robust.

If anyone has any ideas, thoughts, resources, or avenues I should investigate, please do not hesitate to let me know. I will also be pursuing this question with Keil and U-Blox.

Thank you in advance for any time and thought you put into this,

-Brian

[GLASS]

Give a try to catch an eventual erroneous pin mux reconfiguration...?

[TDK]

The most likely reason is a software bug. There are not glaring errors in the hardware. Perhaps you are busy in another interrupt, perhaps interrupts are disabled. Look at error flags in particular, as some of them will prevent the peripheral from working. Ensure you have a working UART error handler outputting relevant data.

If it happens when the RTOS is idle, consider adding checks for relevant NVIC/UART flags and outputting that info or toggling GPIOs to confirm.

[waclawek.jan]

> Using Keil's event viewer, we can see that even when the RTOS is in idle mode (i.e. it is not in handler mode), the UART Rx interrupt will sometimes not fire.

I don't know what is "Keil's event viewer" exactly, but can't this be the case of debugger (or any monitoring piece of software) reading the UART registers causing status bits (namely RXNE) to be cleared by hardware?

In other words, drop all fancy debugger and monitor, and debug using old fashioned tools - toggle pins, output debug printouts on any line you have available.

JW

[Tesla DeLorean]

Get your priorities right, watch for it to use the interrupt for task swapping.

Callbacks occur under interrupted context, watch you doing have code that blocks or yields. Service interrupt should get data and leave.

Look for overrun, noise, framing or parity errors.

Don't use peripheral views in the debugger, have your own copies of registers​ if you want to check content.

Writing to Flash will block execution, potentially breaking real-time deadlines.

Problem seen with DMA? or just IRQs.​

[BCogh.2 · Accepted solution]

All the suggested solutions are absolutely valid and anyone referring to this post should check all of them if they are having a similar issue, However none of them are the answer to my specific case though I did find the cause.

The cause of the missed interrupts was due to writing to the STM32L1's EEPROM during time that interrupts occurred. The EEPROM banks share memory controllers with the flash banks. Therefore, writing to EEPROM bank 1 occupies the memory controller and bus resources of flash bank 1. However, you can write to one bank while reading from the other, which was the case for the majority of the program execution. The issue arose because an engineer had placed data manipulated by the device on USART2 into bank 1 of EEPROM which was not caught since the memory addresses of the two banks are contiguous. When the device on USART2 would generate a command to update this data, the EEPROM write would stall (~3.2ms) the instruction fetch for the processor meaning that the ISR code pointed to by the interrupt vector could not be fetched and executed to address any interrupts generated. For most EEPROM writes this did not occur since the data memory addresses were in the bank we were not executing code from. Turns out there is an excellent application note explaining it: Writing to non-volatile memory without disrupting code execution on microcontrollers of the STM32L0 and STM32L1 Series

Thank you to everyone who took the time to answer!