STM32F410 uart1 lost break/framing error interrupt

[law__]

Hi all,

Im using STM32F410 uart 1 to transmit and receive.
I use DMA to transmit bytes. I then use TC interrupt to send a break,
then push an event to an event queue to signal that the transmitter is free.

void interface_uart_tx_eot_isr(if_uart_t* p) {
    if (p->tx_last_in_chunk) {
        // Transmit break at end of tx work
        LockFromISR();
        p->uart_dev->usart->CR1 |= USART_CR1_SBK;
        UnlockFromISR();
        // register event for more tx data
        p->transmit_empty_handlerI(p->event_source);
    }
}

On the same uart I use DMA in circular mode to dump incoming bytes.
Im using framing error interrupt to wake up a worker to deal with the received data.
For the sake of this explanation, I have three stm32f410 devices connected in a ring
1Tx -> 2Rx, 2Tx -> 3Rx, 3Tx -> 1Rx.
So it is possible for, eg, device 2 to be recieving data as it is sending data.

This is working well except:
I find occasionally that I am missing a break interrupt and I'm pretty lost figuring it out.

Oscilloscope shot uart_break_1.jpg shows 3 cycles of data.
The first is successful (5 uart interrupts - Blue trace).
500us later the second fails (3 uart interrupts), then 500us after that the error is detected and a breakpoint hit (yellow going high on another device - data exchange is stopped).
Note the last one is semi-successful (5 interrupts).

The oscilloscope traces are:
Green: device TX
Pink: device RX
Blue: uart interrupt (gpio toggled in interrupt)

In oscilloscope screen uart_break_2.jpg:
See data is coming in via pink. The break is also present.
At the same time, data is being sent out via green.
Blue going low indicates the uart interrupt firing, then the TC interrupt handler transmits break.

I tracked the SR register of uart 1 to see if the framing error bit is actually set:

  }, {
    sr_val = 208,
    time_us = 19787747  <--- one where error occurs. FE bit is not set. SR flags: TXE, TC, IDLE
  }, {
    sr_val = 210,       <--- FE correctly captured
    time_us = 19788109
  }, {
    sr_val = 0,
    time_us = 19788138
  }, {
    sr_val = 192,
    time_us = 19788177
  }, {
    sr_val = 210,       <--- FE correctly captured
    time_us = 19788247
  }, {
    sr_val = 192,
    time_us = 19788379  <--- Last gpio toggle

These are the last 6 SR register values captured in the interrupt.
Note in uart_break_2.jpg the uart interrupt with TC flag set fires about 200ns after RX break (pink) goes low.

I'm using Chibios and will add the usart handler.
Break is capture via FE and _uart_rx_error_isr_code and TC via _uart_tx2_isr_code.
I'm at a bit of a loss. I suspect break is not quiet set when TC interrupt happens. Maybe it is then getting cleared during servicing of TC.

Note, this is working pretty well, I just get these dropped FE interrupts about every 10-20 seconds.
Any thoughts?

void serve_usart_irq(UARTDriver *uartp) {
  uint16_t sr;
  USART_TypeDef *u = uartp->usart;
  uint32_t cr1 = u->CR1;

  sr = u->SR;   /* SR reset step 1.*/

  // Record SR and signal
  LED_LCL_TOGGLE;
  sr_cap_t cap = {
    .sr_val = sr,
    .time_us = mod_time_util_us_get()
  };
  sr_capture_history[sr_history_idx] = cap;
  sr_history_idx++;
  if (sr_history_idx >= 16) {
    sr_history_idx = 0;
  }

  if (sr & (USART_SR_LBD | USART_SR_ORE | USART_SR_NE |
            USART_SR_FE  | USART_SR_PE)) {

    (void)u->DR;  /* SR reset step 2 - clear ORE.*/

    u->SR = ~USART_SR_LBD;
    _uart_rx_error_isr_code(uartp, translate_errors(sr));
  }

  if ((sr & USART_SR_TC) && (cr1 & USART_CR1_TCIE)) {
    /* TC interrupt cleared and disabled.*/
    u->SR = ~USART_SR_TC;
    u->CR1 = cr1 & ~USART_CR1_TCIE;

    /* End of transmission, a callback is generated.*/
    _uart_tx2_isr_code(uartp);
  }

  /* Timeout interrupt sources are only checked if enabled in CR1.*/
  if ((cr1 & USART_CR1_IDLEIE) && (sr & USART_SR_IDLE)) {
    _uart_timeout_isr_code(uartp);
  }
}

[waclawek.jan]

UART_SR.FE is cleared by a sequence of read-SR-then-read-DR.

If your ISR fires because of UART_SR.TC, reads UART_SR (priming the sequence) with UART_SR.FE=0, then the break's end occurs, UART_SR.FE gets set, the Rx DMA reads UART_DR, that completes the sequence and clears UART_SR.FE.

What's IMO more dangerous is reading UART_DR in the ISR, while simultaneously DMA on Rx is set. Consider a scenario where say UART_SR.NE is set, so that the ISR branches into the UART_DR-reading code. Now if a byte arrives so that UART_SR.RXNE is set a few cycles before code reads UART_DR, the Rx DMA starts to go through its paces when the processor reads UART_DR and thus removes the UART_SR.RXNE signal, that may lead to the Rx DMA Stream to lock up. The coincidence needed for this to happen is high and the probability low, thus you'd end up with a rarely occuring problem detrimentally affecting your system reliability.

There may or may not be a solution for the primary issue in the method you've devised, I don't know. UART Rx and DMA don't mix that well and there are several corner/race cases like these to be considered, using interrupts are simpler thus potentially safer. The newer STM32 UARTs are slightly different in the way how flags are cleared. Another option to contemplate is also to use two separate UARTs for Tx and Rx.

JW