Data not updating on each UART_RxCpltCallback

[robotwhisperer]

I have a 2-board communication project set up with 2 NUCLEO-H753ZI boards, where they communicate over UART. The sender side sends a 22-byte stream of data that is received by the receiver side and gets parsed into data. This data can then be streamed to a PC over Ethernet. 

Inside of the UART_RxCpltCallback, I toggle the Red on-board LED, which gives me the rate at which packets are coming in. Inside the UART_TxCpltCallback I am calling a function that takes the data from the UART buffer, and moves it into a member variable of an object. All this should happen at 2KHz, but for some reason the data is only updating every few seconds, and it is not a specific rate at which it updates, it seems random. The graph might better illustrate what I am talking about. The graph *should* look like a sawtooth waveform. 

The confusing part is that the LEDs work as expected, i.e. those blocks of code that are responsible for getting the data where it needs to be are surely executing, they just don't seem to do the thing I think they are doing. 

AFAIK the receive buffer being used is in a non-cacheable region, so the issue should not be MPU or cache related. 

void MPU_Config(void)
{
  MPU_Region_InitTypeDef MPU_InitStruct = {0};

  /* Disables the MPU */
  HAL_MPU_Disable();

  /** Initializes and configures the Region and the memory to be protected
  */
  MPU_InitStruct.Enable = MPU_REGION_ENABLE;
  MPU_InitStruct.Number = MPU_REGION_NUMBER0;
  MPU_InitStruct.BaseAddress = 0x0;
  MPU_InitStruct.Size = MPU_REGION_SIZE_4GB;
  MPU_InitStruct.SubRegionDisable = 0x87;
  MPU_InitStruct.TypeExtField = MPU_TEX_LEVEL0;
  MPU_InitStruct.AccessPermission = MPU_REGION_NO_ACCESS;
  MPU_InitStruct.DisableExec = MPU_INSTRUCTION_ACCESS_DISABLE;
  MPU_InitStruct.IsShareable = MPU_ACCESS_SHAREABLE;
  MPU_InitStruct.IsCacheable = MPU_ACCESS_NOT_CACHEABLE;
  MPU_InitStruct.IsBufferable = MPU_ACCESS_NOT_BUFFERABLE;

  HAL_MPU_ConfigRegion(&MPU_InitStruct);

  /** Initializes and configures the Region and the memory to be protected
  */
  MPU_InitStruct.Number = MPU_REGION_NUMBER1;
  MPU_InitStruct.BaseAddress = 0x30010000;
  MPU_InitStruct.Size = MPU_REGION_SIZE_64KB;
  MPU_InitStruct.SubRegionDisable = 0x0;
  MPU_InitStruct.TypeExtField = MPU_TEX_LEVEL1;
  MPU_InitStruct.AccessPermission = MPU_REGION_FULL_ACCESS;
  MPU_InitStruct.IsShareable = MPU_ACCESS_NOT_SHAREABLE;

  HAL_MPU_ConfigRegion(&MPU_InitStruct);

  /** Initializes and configures the Region and the memory to be protected
  */
  MPU_InitStruct.Number = MPU_REGION_NUMBER2;
  MPU_InitStruct.BaseAddress = 0x30020000;
  MPU_InitStruct.Size = MPU_REGION_SIZE_128KB;

  HAL_MPU_ConfigRegion(&MPU_InitStruct);

  /** Initializes and configures the Region and the memory to be protected
  */
  MPU_InitStruct.Number = MPU_REGION_NUMBER3;
  MPU_InitStruct.BaseAddress = 0x30040000;
  MPU_InitStruct.Size = MPU_REGION_SIZE_512B;
  MPU_InitStruct.TypeExtField = MPU_TEX_LEVEL0;
  MPU_InitStruct.IsShareable = MPU_ACCESS_SHAREABLE;
  MPU_InitStruct.IsBufferable = MPU_ACCESS_BUFFERABLE;

  HAL_MPU_ConfigRegion(&MPU_InitStruct);
  /* Enables the MPU */
  HAL_MPU_Enable(MPU_PRIVILEGED_DEFAULT);

}

Linked script snippet:

  {
    . = ALIGN(8);
    . = ABSOLUTE(0x30000000);
    *(.RxUsartBuffer)

    . = ALIGN(8);
    . = ABSOLUTE(0x30008000);
    *(.TxBuffer)

    . = ALIGN(8);
    . = ABSOLUTE(0x30010000);
    *(.Adc1Buffer)

    . = ALIGN(8);
    . = ABSOLUTE(0x30014000);
    *(.Adc2Buffer)

    . = ALIGN(8);
    . = ABSOLUTE(0x30018000);
    *(.Adc3Buffer)

    . = ALIGN(8);
    . = ABSOLUTE(0x30040000);
    *(.RxDecripSection)

    . = ALIGN(8);
    . = ABSOLUTE(0x30040100);
    *(.TxDecripSection)

    . = ALIGN(8);
    . = ABSOLUTE(0x30040200);
    *(.Rx_PoolSection)

    . = ALIGN(8);

  } >RAM_D2

and the buffer 

std::array<uint8_t, SerialRxBuffer::kSize> SerialCommunication::rx_buffer_
    __attribute__((section(".RxUsartBuffer")));

[BGeor.1]

Make sure DMA buffer mode is correctly set, such as circular buffer or normal. Set FIFO Mode to "Disable",if enabled. Set Data Width for both Peripheral and Memory to be Byte and Byte for debugging.

If you are using the circular buffer for DMA, data in the buffer needs to be indexed appropriately using the DMA DMA_SxNDT register.

You can also look at various DMA registers tied to the UART channel for any problems that DMA could have encountered.

[robotwhisperer]

I have disabled the MPU completely from the IOC File, regenerated code, and there is no change in the behaviour, which suggests that the previous setup did not allow the MPU to do anything weird with the buffers to begin with. 

I also noticed that when I debug the code and insert breakpoints in the UART_RxCpltCallback and watch the variable of interest, the sensor_data_raw, it updates just fine from one iteration to the next. 

[robotwhisperer]

Make sure DMA buffer mode is correctly set, such as circular buffer or normal. Set FIFO Mode to "Disable",if enabled. Set Data Width for both Peripheral and Memory to be Byte and Byte for debugging.

I checked the DMA Configuration and it is already how you described in the IOC file. 

If you are using the circular buffer for DMA, data in the buffer needs to be indexed appropriately using the DMA DMA_SxNDT register.

I am not using circular DMA buffer for this.

You can also look at various DMA registers tied to the UART channel for any problems that DMA could have encountered.

---

I disabled all other DMA streams except this one that I am interested in, and the behaviour has not changed. Could you elaborate more on checking the UART DMA registers to check the state of the DMA and if it has encountered any problems? 

[robotwhisperer]

In cases like this, where the code for the project is very large with many different files, and importantly, proprietary and needing to be kept confidential, what is the best course of action to get a solution while not disclosing all my code? 

[TDK]

The best would be to replicate the issue in a minimal working example and share that here.

Minimal reproducible example - Wikipedia

Barring that, you could send a support ticket to ST and they may be able to assist.

https://ols.st.com/s/

Or you could pay someone to assist. Hire a consultant and have them sign an NDA.

Unfortunately, unless you share the piece of the code with the bug in it, there's not much help that can be provided here. If it's a silicon error, it should be replicable in a MWE.

[robotwhisperer · Accepted solution]

I fixed it. I disabled the MPU on the Transmit side, the issue was never where I was looking all along. Thank you everyone for your help. 

[BGeor.1]

Assuming you are using DMA, not MDMA or BDMA, following register contents should be reviewed for errors.

• DMA low interrupt status register (DMA_LISR),  DMA high interrupt status register (DMA_HISR):
  • This register can flag transfer errors. It will be a good idea to check these registers from within UART_RxCpltCallback since these fields may get cleared across DMA transfers.
• DMA stream x configuration register (DMA_SxCR)
  • Verify to make sure configuration is correct for the specific stream
• DMA_SxNDTR register
  • Even though you are not using circular buffer, I think SxNDTR should still get updated after every receive with the number of bytes transferred from UART to memory.

You can also review other related registers for the specific DMA stream.

You may also want to implement HAL_UART_ErrorCallback() to get notified on any errors.

I am not sure whether it will be an option for you. But it may also be a good idea to change DMA buffer to circular buffer. With circular buffer, up to 64 KB max, you get to see multiple packets in DMA buffer that has been received at the same time up to 64KB. With normal buffer instead of circular buffer, each receive over-writes the same DMA buffer, so it is little difficult to debug. Also, try to characterize data loss using some sort of sequence number.

If not using circular buffer, you need to make sure UART_RxCpltCallback() completes quickly to keep up with incoming data from the board that is sending the data. Most probably, only thing you want to do from UART_RxCpltCallback() is a buffer copy.