UART DMA Memory Corruption during Buffer Wrap-Around

[lgacnik97]

I'm using HAL UART Receive with DMA (configured using CubeMX). I'm encountering a problem where data via UART, send in chunks, is corrupted if a chunk doesn't fit at the end of buffer perfectly. By chunks I mean a set of byte-sized data that is transmitted one following another in immediate manner.

I'm using a simple circular buffer struct where DMA is writing to. I check the CNDTR DMA Channel register to update "head" index of this software buffer. The only process that writes to this buffer is the DMA channel assigned to UART RX.

The following code shows how DMA and UART is configured from CubeMX:

/**
  * @brief USART1 Initialization Function
  * @PAram None
  * @retval None
  */
static void MX_USART1_UART_Init(void)
{
  huart1.Instance = USART1;
  huart1.Init.BaudRate = 115200;
  huart1.Init.WordLength = UART_WORDLENGTH_8B;
  huart1.Init.StopBits = UART_STOPBITS_1;
  huart1.Init.Parity = UART_PARITY_NONE;
  huart1.Init.Mode = UART_MODE_TX_RX;
  huart1.Init.HwFlowCtl = UART_HWCONTROL_NONE;
  huart1.Init.OverSampling = UART_OVERSAMPLING_16;
  if (HAL_UART_Init(&huart1) != HAL_OK)
  {
    Error_Handler();
  }
}

/**
  * Enable DMA controller clock
  */
static void MX_DMA_Init(void)
{
  /* DMA controller clock enable */
  __HAL_RCC_DMA1_CLK_ENABLE();

  /* DMA interrupt init */
  /* DMA1_Channel1_IRQn interrupt configuration */
  HAL_NVIC_SetPriority(DMA1_Channel1_IRQn, 5, 0);
  HAL_NVIC_EnableIRQ(DMA1_Channel1_IRQn);
  /* DMA1_Channel5_IRQn interrupt configuration */
  HAL_NVIC_SetPriority(DMA1_Channel5_IRQn, 5, 0);
  HAL_NVIC_EnableIRQ(DMA1_Channel5_IRQn);
}

The following code shows how DMA UART channel is used:

HAL_UART_Receive_DMA(monHuart, (uint8_t *)rxCirBuff.buff, MONRX_CIRBUFF_SIZE);
while (1) {
  rxCirBuff.head = MONRX_CIRBUFF_SIZE - __HAL_DMA_GET_COUNTER(monHuart->hdmarx);
}

The following two images show buffer state as it is loaded with F1 key codes delivered continuously by external keyboard. The valid chunk of bytes is as follows (decimal values): 27 -> 91 -> 49 -> 49 -> 126. The first image shows the buffer state before the last chunk arrives:

The second image shows the buffer state after the last chunk arrives, when buffer wrap-around happens. Note that now 30th byte is overwritten (probably) with the byte 91, however it should be 27 as for all other chunks. Note that DMA channel properly updates its counter index and the rest of the bytes are written properly without being corrupted.

Additionally note that this happens spontaneously, meaning sometimes the last chunk, being split into two parts, comes out valid. What could be the issue?

[waclawek.jan]

There's apparently lots of code we don't see (at least the while(1) is suspicious).

Prepare a minimal (i.e. only UART and DMA) but complete compilable code which reproduces the problem, and post.

JW

[Tesla DeLorean]

The HAL implementation here is honestly a mess, can be done much more simply.

The DMA can interrupt at HT (Half Transfer) and TC (Transfer Complete). The buffer can be quite large.

In a large buffer, continuous mode, the ring can be swept periodically for content into deeper buffers, and processed. The depth of the ring buffer then determined by the peak data rate, and processing interval.

[Karl Yamashita]

Are your chunks of data always 5 bytes? Are each chunks being parse separately? 

If so, you can use HAL_UARTEx_ReceiveToIdle_DMA and HAL_UARTEx_RxEventCallback

[lgacnik97]

I'll see what I can do but that's really pretty much it as far the DMA and UART is concerned. Before DMA I was trying out a setup with UART IT, however at certain UART baud rate sometimes bytes were not caught at proper times by the software. Also, it was a 1-byte based reception where ISR callback would trigger per byte. This would be completely fine for pressing normal keys, however when pressing special function keys (e.g. F1) where a sequence of 1-byte characters is send, this would randomly cause issues occasionally.

Meanwhile DMA seems to catch all bytes at all times and high baud rates. Except there seems to be some issue when DMA tries to wrap-around in a circular buffer. And that's what I'm trying to figure out what's the issue.

I will get back to you with a compile-able example on Monday.

[lgacnik97]

The HAL implementation here is honestly a mess, can be done much more simply.

---

Unless HAL configures DMA in some inappropriate manner I don't think HAL is to be blamed in this case. As you see from my question I'm only using CNDTR register to check specific channel's current count and update my own buffer's state (DMA is writing to buffer, head index updated periodically, as well as the tail index is updated where the buffer is read). Besides this only HAL's DMA start and CubeMX DMA configure is used.

The problem with half-complete and full-complete buffer in this case is that I'm trying to read (parse) incoming characters (from terminal) as soon as possible and waiting for buffer half full or completely full would not suit my needs.

The only reason I could use TC (Transfer Complete) would be to check if my buffer's head wrapped around however that is already done manually each time head is updated.

[lgacnik97]

No, not always. As mentioned above I'm using DMA with UART to effectively catch all the bytes arriving from serial terminal. Most of the time received chars are 1-byte per key press. However, sometimes when pressing special function keys (e.g. F1 key) a multi-byte code is send by terminal. As mentioned above I would initially use UART IT (without DMA), however 1-byte continuous receptions (ISR triggered for each byte) would cause certain problems at higher baud rates and sometimes bytes would not be caught on time - therefore timing issues.

However this is not the case with DMA as it seems it smoothly handles storing of incoming bytes into a user-defined buffer. Whereas I'm only updating circular buffer's head index (for later processing) by checking DMA channel's CNDTR register. This register seems to be properly updated but DMA writes to user-defined buffer seem to get corrupted somehow sometimes when DMA circular buffer is about to wrap-around.

I will test the proposed callbacks on Monday and get back to you with some results.

[Karl Yamashita]

If it is key presses then there will be some idle time. Unless you can type like Superman and the key presses streams with no idle time. You can use HAL_UARTEx_ReceiveToIdle_DMA even if its a special multi-byte key. Just need to know what is the maximum amount of bytes that a special key will send? 

Based off that number, you can follow this tutorial https://github.com/karlyamashita/Nucleo-G431RB_Three_UART/wiki

If you're using UART_Receive_IT and you're missing bytes, then more than likely you're doing something time consuming in the callback while incoming bytes are overflowing. Show your code so we can dissect it.

[lgacnik97]

I was able to solve this issue by figuring out I was still using "TransferComplete()" UART IT callback which contained code from previous UART reception setup that used IT only without DMA. After releasing that callback DMA writes during wrap-around event executed smoothly without issues.

[waclawek.jan]

Thanks for coming back with the solution. Please click on "Accept as solution" in that post so that the thread is marked as solved.

JW