how to sync a uart dma receive for gps parsing

[Lgarb.1]

Hi, I'm working with a stm32f722 and a ublox f9p gps. I just have 2 ubx messages. One with 36 bytes long and the other with 72 bytes. They income with a variable interval between them (a couple of byte time at most, usually they come together). This two messages are coming at a 5Hz data rate.

So I've tried several ways to parse them and for one reason or another I can make it work.

My goal is to parse the 2 messages as soon as they are available.

My first guess was to config a circular DMA between uart and memory. On Half complete CB ISR I take the first 54 bytes and pass trough ubx state machine parser. If the 72 bytes long msg is coming first the parser will not get any valid msg, but on Rx Complete CB the other 54 bytes will be send to SM and I will obtain the 2 messages. If the first msg is the 36 byte long, then I will get a valid msg on HC CB and the other on TC CB. That's will be ok.

The problem here is that I need to sync the DMA Rx call to the data stream. If I call DMA Rx in the middle of the stream I will process some bytes of current frame and some bytes of the next frame.

So, does anybody suggest a way to make that synchronization? I've think on a timer that interrupt at 20 msec. At the beginning each byte received will reset the timer counter. On the timer ISR I've set a flag indicating that the blank part of the frame is reach, then abort the DMA transfer and start a new DMA Rx request. The problem here is that for some reason it not sync. I've noticed that some overrun is happening and in order to get the uart working again I need to deinit and reinit the uart and the sync process fail.

uint8_t rxbuff[108]={0};
ublox_t ublox;
 
uint8_t ubx_msgids[100]={0};
uint8_t idx=0;
 
uint8_t f_is_sync = 0;
 
void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart)
{
  if(huart->Instance == USART1)		//GNSS
  {
	  if (!f_is_sync){
		  __HAL_TIM_SET_COUNTER(&htim14, 0);
	  }else{
		HAL_GPIO_WritePin(GP2_GPIO_Port, GP2_Pin, GPIO_PIN_RESET);
	  }
  }
}
void HAL_UART_RxHalfCpltCallback(UART_HandleTypeDef *huart)
{
  if(huart->Instance == USART1)		//GNSS
  {
	  if (!f_is_sync){
		  __HAL_TIM_SET_COUNTER(&htim14, 0);
	  }else{
		HAL_GPIO_WritePin(GP2_GPIO_Port, GP2_Pin, GPIO_PIN_SET);
	  }
  }
}
void HAL_UART_ErrorCallback(UART_HandleTypeDef *huart){
	if (huart->Instance == USART1)  // ECT-400 UART
	{
	    HAL_GPIO_WritePin(LED_G_GPIO_Port, LED_G_Pin, GPIO_PIN_RESET);
	    __HAL_TIM_SET_COUNTER(&htim14, 0);
            HAL_UART_DeInit(&huart1);
            MX_USART1_UART_Init(); //my initialization code
	    __HAL_TIM_SET_COUNTER(&htim14, 0);
	    HAL_UART_Receive_DMA(&huart1, rxbuff, 108);
            HAL_GPIO_WritePin(LED_G_GPIO_Port, LED_G_Pin, GPIO_PIN_SET);
	}
 
}
void HAL_TIM_PeriodElapsedCallback(TIM_HandleTypeDef *htim)
{
	if (htim->Instance == TIM14){
	    HAL_GPIO_TogglePin(GP1_GPIO_Port, GP1_Pin);
		HAL_TIM_Base_Stop_IT(&htim14);
		HAL_DMA_Abort(&hdma_usart1_rx);
		f_is_sync = 1;
		HAL_UART_Receive_DMA(&huart1, rxbuff, 108);
	}
 
}
 
int main(void)
{
  HAL_Init();
  SystemClock_Config();
  MX_GPIO_Init();
  MX_USART1_UART_Init();
  MX_DMA_Init();
  MX_TIM14_Init();
  ublox_init(&ublox);
 
  HAL_UART_Receive_DMA(&huart1, rxbuff, 108);
 
  __HAL_TIM_CLEAR_IT(&htim14, TIM_IT_UPDATE);
  __HAL_TIM_SET_COUNTER(&htim14, 0);
  HAL_GPIO_TogglePin(GP1_GPIO_Port, GP1_Pin);
  HAL_TIM_Base_Start_IT(&htim14);
 
  while (1);
}

This is part of the code. The GPIO set and reset are for debug with an oscilloscope what is happening.

 EDIT:

I'm testing this code (it was working on old HAL library)

    uint8_t sync=1;
	// Rutina para sincronizar el comienzo del mensaje
	while(sync)
	{
		// Read two bytes
		HAL_UART_Receive(&huart1, rxbuff, 2, 200);
		// If they are SYNC bytes
		if(rxbuff[0]==0xB5 && rxbuff[1]==0x62)
		{
			// Receive remaining 106 bytes
			HAL_UART_Receive(&huart1, &rxbuff[2], 106, 10);
			// We are in sync with GPS 
			sync=0;
		}
	}
	// Ask for 108 bytes in DMA mode
	HAL_UART_Receive_DMA(&huart1, rxbuff, 108);

The problem here is that the call to HAL_UART_Receive_DMA produce an Overrun error and DMA process is not working at all.

[Tesla DeLorean]

>> I've noticed that some overrun is happening and in order to get the uart working again I need to deinit and reinit the uart and the sync process fail.

Are you sure it is not flagging some noise, framing or other reception error condition you need to clear from the UART?

Doing fixed lengths does tend to make you prone to synchronization errors, or situations when you get additional messages, etc.

I'm parsing a pair of ZED-F9 modules on a 32 MHz STM32L072CZ so the F722 should have plenty of horsepower, although I'm not using the HAL for anything other than initialization. My UART approach is to simply collect and buffer the data, other tasks process/digest that data.

https://portal.u-blox.com/s/question/0D52p0000BrDqbPCQS/time-between-two-consecutive-ubx-messages

[Lgarb.1]

Thank for your answer. In fact I was adding this code:

void HAL_UART_ErrorCallback(UART_HandleTypeDef *huart){
	if (huart->Instance == USART1)  // ECT-400 UART
	{
		HAL_GPIO_WritePin(LED_G_GPIO_Port, LED_G_Pin, GPIO_PIN_RESET);
	}

Just to confirm that there is an error on the uart. If I put a break point on HAL_GPIO_... I can see that huart->ErrorCode is equal to 8, that is OVERRUN.

And the ErrorCallback is called as soon as HAL_UART_Receive_DMA(&huart1, rxbuff, 108); is called

I was trying reading byte to byte and storing on some buffer, byte to bye, but then I couldn't find a way to detect that I reach the end of the frame (calling a frame the 2 messages I've enabled) in order to process the data. I'm not using an rtos, so the only thing I can imagine is a timer with a 2 msg time timeout, on each received byte I reset the counter, when the timer reach it's time then I can expect that I have 200 msec to process the buffer.

I need a way to know when the end of the second message is complete on the buffer in order to start decoding the data.

[TDK]

I would set up a circular buffer and have DMA constantly receive into it. Poll the buffer periodically to detect new data using the DMA's NDTR register. I seem to recall uBlox has beginning and end of line characters, so ignore character until you see the start, then wait until the end characters are present and read out your message. Keep a head/tail pointer so you know what bytes have been processed and which are new.

You can code up something with the HAL stuff, but having UART constantly-on is a much more robust solution.

[Lgarb.1]

"Poll the buffer periodically to detect new data using the DMA's NDTR register"

The periodically part is my problem. I need to parse the messages as soon as possible. While data are incoming there is no free time to parse. So I need to detect the end of the messages frame.

I finally use a timer with 2msec timeout, resetting the counter on each received byte and parse de messages on timer timeout interrupt. The only issue with this is the variable time gap between 2 consecutive messages, ie NAV-RELPOSNED and NAV_HPPPOSECEF.

It's not a perfect solution, but I've never seen an implementation of this kind.

[TDK]

The UART IDLE interrupt can be used to detect and trigger at the exact end of the data.

[Lgarb.1]

yes, but the time gap between 2 consecutive messages is variable. Some times they are one next to the other and some times they are several byte time apart. I was trying to use IDLE interrupt to parse each messages as soon as they are on the receive buffer, but some times I have the time to parse the first message (On IDLE ISR) and some times I lost some bytes of the second message.

I'm really lost of how to implement this on a robust manner without a RTOS.

[TDK]

These are all solvable issues. If the IDLE interrupt happens and you don't yet have all the bytes, just return and check again on the next IDLE event.
If you are dead set on only generating an interrupt after exactly N bytes, and you have no guarantees on the data stream, and you have very tight timing requirements, I don't see a way of getting around direct register access using DMA. It is possible to mix HAL and direct register access, however the set of programmers familiar with both of those is limited. And the subset willing to dive into your program to determine the details of how to do this may be the null set.

[Lgarb.1]

Once the 2 consecutive messages are buffered, there is 200msec to the next couple of messages. So, there is time to process the messages. The problem is how to determine the end of the last message (on this frame) in order to process the them.

Let's say the DMA start reading data in the middle of a msg of the current frame. On IDLE ISR I need to know if it is the first msg or the second one. If the bytes on the buffer correspond to the first message I will not have time to process the buffer (in the IDLE ISR) and need to wait until the next IDLE IRQ, because the bytes of the second msg will be streamed in a short period of time (a couple of byte time). If the bytes on the buffer are from de second message, then there is no problem, I have 200msec to process them and decide if I discard them (because they don't contain a complete message), or parse a portion of them.

So, the problem is, I think, detect the end of the second message.

Using a RTOS I can imagine 2 task, one with high priority that read data from the uart and put into a buffer and one with low priority that consume data. While receiving bytes from the first message the first task will be blocking the second one. When the first msg end the uart will be free and this task will be sleeping, so the low priority task will have time to execute. It will have the CPU until the next message arrive (it could be 0 usec or could be almost 1 msec depending on the variable time between 2 messages of the current frame) During this time the task could process some bytes or none of them. Then the next msg will be streamed and the first task will take the control again.

But I'm not sure how to do that without a RTOS