Why does my UART receive buffer becomes missaligned after first receive?????

[CBarb.1]

I am working with a Nextion display and a STM32F411CEx on STM32 Cube Ide I need to both send and receive serial commands to the display at 9600 baud with standard serial port settings. I have no problems sending data to the display but I seem to have issues receiving data from the display like a button press or a response from some other GUI object on the display.
The display sends a serial stream that contains five bytes as such: 0x24 0x30 0x01 0x01 0x26 The first and last byte are delimiters where 0x24 = ascii '$' and 0x26 = ascii '&' the intermidiate bytes are for object ID, page ID and object value. So I have a five byte buffer for the UART rceive buffer, I am using UART1 on interrupt mode for receiving data

below is a typical serial buffer Rx_Data[5] receive sequence from pressing a button on the display:

Rx_Data[0] = 0x24
Rx_Data[1] = 0x30
Rx_Data[2] = 0x01
Rx_Data[3] = 0x01
Rx_Data[4] = 0x26

The above is the correct sequence and when the data comes in this way everything works just fine, however after the first data frame is sent the data alignment goes out of whack
as you can see below and when this happens I can no longer decode the data propperly

Rx_Data[0] = 0x26
Rx_Data[1] = 0x24
Rx_Data[2] = 0x30
Rx_Data[3] = 0x01
Rx_Data[4] = 0x01

As you can see above the last byte of the buffer is now coming in where the first byte should be and everything is now shifted by one byte. It appears that once this happens it does not
continue to keep shifting but in the long term I can't be sure of that. I can prpbably fix the above by examining the buffer contents and re-aligning the data prior to decoding but that is a patch solution as I need to know why this is happening???? I have observed the serial data coming out of the display on a logic analyzer and the data is always framed correctly so this is a problem on my UART receive system..........Any suggestions will be greatly appreciated!!!

//////////// Some of my relevant code is below ///////////////////////////////////////

/* Macro to define display decoding sequence */
#define READY_BUTTON (Rx_Data[0] == 0x24 && Rx_Data[1] == 0x30 && Rx_Data[2] == 0x01 && Rx_Data[3] == 0x01 && Rx_Data[4] == 0x26) // Page0 Ready Button Pressed

uint8_t Rx_Data[5]; // my receive buffer data

/**********************************************************
* Receive serial data via interrupts from Nextion display
* Interrupt handler callback
**********************************************************/
void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart)
{

if(READY_BUTTON || READY_BUTTON1)
{ RDY_BUTTON_FLAG = 1; START_BUTTON_FLAG = 0;}

if(START_BUTTON || START_BUTTON1)
{ RDY_BUTTON_FLAG = 0; START_BUTTON_FLAG = 1;}

//memset(&Rx_Data[0],0,sizeof(Rx_Data)); // tried clearing the receive buffer, did not help

HAL_UART_Receive_IT(&huart1, Rx_Data, 5);

}

/* Initialize all configured peripherals in main.c */
MX_GPIO_Init();
MX_ADC1_Init();
MX_I2C1_Init();
MX_TIM2_Init();
MX_TIM3_Init();
MX_USART1_UART_Init();
MX_USB_DEVICE_Init();
MX_TIM4_Init();
HAL_UART_Receive_IT(&huart1, Rx_Data, 5); // Display serial on interrupt must be placed here otherwise there will be no Rx interrupt
HAL_TIM_Base_Start(&htim2); //// Start timer2
HAL_TIM_PWM_Start(&htim3,TIM_CHANNEL_3); //Initialize PWM on timer3 CH3 with 16bit res ~1.11KHz
HAL_TIM_Base_Start_IT(&htim4); //// Start timer4 with interrupt

[waclawek.jan]

Some lengthy processing (in the parts of program we don't see), initiated by some of the flags set in the ISR, involving interrupts being disabled for too long, leading to UART overruns?

JW

[David Littell]

Yes, the "Display serial on interrupt must be placed here otherwise there will be no Rx interrupt" comment is a clear indication that there are interrupt priority or interrupts-off-too-long problems.  At 9600 baud you have about 1 mSec per character, so if the thing is unable to handle that it's rather seriously balled up somewhere.

Haha, it's always fun to see "relevant code" - the real problem is almost invariably elsewhere!  :face_screaming_in_fear:

[Karl Yamashita]

As @waclawek.jan  and @David Littell indicated, there are other parts of the source code that you are not showing which is causing your data misalignment.

You're better off using HAL_UARTEx_ReceiveToIdle_IT and HAL_UARTEx_RxEventCallback. Save the data to a larger byte ring buffer as you get an interrupt. If you know you only have 5 bytes to receive then set HAL_UARTEx_ReceiveToIdle_IT  for 10 bytes. It has half callback interrupt so it'll interrupt at 5 bytes. 

If you missed some bytes, it'll still interrupt between packets and you can check the size is 5. If not, then don't save the data. Then in you main while loop, parse through your larger buffer and extract each packet using your delimiters and size of 5. If those don't match then you know you have a bad packet.