Trying to receive large unknown size data over UART (over 1MB)

[Victorious]

So I am new to STM32 , what I want to implement is to receive large data(preferably over 1MB) which is greater than the size of receive buffer of UART so how should I proceed with it. Since the data I am receiving is in the form of hex, I also have to program the flash using the same data.

[Andrew Neil]

welcome to the forum.

Please see How to write your question to maximize your chances to find a solution for best results.

---

@Victorious wrote:

I also have to program the flash using the same data.

---

So are you trying to implement in-application firmware update (IAP)?

If so, see Application note AN4657, STM32 in-application programming (IAP) using the USART.

The general principle is that you receive a block of data, program that to Flash, then receive the next block, etc, etc,...

So you need some kind of "protocol" to tell the sender when the receiver is ready for the next block.

The above-mentioned App Note uses YMODEM; other protocols, of course, are available.

PS:

If firmware update is the requirement here, note that you can also use the built-in bootloader - then you don't have to write any code yourself at all!

See AN2606, Introduction to system memory boot mode on STM32 MCUs.

[TDK]

Data is sent byte by byte. When you receive some amount of bytes, do what you want with it and be ready to receive more.

Putting the UART in circular receive mode with DMA will generally be best here. Use the half-transfer and full-transfer complete callbacks to process those halves of the buffer.

Ensure programming can complete before the next half of the buffer is ready. Look at the data sheet for flash programming times and use the UART baud rate to calculate how long you have.

[Victorious]

I dont need more bytes after I have received the unknown number of bytes once(lets say in this case 1MB). You could say I am uploading Intel Hex File whose size would differ depending upon the application.

[Victorious]

Yes, I am trying to implement IAP. I did make my bootloader but it works only for file sizes that are small. I have heard about different techniques like ring buffer, double buffers etc to counter this problem but I have problem implementing them.

[Victorious]

/* USER CODE BEGIN Header */
/**
  ******************************************************************************
  * @file           : main.c
  * @brief          : Main program body
  ******************************************************************************
  * @attention
  *
  * Copyright (c) 2025 STMicroelectronics.
  * All rights reserved.
  *
  * This software is licensed under terms that can be found in the LICENSE file
  * in the root directory of this software component.
  * If no LICENSE file comes with this software, it is provided AS-IS.
  *
  ******************************************************************************
  */
/* USER CODE END Header */
/* Includes ------------------------------------------------------------------*/
#include "main.h"

/* Private includes ----------------------------------------------------------*/
/* USER CODE BEGIN Includes */
#include <string.h>
#include <stdio.h>
/* USER CODE END Includes */

/* Private typedef -----------------------------------------------------------*/
/* USER CODE BEGIN PTD */

/* USER CODE END PTD */

/* Private define ------------------------------------------------------------*/
/* USER CODE BEGIN PD */

/* USER CODE END PD */

/* Private macro -------------------------------------------------------------*/
/* USER CODE BEGIN PM */

/* USER CODE END PM */

/* Private variables ---------------------------------------------------------*/
UART_HandleTypeDef huart1;
DMA_HandleTypeDef hdma_usart1_rx;

/* USER CODE BEGIN PV */

/* USER CODE END PV */

/* Private function prototypes -----------------------------------------------*/
void SystemClock_Config(void);
static void MX_GPIO_Init(void);
static void MX_DMA_Init(void);
static void MX_USART1_UART_Init(void);
/* USER CODE BEGIN PFP */
#define RxBuf_SIZE   1024

static int count=0;
uint8_t RxBuf[RxBuf_SIZE];
  uint32_t dat32=0;

uint32_t static address=0x08040000;
int isOK = 0;


void HAL_UARTEx_RxEventCallback(UART_HandleTypeDef *huart, uint16_t Size)
{


if(count==0){
  HAL_FLASH_Unlock();
  FLASH_Erase_Sector(6,2.7);
  HAL_FLASH_Lock();
}

  HAL_FLASH_Unlock();
           printf("Programming of the Flash Starting.\n");
           for (int i = 0; i<=1024; i += 4) {

               dat32 =  (uint32_t)RxBuf[i] | ((uint32_t)RxBuf[i+1] << 8)| ((uint32_t)RxBuf[i+2] << 16)| ((uint32_t)RxBuf[i+3] << 24);

               HAL_FLASH_Program(FLASH_TYPEPROGRAM_WORD, address, dat32);

               address += 4;
           }
      HAL_FLASH_Lock();
      count++;
/* start the DMA again */
HAL_UARTEx_ReceiveToIdle_DMA(&huart1, (uint8_t *) RxBuf, RxBuf_SIZE);
__HAL_DMA_DISABLE_IT(&hdma_usart1_rx, DMA_IT_HT);



}

/* USER CODE END PFP */

/* Private user code ---------------------------------------------------------*/
/* USER CODE BEGIN 0 */

/* USER CODE END 0 */

/**
  * @brief  The application entry point.
  * @retval int
  */
int main(void)
{

  /* USER CODE BEGIN 1 */

  /* USER CODE END 1 */

  /* MCU Configuration--------------------------------------------------------*/

  /* Reset of all peripherals, Initializes the Flash interface and the Systick. */
  HAL_Init();

  /* USER CODE BEGIN Init */

  /* USER CODE END Init */

  /* Configure the system clock */
  SystemClock_Config();

  /* USER CODE BEGIN SysInit */

  /* USER CODE END SysInit */

  /* Initialize all configured peripherals */
  MX_GPIO_Init();
  MX_DMA_Init();
  MX_USART1_UART_Init();
  /* USER CODE BEGIN 2 */


HAL_UARTEx_ReceiveToIdle_DMA(&huart1, RxBuf,RxBuf_SIZE);
__HAL_DMA_DISABLE_IT(&hdma_usart1_rx,DMA_IT_HT);
  /* USER CODE END 2 */

  /* Infinite loop */
  /* USER CODE BEGIN WHILE */
  while (1)
  {
    /* USER CODE END WHILE */

    /* USER CODE BEGIN 3 */
  }
  /* USER CODE END 3 */
}

/**
  * @brief System Clock Configuration
  * @retval None
  */
void SystemClock_Config(void)
{
  RCC_OscInitTypeDef RCC_OscInitStruct = {0};
  RCC_ClkInitTypeDef RCC_ClkInitStruct = {0};

  /** Configure the main internal regulator output voltage
  */
  __HAL_RCC_PWR_CLK_ENABLE();
  __HAL_PWR_VOLTAGESCALING_CONFIG(PWR_REGULATOR_VOLTAGE_SCALE3);

  /** Initializes the RCC Oscillators according to the specified parameters
  * in the RCC_OscInitTypeDef structure.
  */
  RCC_OscInitStruct.OscillatorType = RCC_OSCILLATORTYPE_HSI;
  RCC_OscInitStruct.HSIState = RCC_HSI_ON;
  RCC_OscInitStruct.HSICalibrationValue = RCC_HSICALIBRATION_DEFAULT;
  RCC_OscInitStruct.PLL.PLLState = RCC_PLL_NONE;
  if (HAL_RCC_OscConfig(&RCC_OscInitStruct) != HAL_OK)
  {
    Error_Handler();
  }

  /** Initializes the CPU, AHB and APB buses clocks
  */
  RCC_ClkInitStruct.ClockType = RCC_CLOCKTYPE_HCLK|RCC_CLOCKTYPE_SYSCLK
                              |RCC_CLOCKTYPE_PCLK1|RCC_CLOCKTYPE_PCLK2;
  RCC_ClkInitStruct.SYSCLKSource = RCC_SYSCLKSOURCE_HSI;
  RCC_ClkInitStruct.AHBCLKDivider = RCC_SYSCLK_DIV1;
  RCC_ClkInitStruct.APB1CLKDivider = RCC_HCLK_DIV1;
  RCC_ClkInitStruct.APB2CLKDivider = RCC_HCLK_DIV1;

  if (HAL_RCC_ClockConfig(&RCC_ClkInitStruct, FLASH_LATENCY_0) != HAL_OK)
  {
    Error_Handler();
  }
}

/**
  * @brief USART1 Initialization Function
  * @PAram None
  * @retval None
  */
static void MX_USART1_UART_Init(void)
{

  /* USER CODE BEGIN USART1_Init 0 */

  /* USER CODE END USART1_Init 0 */

  /* USER CODE BEGIN USART1_Init 1 */

  /* USER CODE END USART1_Init 1 */
  huart1.Instance = USART1;
  huart1.Init.BaudRate = 115200;
  huart1.Init.WordLength = UART_WORDLENGTH_8B;
  huart1.Init.StopBits = UART_STOPBITS_1;
  huart1.Init.Parity = UART_PARITY_NONE;
  huart1.Init.Mode = UART_MODE_TX_RX;
  huart1.Init.HwFlowCtl = UART_HWCONTROL_NONE;
  huart1.Init.OverSampling = UART_OVERSAMPLING_16;
  if (HAL_UART_Init(&huart1) != HAL_OK)
  {
    Error_Handler();
  }
  /* USER CODE BEGIN USART1_Init 2 */

  /* USER CODE END USART1_Init 2 */

}

/**
  * Enable DMA controller clock
  */
static void MX_DMA_Init(void)
{

  /* DMA controller clock enable */
  __HAL_RCC_DMA2_CLK_ENABLE();

  /* DMA interrupt init */
  /* DMA2_Stream2_IRQn interrupt configuration */
  HAL_NVIC_SetPriority(DMA2_Stream2_IRQn, 0, 0);
  HAL_NVIC_EnableIRQ(DMA2_Stream2_IRQn);

}

/**
  * @brief GPIO Initialization Function
  * @PAram None
  * @retval None
  */
static void MX_GPIO_Init(void)
{
  /* USER CODE BEGIN MX_GPIO_Init_1 */

  /* USER CODE END MX_GPIO_Init_1 */

  /* GPIO Ports Clock Enable */
  __HAL_RCC_GPIOA_CLK_ENABLE();

  /* USER CODE BEGIN MX_GPIO_Init_2 */

  /* USER CODE END MX_GPIO_Init_2 */
}

/* USER CODE BEGIN 4 */

/* USER CODE END 4 */

/**
  * @brief  This function is executed in case of error occurrence.
  * @retval None
  */
void Error_Handler(void)
{
  /* USER CODE BEGIN Error_Handler_Debug */
  /* User can add his own implementation to report the HAL error return state */
  __disable_irq();
  while (1)
  {
  }
  /* USER CODE END Error_Handler_Debug */
}
#ifdef USE_FULL_ASSERT
/**
  * @brief  Reports the name of the source file and the source line number
  *         where the assert_param error has occurred.
  * @PAram  file: pointer to the source file name
  * @PAram  line: assert_param error line source number
  * @retval None
  */
void assert_failed(uint8_t *file, uint32_t line)
{
  /* USER CODE BEGIN 6 */
  /* User can add his own implementation to report the file name and line number,
     ex: printf("Wrong parameters value: file %s on line %d\r\n", file, line) */
  /* USER CODE END 6 */
}
#endif /* USE_FULL_ASSERT */

I am using this code but it is not working

---

Edited to apply source code formatting - please see How to insert source code for future reference.

It was also explained in the  How to write your question to maximize your chances to find a solution article, mentioned earlier.

[Ozone]

Fact is, you don't have enough resources (RAM) do receive the complete application firmware, so you need to do it piece by piece.

A complete line of a Hex or S19 file for instance would be such a unit.
Complicating side conditions are, you can erase Flash only per sector, and code execution from the same bank stalls while an erase/program is ongoing.

But that aside, you would need to coordinate both sides, your bootloader and the host application to perform the update. An erase can take dozens or hundreds of milliseconds, flashing a short sequence takes several milliseconds. You might not be able to receive new data at that time, so you need to implement a stop/go mechanism.

And you might choose to erase the whole (main application) Flash , or erase sectors ad hoc when you need to reflash them.

And perhaps add a plausibility check to avoid killing the bootloader, persistent data, or accessing non-existent Flash.

[Andrew Neil]

---

@Victorious wrote:
I am using this code but it is not working

---

So what have you done to debug it ?

What, exactly, does "not working" mean?

How To Debug.

[Andrew Neil]

---

@Victorious wrote:

Yes, I am trying to implement IAP. I did make my bootloader but it works only for file sizes that are small.

---

So take a look at that Application Note - it works for any size file.

---

@Victorious wrote:

I don't need more bytes after I have received the unknown number of bytes once.

---

That's not the point.

As @Ozone said, your STM32 does not have enough resources to do the whole thing in one go - so you have to do it in parts.

Again,  AN4657 shows you a working way to do this.

---

@Victorious wrote:

You could say I am uploading Intel Hex File whose size would differ depending upon the application.

---

Yes, AN4657 shows how to do that.

Note that handling Intel Hex is non-trivial - a straight binary is far easier.