Summary
This article provides a step-by-step description of a lightweight algorithm to write and retrieve data from flash using virtual addresses. The key aspect is the low footprint and overall simplicity. The code example in this guide is provided using the NUCLEO-C031C6 board, but can be tailored to any other STM32.
The code sample is attached as a .zip file at the bottom of this article.
Introduction
The storage of data is essential for multiple applications in the embedded world, for example: storage of calibration data for sensors and other devices and data logging. However, especially in low-cost applications, additional data storage components, like an external EEPROM, may increase costs or board size beyond the desired range. For this situation, taking advantage of unused flash memory pages as storage to organize low-cycle and nonvolatile data may be a viable solution.
While the official package, X-CUBE-EEPROM1 can be used, the memory footprint used by this middleware can be overwhelming for low-density MCUs. Currently, the memory footprint for the STM32C0 series is 5098 bytes of flash using the IAR EWARM 8.42.2 (or later) tool with a high size optimization level. Using the same settings, the proposed algorithm requires 864 bytes of flash memory for code and 16 bytes of RAM for data
1. Development
The main technical challenges of using flash memory as a data storage are:
- Flash memory can only be programed 64 bits at a time (128 bits for STM32U5 and STM32H5 series, or 16 bits for STM32H5 series using high-cycle data area), due to its Error Correcting Code (ECC) that cannot be switched off.
- The target address can only be written if it has been previously cleared (0XFFFFFFFF).
- A specific target address cannot be independently cleared, at minimum an entire page (2 KB in C031) must be erased.
- Locating the written data inside from flash memory.
The next topics will dive in how the proposed algorithm attempts to circumvent these challenges to create a simple and lightweight flash memory organization algorithm.
1.1 Getting around the ECC
To get around the 64-bit restriction, the proposed algorithm implements multiple functions that compact different sizes of variables in a 32-bit block. For example, combining two 16-bit numbers into one 32-bit variable. The remaining 32 bits are used to store the organization data, important for verification and information retrieval.
The complete organization of the 64-bit block is described below:
- The first 32 bits are used to store the saved data (DATA).
- The following 16 bits stores a virtual address number that can be used to retrieve the data from the block (VIRTUAL ADDRESS).
- The next 8 bits are empty (PLACEHOLDER).
- The last 8 bits are divided into two functions:
- The initial 4-bits space is used for the algorithm to interpret the placement of variables in memory (ORDER).
- The last 4 bits store the number of different variables inside the DATA section (SIZE).
The illustration below is a visual representation of the 64-bits organization that this algorithm creates.
It is important to note that the efficiency of the data compression inside the 32 bits depends heavily on all the variables being ready to write at the same time. For example, needing to save a variable current value and a timestamp of its creation together. Otherwise, a single 32-bit data point write is the preferred use case for most applications.
1.2 Writing and reading data
The algorithm organizes each assembled 64-bit block to be written sequentially into flash memory, starting from the first available address in the beginning of an allotted page until the end. The user must pass the required data and a virtual address to identify it.
Since flash memory can only be written in a clear 64-bit address, modifying an already written block is not possible. Because of this challenge, a data update is made by writing a new 64-bit block with the updated data and using the same virtual address as the previous information.
The reading function works as a basic reverse sequential search that starts at the end of the allotted page and goes to the beginning. Searching for a specific virtual address, the first matching value gets its data block recovered for reading, all other values are ignored.
Important: Applications that need to be resilient against unreliable power conditions must carefully manage any ongoing flash program or erase operations. If these operations are interrupted by events such as a reset or power loss, the flash memory data and/or ECC bits may become corrupted.
Another important consideration is to check the size of the main application and assign the storage pages only within the remaining space. This ensures that the allocated memory is not accidentally overwritten by the main application. The recommended practice is to use the last two available pages of the flash memory.
1.3 Page organization
The writing operations for the algorithm are limited to two subsequent pages of the flash memory where the data blocks are allocated. Once one of the pages is full, a page transfer process is initiated. This is with the goal of saving the most recent data from each virtual address and then erase the data from the last page.
The current active page is detected by verifying the initial values given to each virtual address at the start of the device. These values are preprogrammed and chosen by the user. Every time a page swap is initialized, these values are copied into the new page.
One important thing to consider in this scenario is the flash endurance. The flash endurance is the maximum number of erase and programming sequences that the flash memory can support without affecting its reliability. In the case of the STM32C031, the datasheet in the flash memory endurance and data retention table, warrants that flash can support at minimum 10k cycles, where one cycle is a page erase or page programming command. That is the main reason why using this algorithm for high cycle data is not recommend. If a more robust wear-leveling scheme is required for the application, you may want to check the X-CUBE-EEPROM package.
2. Execution precautions
The most time critical execution in the operation of this algorithm, specifically in applications resistant to reset and power-off, is the page erase time. It takes between 22 ms and 40 ms to complete. The given code does not take special precautions for time efficient or fault prevention in page erase time. However, it is recommended that users who wish to implement this algorithm verify the need in their applications.
Aside from this, in terms of time complexity, the search for a matching virtual address would be considered the second most time-critical operation. Since the linear search is used, the time to complete the search may vary according to the disposition of the data in flash memory.
2.1 The algorithm functions
The functions from the program are divided in two main groups. The first group is the flash memory operations responsible for erasing, programming, and reading from flash memory. However, these implementations may change depending on the microcontroller, since this example was coded using without the use of any HAL or external library functions. More information of specific implementation for each device may be found in its respective reference manual.
The second group is the EEPROM operations responsible for implementing the emulation algorithm logic and the focus of this article. There are four kinds of functions in total:
- Initialization: Setting up the virtual EEPROM and determining the active memory page.
- EEPROM_Status EEPROM_Init(uint32_t* init_variables)
This function initializes the algorithm determining if there is an already initialized page by checking the first addresses for the preprogrammed initialization variables, which are defined by the user. If neither page contains this initialization data, both pages are erased, and the FIRST_PAGE is initialized. The function returns EEPROM_OK if the initialization is successful and EEPROM_ERROR otherwise.
- uint32_t EEPROM_CurrentAddress(void)
This function retrieves the current address in the flash memory by iterating through the page memory in steps of 8 bytes until it locates a written address. It then returns it as the current flash memory address. If a written address is not found, it returns EEPROM_ERROR.
- Writing: Functions to write data to the flash.
- EEPROM_Status EEPROM_WriteData(uint32_t *data, uint8_t dataSize, uint8_t numberOfData, uint16_t virtualAddress)
This function writes data to the flash memory. It takes the data, data size, number of data segments, and virtual address as parameters. The function converts the data size macro into data size divisions, assembles the data block, and writes it to the flash memory. It returns EEPROM_OK if the write operation is successful, EEPROM_ERROR if there is an error, and EEPROM_OVERFLOW if the data size exceeds the maximum allowed.
- EEPROM_Status EEPROM_AssembleDoubleWord(struct EEPROMData blockData)
This function assembles a double word (64 bits) to be written to the flash memory. It combines the received data into a data block and assembles the information block. The function writes the assembled data block and information block to the current flash memory address and verifies the integrity of the written data. It returns EEPROM_OK if the operation is successful, EEPROM_ERROR if there is an error, and EEPROM_OVERFLOW if the total data size exceeds 32 bits.
- Reading: Functions to read data from the flash memory:
- uint32_t FLASH_ReadMemory(uint16_t virtualAddress)
The function iterates through every address from the current page looking for the given virtual address. When a match is located, it uses the auxiliary functions locateData and extract_bits to extract the desired data. The function returns the data if found and EEPROM_ERROR otherwise.
- uint32_t locateData(uint32_t address, uint32_t virtualAddress)
This function locates the data for a given virtual address. It reads the data and control bits at the specified address, determines the allocation sizes based on the EEPROM data size, and iterates through the data segments to locate the data for the given virtual address. The function returns the data if found and EEPROM_ERROR otherwise.
- uint32_t extract_bits(uint32_t data, uint8_t sizes[], uint8_t index)
This function extracts bits from the data based on the sizes array. It calculates the starting bit position for the given index, applies a mask to extract the required bits, and returns the extracted bits.
- Page management: Handling memory pages and ensuring data integrity.
- void EEPROM_SwapPages(void)
This function swaps the pages in the EEPROM. It switches between FIRST_PAGE and SECOND_PAGE, updating the page number, page start, and page end accordingly.
- EEPROM_Status verifyFlashIntegrity(struct EEPROMData blockData, uint32_t address)
This function verifies the integrity of data stored in flash memory. It rebuilds the data block and compares it with the data read from flash memory. The function returns EEPROM_OK if the data matches and EEPROM_WRITE_ERROR otherwise.
Conclusion
This article provided a comprehensive overview of the algorithm used for flash memory storage organization, the methods and its functionality, and the other considerations associated with this approach. By leveraging this approach, it is possible to optimize page usage and achieve significant endurance cycles for low-cycle applications.
