STM32L4 : Using a part of QSPI external Flash Memory as a frequently updated data log, accessible by a PC with USB MSC (Faisability demand)

[MLair.1]

Hello everyone,

I'm working on a STM32L4 project which embeds a QSPI external Flash Memory (not yet sourced but doesn't matter).

This QSPI Flash Memory would be splitted into 2 parts, one for TouchGFX Images and one for 10MB Filesystem. This FS will have some configurations files and above all a 300kB DataLog file updated each second.

The aim is that this DataLog would be easily exported to a PC by USB.

So this is how the stage is set. The DataLog file will be updated more than 500 000 times per year, so I obviously have to implement a wear leveling algorithm.

I've been thinking for LittleFS rathan than FatFS for his wear leveling feature and then use this FS as a USB MSC.

Will these files be readable/writable by a PC ?

Are my ideas faisible ? If not, would you have some recommendations ?

I hope that everythings's clear,

Many thanks for being interested to my demand,

Maël

[gbm]

What is the usage scenario? How many files you want to keep. Is the log stored in a single file or in many small files. What are the sizes of files and memory. I do have some idea of solving the problem but it depends on the answers to the above questions. With only the log being stored, you could write the data sequentially and "virtualize" the FAT for log file access from PC.

[MLair.1]

I want to keep 2 DataLog files, one of 300kBytes to retain a month of a sensor value, updated each minute (not second as I wrote, so more than 500 000 updates per year), the second one is 40KBytes for something else. So I'll have 2 log files and one more for 1kByte configuration file. The whole memory space is 10MBytes for the moment to handle some wear leveling.

The goal is to readily open them from a PC.

[Pavel A.]

@Maël Lair​ How do you calculate the number of updates for wear leveling?

If 300 kB are accumulated per month, there will be 12 updates (rewrites) per year 🙂

Number of sequential writes does not matter, only number of sector erases matters.

These flash chips usually can be written with very small granularity: bytes or even single bits.

You can keep two log areas: current and previous.

For the PC interface, IMHO a filesystem with USB MSC is not worth it. Much simpler would be MTP ("object storage") or even virtual serial port. Windows 10/11 supports both these classes.

Consider that a PC can automatically detect connection of USB device, run a program that opens the virtual serial port and reads the log via X/Ymodem protocol. 300 KB is a trivial size.

[MLair.1]

Hi Pavel,

It's a sliding log, each minute I erase the oldest value and I write the new value. I had planned to make a custom wear leveling with raw dataLog into the Flash Memory without any FS.

But the objective is to store the log in a file (.txt, .log etc) in order to be readable by a PC without any software development on PC side.

I do agree that a USB Virtual COM Port with AT commands would be mush easier but I suppose that we need a python script to parse the raw bytes in a texte file.

[Pavel A.]

> each minute I erase the oldest value and I write the new value. 

You don't *have to* erase the oldest value every minute. Erase works by large sectors and takes a long time, so you want to erase only once in a while.

Just append the new values to the log, and keep it until uploading the data to the PC

(it is only 300 K per month, right? so at worst, only 3.6 MB per year?)

>  I suppose that we need a python script to parse

May be. Python scripts are fun, you're gong to enjoy writing them.

[MLair.1]

Yeah sure, I wasn't clear, I don't erase each minute, I shift a pointer and when the sector is complete and passed, I erase it, which means that I erase each sector once per month.

[Andreas Bolsch · Accepted solution]

This could be problematic. Sooner or later parts of the external flash will have to be erased. Depending on the particular chip and the block size this can take several hundred miliseconds or even seconds. During that time the flash chip will be completely inaccessible! If this is acceptable, ok, but if not:

A more viable solution is to use two chips, one for program/data and one for the flash filesystem. To save pins, you could connect both chips in parallel except for NCS. Most STM32 with QSPI allow several pins for NCS to be used, so a small routine in internal flash could switch do the file system flash, issue a possibly time-consuming command and immediately switch back to the program flash chip. Another way would be to use "dual flash configuration" but have only one flash active (via flipping the FSEL bit).

[raptorhal2]

Parkinson's Law for Computing: Usage grows to meet the resources available for it.

If there is room in the packaging, mount the flash on removable daughter boards and swap out the daughters at end of life.