Random Glitches in CSV Output Using SDIO, DMA, and FreeRTOS on STM32F405

[AhaUhux]

I am working on a data logger project using SDIO with DMA and FreeRTOS. The only sensor currently in use is the BME280, which is working fine. However, I am encountering issues when writing the CSV file, such as glitches, character repetition, and other seemingly random problems.

Here is the relevant code:

/* USER CODE BEGIN Header_StartSDTask */
#define BUFFER_SIZE 4096
#define BUFFER_THRESHOLD BUFFER_SIZE/4*3
#define HEADER "Time;BaroTemperature;BaroPressure;BaroHumidity;PitotTemperature;PitotDiffPressure;Latitude;Longitude;IMUax;IMUay;IMUaz;IMUgx;IMUgy;IMUgz;IMUmx;IMUmy;IMUmz\n"

char buffer[BUFFER_SIZE] = HEADER;
uint8_t sensorsResponse;
uint16_t adress = sizeof(HEADER) - 1;
uint32_t maxAdress;
uint32_t lastTime;
uint16_t count = 0;
uint16_t count2 = 0;

char adressCarac;

void bufferize() {
adress += snprintf(&buffer[adress], (BUFFER_SIZE - adress), "%lu;", lastTime);
// Additional sensor data processing...
}
/**
* @brief Function implementing the SDTask thread.
* @PAram argument: Not used
* @retval None
*/
/* USER CODE END Header_StartSDTask */
void StartSDTask(void *argument) {
/* USER CODE BEGIN StartSDTask */
uint8_t sensorID;
uint8_t writeCount;

logger.initialize();
logger.createFile("log.csv");
/* Infinite loop */
for (;;) {
while (osMessageQueueGet(SensorsQueueHandle, &sensorID, NULL, osWaitForever) == osOK) {
// Sensor data processing...
if (adress >= BUFFER_THRESHOLD) {
// Clearing the buffer after the threshold
logger.write(buffer, adress + 1);
writeCount++;
memset(buffer, 0, BUFFER_SIZE);
maxAdress = maxAdress > adress ? maxAdress : adress;
adress = 0;
if (writeCount >= 10) {
logger.flush();
writeCount = 0;
}
}
}
}
/* USER CODE END StartSDTask */
}

• The BME280 sensor is functioning properly.
• The issue arises when writing to the .csv file, with glitches, character repetition, and other random problems.
• I have monitored the task stack, which is using about 500 words of the 9000 available.
• I suspected that the problem could be related to the use of snprintf, but I did not find any issues like .. or .xx. in the buffer.
• I tried increasing the clockdivider to 4, but the problem persists.
• I believe the issue may be related to FreeRTOS preemption, but I have enabled NewLibReentrant and FS_REENTRANT, and no other task uses the SDIO or FATFS.

[Tesla DeLorean]

No value to using uint16_t, the MCU is 32-bit

You'd need to ensure the serialization of the buffer, so there isn't concurrent interaction of the buffer vs the writing. The writing (read/write) can take 100's of milli-seconds

I'd probably do writing in multiples of the sectors or clusters, and pull-down the overspill.

Make sure the glitches here aren't related to your write points.

[AhaUhux]

Thank you for the insights, @Tesla DeLorean !

I initially thought that, even with DMA, the write and sync operations in FatFS were blocking—waiting for interrupts to complete before moving on. However, I personally observed that during these operations, the DMA handler is called multiple times, and the SDIO interrupts don’t seem to work as reliably as I expected.

Is there a straightforward way to confirm when write and sync operations are fully completed? Ideally, I’d like to ensure that the data is safely written to the SD card before proceeding, but the current behavior makes it hard to tell if the operation is truly done. Any recommendations on how to manage this or work around it?

Additionally, it would be helpful to have an idea of how long the writing process actually takes. This would allow me to better estimate an appropriate size for the buffer used to receive data, ensuring it aligns well with the write buffer capacity. Do you have any suggestions for a suitable buffer size?

[Tesla DeLorean]

Writing oddly sized blocks to unaligned offset into the file are going to result in a lot of reads and writes to deblock.

The machine has counters and timers, you can measure elapsed time if it's concequential.

Operation over the bus needs to be serialized, are you sure you're not getting errors as a failure in the prior command will block subsequent ones.

I'd be shooting for whole 16 or 32KB blocks depending on available resources. The fill buffer can be independent of the write buffer. You can do them in independent threads but there needs to be a clean hand-off, so things don't change in-flight, and you don't create race conditions.