FatFs Writing Cmds(f_printf) Slows Down ADC

[ADynr.1]

Hello,

I'm using SDMMC to save ADC DMA data to an SD card on Nucleo H723ZG. I made some tests and got distorted results. Then I realized a problem, sampling speed is too slow when I use fatfs writing commands. What might be the problem?

I use a 64K buffer for ADC.

The code for the writing process with half and full callback:

/* USER CODE BEGIN 4 */
void HAL_ADC_ConvHalfCpltCallback(ADC_HandleTypeDef* hadc) {
		for (int i = 0; i < ADC_BUF_LEN/2; i++) {
		fres = f_printf(&fil, "%d\n", adc_buf[i]);
		}
}
void HAL_ADC_ConvCpltCallback(ADC_HandleTypeDef* hadc) {
	HAL_GPIO_TogglePin(GPIOB, GPIO_PIN_11);
	for (int i = ADC_BUF_LEN/2; i < ADC_BUF_LEN; i++) {
		fres = f_printf(&fil, "%d\n", adc_buf[i]);
	}
	if(HAL_GPIO_ReadPin(B1_GPIO_Port, B1_Pin)==GPIO_PIN_SET){  //Check if button pressed
			      f_close(&fil);
			      f_mount(NULL, "", 0);
			      myprintf("closed\r\n");
		  }
}
/* USER CODE END 4 */

The first pic is from full callback without fatfs write cmds:

 

The second pic is with fatfs write cmds.

 

Another problem is it gets much slower if f_close is not done yet.

[ADynr.1]

Thank you so much again!

I was using DMA in circular mode(with half word), sorry I forgot to mention that.

I changed the code again. It can continuously save the data now but there are some problems.

The board barely recognizes the sd card, after many tries(it may take over 30mins) it starts to write. I changed the frequency, and settings many times but none of them make a difference. It usually gives f_mount error(13) or f_open(1). Once it starts to write without any problem, it keeps that until I stop working on it. (Sometimes it still gives the same errors when it is working). And while saving data, randomly it just stops writing(I'm watching the data-saving part from pins). Do you have any idea about that? Should I get an adapter for SDMMC? There was no problem with SPI. Do you think that SPI can handle this speed?

The pin is high while writing. In this picture. There are 2 different speeds, I don't know why. It saves a half faster.

 

This is from the full callback pin toggle(14-bit ADC)

 

From 16-bit ADC

 

Seems like it is fast enough to save 14 bits and 16 bits. But plots look distorted from 16 bits ADC.

(These plots are from 32k buffer size, I couldn't get from 64k buffer because the board started to fail again)

14Bit

 

16Bit

 

Also, I realized another problem. Some data gets lost between buffers(14 Bit)

 

#define BUFFER_SIZE 1024*64
uint16_t adc_buf[BUFFER_SIZE];
 
 
FRESULT res; //Result after operations
// Initialize FatFS and file object
FATFS fs;
FIL file;
 
// Declare flags to indicate when to write to file
volatile uint8_t write_flag_half = 0;
volatile uint8_t write_flag_full = 0;
 
 
void myprintf(const char *fmt, ...) {
  static char buffer[256];
  va_list args;
  va_start(args, fmt);
  vsnprintf(buffer, sizeof(buffer), fmt, args);
  va_end(args);
 
  int len = strlen(buffer);
  HAL_UART_Transmit(&huart3, (uint8_t*)buffer, len, -1);
 
}
// Function to write data to file
void write_to_file(uint16_t* data, uint32_t size) {
	HAL_GPIO_WritePin(GPIOB, GPIO_PIN_8, GPIO_PIN_SET);
  UINT bytes_written;
  FRESULT result = f_write(&file, (void*)data, size, &bytes_written);
	HAL_GPIO_WritePin(GPIOB, GPIO_PIN_8, GPIO_PIN_RESET);
 // if (result != FR_OK) {
 
	//	myprintf("help??\r\n");
		//HAL_Delay(2000);
    // Handle error
  //}
}
 
 
// Set DMA callback functions
void HAL_ADC_ConvHalfCpltCallback(ADC_HandleTypeDef* hadc) {
  // Set flag to indicate first half of buffer is ready for writing to file
  write_flag_half = 1;
}
 
void HAL_ADC_ConvCpltCallback(ADC_HandleTypeDef* hadc) {
	HAL_GPIO_TogglePin(GPIOC, GPIO_PIN_6);
  // Set flag to indicate second half of buffer is ready for writing to file
  write_flag_full = 1;
}
 
//main loop
// Initialize DMA with circular mode and double buffering
	HAL_DMAEx_MultiBufferStart_IT(&hadc1, (uint32_t)&hadc1.Instance->DR, (uint32_t)&adc_buf[0], (uint32_t)&adc_buf[BUFFER_SIZE/2], BUFFER_SIZE/2);
 
  	myprintf("\r\n~ SD card demo by kiwih ~\r\n\r\n");
 
  	    HAL_Delay(1000); //a short delay is important to let the SD card settle
 
  	    //Open the file system
  	  res = f_mount(&fs, "", 1); //1=mount now
  	    if (res != FR_OK) {
  	  	myprintf("f_mount error (%i)\r\n", res);
  	  	while(1);
  	    }
 
      //Now let's try and write a file "write.txt"
  	  res = f_open(&file, "write.txt", FA_WRITE | FA_OPEN_ALWAYS | FA_CREATE_ALWAYS);
      if(res == FR_OK) {
    	myprintf("I was able to open 'write.txt' for writing\r\n");
      } else {
    	myprintf("f_open error (%i)\r\n", res);
      }
      HAL_ADC_Start_DMA(&hadc1, (uint16_t*)adc_buf, BUFFER_SIZE);
 
  while (1)
  {
	  if (write_flag_half) {
	      write_to_file(&adc_buf[0], BUFFER_SIZE/2);
	      write_flag_half = 0;
	    }
	    if (write_flag_full) {
	      write_to_file(&adc_buf[BUFFER_SIZE/2], BUFFER_SIZE/2);
	      write_flag_full = 0;
	    }
	    if(HAL_GPIO_ReadPin(B1_GPIO_Port, B1_Pin) == GPIO_PIN_SET) {
	    		// Button is pressed, flush the remaining data and close the file
	    		f_close(&file);
	    		f_mount(NULL, "", 0);
	    		myprintf("done\r\n");
	    	}
  }
}

[AScha.3]

better now !

>It can continuously save the data now

good !

>but there are some problems.

not good.

> board barely recognizes the sd card

it can work perfect ! i have my adio player here, sd 4bit mode at 50Mhz, always instant mounting 64GB card; also at work, sd 1bit mode, 50MHz, no problem (in > 1000 boards now).

you just have to respect some rules (see my tips, in your first thread about this)short wires, try different pin-speed settings (all pins same speed).

maybe: try 1bit mode first, this is more "forgiving" on signal distortion on connection to sd card.

>Do you think that SPI can handle this speed?

dont know, i never tried. from dso-pics i see: in 5ms write 32KB , -> about 6MB/s ;

to do this with spi , 1 data line, need > 6x8 = 48Mbit .

try, if you get it working at 24...48 MHz clk, it could work.

try sd 1bit mode first.

16 b "distortion" :

is wrap around , probably using a int16 for this uint16 data -> all >0x8000 getting negative by wrong variable use. or wrong bit shift, looks like this.

 

[ADynr.1]

Thanks again!

I changed all uint16t to uint32t, but 16bit ADC plots still wrapped and uint32t brings back the zeros. Zeros still exist on 14 bit ADC with uint32t too(just changing adc_buf's uint16t to uint32t brings zeros)

Wrapped plots:

From zoomed out view(zeros again)
 

And I have data loses between half buffers.(14 bit adc, 64000 buffer) Writing speed is good, so it shouldn't be the problem. Also, I added an if code to get error message printer in case if it fills a half buffer before writing is done. So far no error. Is there anything to avoid this data loss?

 

Today, the board never gave any error about mounting. So, I didn't want to touch it and try SPI. But, if it does it again, I will change SDMMC pins to make soldered pins on sd card to sit directly on the board. Would it be problem if I use an I/O pin as a 3.3V power source?

[AScha.3]

>sd card to sit directly on the board

no need, just wires < 70mm , all same lenght; and (i have) a cer.cap 10uF vcc-gnd at card socket.

>problem if I use an I/O pin as a 3.3V power

yes. card can pull 200mA (!) for short spikes, need good cap on vcc to deliver the current; average is only some mA , depends on how many r/w accesses are done.

>anything to avoid this data loss?

just guessing: caches ON? set D-cache OFF , I-cache ON, optimizer -O2 .

>Wrapped plots:

first question: is data ok and just read/diagram is wrong or is data stored as "wrapped"? check stored data with hex editor, to see whats there on sd card.

and check adc setting in cube: 16bit, no oversampling, no bitshift left and no right shift.

[ADynr.1]

Thanks again!

>I understand. It's interesting, I still have no errors about mounting. If I get it again, I will solder a cap like yours.

>Got it, then I'm not going to use I/Os as a power source.

>As for wrapped plots, the problem was my hex-to-plot phyton code. I totally forgot that, now it's okay.

>I only have the data loss problem now. D-cache and I-cache were off, I turned on I cache, but still have data loss. Tried both of them ON, no difference. Where is the "optimizer -O2"? I couldn't find it.

The current code I use:

#include "main.h"
#include "string.h"
#include "fatfs.h"
#include <stdio.h>
#include <stdarg.h> 
 
 
#define BUFFER_SIZE 64000
uint16_t adc_buf[BUFFER_SIZE];
 
 
FRESULT res; //Result after operations
// Initialize FatFS and file object
FATFS fs;
FIL file;
 
// Declare flags to indicate when to write to file
volatile uint8_t write_flag_half = 0;
volatile uint8_t write_flag_full = 0;
int writing = 0;
 
 
 
void myprintf(const char *fmt, ...) {
  static char buffer[256];
  va_list args;
  va_start(args, fmt);
  vsnprintf(buffer, sizeof(buffer), fmt, args);
  va_end(args);
 
  int len = strlen(buffer);
  HAL_UART_Transmit(&huart3, (uint8_t*)buffer, len, -1);
 
}
// Function to write data to file
void write_to_file(uint32_t* data, uint32_t size) {
	writing = 1;
	HAL_GPIO_TogglePin(GPIOE, GPIO_PIN_1);
	HAL_GPIO_WritePin(GPIOB, GPIO_PIN_8, GPIO_PIN_SET);
  UINT bytes_written;
  FRESULT result = f_write(&file, (void*)data, size, &bytes_written);
  writing = 0;
	HAL_GPIO_WritePin(GPIOB, GPIO_PIN_8, GPIO_PIN_RESET);
	HAL_GPIO_TogglePin(GPIOE, GPIO_PIN_1);
}
 
// Set DMA callback functions
void HAL_ADC_ConvHalfCpltCallback(ADC_HandleTypeDef* hadc) {
  // Set flag to indicate first half of buffer is ready for writing to file
  write_flag_half = 1;
}
 
void HAL_ADC_ConvCpltCallback(ADC_HandleTypeDef* hadc) {
	HAL_GPIO_TogglePin(GPIOC, GPIO_PIN_6);
  // Set flag to indicate second half of buffer is ready for writing to file
  write_flag_full = 1;
}
 
//main loop
 DWORD free_clusters, free_sectors, total_sectors;
 FATFS* getFreeFs;
myprintf("\r\n~ SD card ~\r\n\r\n");
HAL_Delay(1000); //a short delay is important to let the SD card settle
  	    //Open the file system
  	  res = f_mount(&fs, "", 1); //1=mount now
  	    if (res != FR_OK) {
  	  	myprintf("f_mount error (%i)\r\n", res);
  	  	while(1);
  	    }
 
      //Now let's try and write a file "write.txt"
  	  res = f_open(&file, "write.txt", FA_WRITE | FA_OPEN_ALWAYS | FA_CREATE_ALWAYS);
      if(res == FR_OK) {
    	myprintf("I was able to open 'write.txt' for writing\r\n");
      } else {
    	myprintf("f_open error (%i)\r\n", res);
      }
      HAL_ADC_Start_DMA(&hadc1, (uint32_t*)adc_buf, BUFFER_SIZE);
 
 while (1)
  {
	  if (write_flag_half) {
		  write_flag_half = 0;
	      write_to_file(&adc_buf[0], BUFFER_SIZE/2);
 
	    }
	    if (write_flag_full) {
	    	write_flag_full = 0;
	      write_to_file(&adc_buf[BUFFER_SIZE/2], BUFFER_SIZE/2);
 
	    }
	    if(HAL_GPIO_ReadPin(B1_GPIO_Port, B1_Pin) == GPIO_PIN_SET) {
	    		// Button is pressed, flush the remaining data and close the file
	    		f_close(&file);
	    		f_mount(NULL, "", 0);
	    		myprintf("done\r\n");
	    	}
	    if ((write_flag_half == 1 || write_flag_full == 1) && writing == 1) {
	           myprintf("sloww!\n");
	       }
}
}

[AScha.3]

ok. almost good now. 

>set D-cache OFF , I-cache ON, optimizer -O2 ;

optimizer setting is in (while in your main file in editor) : project->properties->C..Build->settings ...

optimizer setting can change a lot , speed of cpu and code size, depends on setting.

ex.: i test on F303 , at 72MHz : pin set/reset in while-loop 65ns (but without Optimizer -O0. -> 260ns ).

data loss should not happen - try with optimizer on.

[ADynr.1]

Yes, almost. :) Today, I got the mounting error, then I placed a 10uF electrolytic capacitor(it's the one I could find for 10uF now). After getting the error again I twisted the cables a little bit, and it fixed. :)

I tried all of the optimizer settings while D-cache OFF, I-cache ON but I still have data loss.

To test, I stopped the adc dma in half and full callbacks, and re-started it at the end of writing callback, but there is still data loss. :\

[AScha.3]

ok, so use : D-cache OFF, I-cache ON , opimizer -O2 .

> To test, I stopped the adc dma in half and full callbacks, and re-started

if ever stop the dma, will make data loss.

the circular buffer with callbacks makes a automatic continuous data flow, as long as half buffer is processed, before the next callback. thats it - no data loss ever.

so-> if there is some data missing, you a: calculate an address wrong or b: there is sometimes something too slow, to be ready before the next callback.

to test with real timing, use 2 pins to check with DSO: pin1 set in 1/2-callback and reset in full-callback. pin2 set when processing of block begins, reset at end of write.

so can see , dual trace: when is data write start*end and is good "safety" time gap.

next test: give adc a (slow) triangle wave, to see, how big is the missing data time, because on slow rising/falling line we can "see" on DSO, about how much is missing.

(on fast sine we dont know: is part of one sine or 15.5 sine waves missing...)

[ADynr.1]

Hi Ascha,

Thank you so much again.

I watched the writing process and half-full callbacks with an oscilloscope.

A half buffer gets ready in 28ms, writing process takes about 9ms. So speed is enough but writing speed fluctuates like 3ms-/+. Rarely it takes more than 30ms which is a problem. Then I prepared a PCB(cap added) to remove cables to make the sd card sit directly. It doesn't change anything in writing speeds and fluctuations, but I can use 100MHz now(It doesn't increase the writing speed as expected, it's like 7ms) and there are still +30ms writing rarely.

I made a mistake before, data loss happens in every quarter of a full buffer, not half. Do you have any idea about it?

[AScha.3]

>data loss happens in every quarter of a full buffer, not half

so you set somewhere a wrong size of a block , maybe something like : sizeof() -> number in bytes , but in dma tranfers maybe "size" in number of transfers - and here it depends on the access you choose, 8/16/32 bit /transfer.

+ did you test with slow triangle, to see how big is the "gap" ?

+ about speed on SDcards...i told you.

card need some 0.2 ...10ms for execute a command, then data transfer coming.

so now you can test:

• card from other brand (32GB sanDisk ultra, 32GB Kingston Canvas select plus = fast !)
• other card size 8-16-32-64 GB and speed group A10, U1 ...
• write block size 16-32-64 KB