STM32L496 - SD Card SDIO write speeds (FreeRTOS, FATFS and USB), how to get "fast" (> 100kbytes/sec) throughput?

[AFahr.1]

Hi all,

I need to stream data into an SD card, at about 25KBytes/sec max. Currently I'm stuck at 11kBytes/sec, but the SDIO data transfer works, kind of. I'm using an STM32L496 board, FATFS and FreeRTOS, as well as USB.

The USB part works actually really well. Every time I plug in I'm able to download at ~400kBytes/sec and copy to the SD card at say ~200Kbytes/sec. I don't need more than that.

When the USB is unplugged, I copy data from different sensors onto a StreamBuffer (FreeRTOS), and once I get 512 bytes, I copy them over to the SD card using f_write(...) through an additional buffer. I have several issues with this:

• If I set the clock on the SD card to be less than 2, it fails very often, seconds after it starts

hsd1.Init.ClockDiv = 2;

• If I set the bus to be 4 bits wide with the clock 2 or more, f_write returns 1 (DISK ERROR) after about 6 seconds.
• If I set the bus to be 1 bit wide with clock 2 or more, f_write returns 1 after a few minutes, 3-5.
• My SD write function for FatFS is below:

DRESULT SD_write(BYTE lun, const BYTE *buff, DWORD sector, UINT count)
{
	DRESULT res = RES_ERROR;
	uint32_t timeout;
	uint32_t time;
	HAL_StatusTypeDef retval;
	HAL_SD_CardStateTypedef cardState;
	bool sd_wait = true;
 
 
	timeout = count * 500; //Timeout in MS per block
	sd_wait = true;
 
	if(true == sd_card_ready)
	{
	  	retval = HAL_SD_WriteBlocks(&hsd1, (uint8_t *)buff, sector, count, timeout);
	  	cardState = HAL_SD_GetCardState(&hsd1);
 
	  	if (HAL_OK == retval)
		{
			time = HAL_GetTick();
			while(true == sd_wait)
			{
				cardState = HAL_SD_GetCardState(&hsd1);
 
				if(HAL_SD_CARD_TRANSFER == cardState)
				{
					sd_wait = false;
					res = RES_OK;
				}
 
				if((HAL_GetTick() - time) > timeout)
				{
					sd_wait = false;
					HAL_SD_Abort(&hsd1);
				}
			}
		}
	}
 
    return res;
}

The above code is the same for the USB transfer.

I'm at a loss a bit, I thought writing a block (512bytes) of data would increase the data throughput to the SD card, and I have a lot of contiguous data blocks, all the time. Data is constantly read in at about 11 KBytes/sec.

I read somewhere else that doing a pre-check on the data alignment before the "HAL_SD_WriteBlocks" call would help. I might need to try that out.

If anyone can point me in the right direction with examples, links, explanations, etc, I'd really appreciate it. I am able to buffer up to ~500ms of data if necessary... maybe more, how can I detect or recover from a slow (250ms or so) write cycle?

Cheers,

Alberto

[AFahr.1]

Hi Jtron,

That would be an alternative, but a slow one. USB allows you to have dual functions, that is, VCOM on one endpoint and MSC on the other. In my case I have a VCOM on COM18 AND a USB drive (see image below).

On the USB MSC side, you have to implement USB read file and write file operations using the Read / Write block functions in usbd_storage_if.c (well... I use them for convenience):

int8_t STORAGE_Read_FS(uint8_t lun, uint8_t *buf, uint32_t blk_addr, uint16_t blk_len)
{
  /* USER CODE BEGIN 6 */
	uint32_t timeout;
	uint32_t time;
	HAL_SD_CardStateTypedef cardState;
	bool sd_wait = true;
 
	USBD_StatusTypeDef usb_ret = USBD_FAIL;
	HAL_StatusTypeDef hal_ret = HAL_ERROR;
 
 
	timeout = blk_len * 500; //Timeout in MS per block
	sd_wait = true;
 
	if(true == sd_card_ready)
	{
		hal_ret = HAL_SD_ReadBlocks(&hsd1, buf, blk_addr, blk_len, timeout);
		cardState = HAL_SD_GetCardState(&hsd1);
 
		if (HAL_OK == hal_ret)
		{
			time = HAL_GetTick();
			while(true == sd_wait)
			{
				cardState = HAL_SD_GetCardState(&hsd1);
 
				if(HAL_SD_CARD_TRANSFER == cardState)
				{
					sd_wait = false;
					usb_ret = USBD_OK;
				}
 
				if((HAL_GetTick() - time) > timeout)
				{
					sd_wait = false;
					HAL_SD_Abort(&hsd1);
				}
			}
		}
	}
  return usb_ret;
  /* USER CODE END 6 */
}

To use both VCOM and MSC drive you have to write a wrapper that selects either VCOM or MSC depending on the Endpoint used. STM32L4 has a few USB Endpoints, 7 or so if I recall correctly, and you can use 2 for MSC and 2 for VCOM, effectively having two separate devices.

I've attached the wrapper as well, mind, it's not pretty, and it's not all the information, but that gives you an idea... I believe Cube won't let you add two types of USB connections on a projects, so you'll have to manually get the second interface into your project. You'll also see the init process is different...

void MX_USB_DEVICE_Init(void)
{
  /* USER CODE BEGIN USB_DEVICE_Init_PreTreatment */
 
  /* USER CODE END USB_DEVICE_Init_PreTreatment */
 
  /* Init Device Library, add supported class and start the library. */
  if (USBD_Init(&hUsbDeviceFS, &FS_Desc, DEVICE_FS) != USBD_OK)
  {
    Error_Handler();
  }
//  if (USBD_RegisterClass(&hUsbDeviceFS, &USBD_MSC) != USBD_OK)
  if (USBD_RegisterClass(&hUsbDeviceFS, &USBD_MSC_CDC_COMPOSITE) != USBD_OK)
  {
    Error_Handler();
  }
//  if (USBD_MSC_RegisterStorage(&hUsbDeviceFS, &USBD_Storage_Interface_fops_FS) != USBD_OK)
//  if (USBD_MSC_RegisterStorage(&hUsbDeviceFS, &USBD_Storage_Interface_fops_FS) != USBD_OK)
  if(USBD_MSC_CDC_RegisterStorage(&hUsbDeviceFS, &USBD_Storage_Interface_fops_FS) != USBD_OK)
  {
    Error_Handler();
  }
  if(USBD_MSC_CDC_RegisterInterface(&hUsbDeviceFS, &USBD_Interface_fops_FS) != USBD_OK)
  {
    Error_Handler();
  }
  if (USBD_Start(&hUsbDeviceFS) != USBD_OK)
  {
    Error_Handler();
  }
  /* USER CODE BEGIN USB_DEVICE_Init_PostTreatment */
 
  /* USER CODE END USB_DEVICE_Init_PostTreatment */
}

The link below helped me understand a lot:

https://www.beyondlogic.org/usbnutshell/usb1.shtml

I suggest getting the MSC side to work using CubeMX and then add VCOM to it.

Hope it helps.

Cheers,

Alberto

[Jtron.11]

Alberto,

Thank you so much for the details information. I have only one question left to confirm my understanding. You have to use FatFS that support either SDIO/SDMMC in your system as well to support the MSC feature is that correct? to support STORAGE_Read/Write_FS() function.

[AFahr.1]

Yes, you have to use FATFS to read/write to the SD card from your microcontroller application, that is implemented in your sd_diskio.c or user_diskio.c file (the naming changes but it's the same idea, and pretty much the same read/write block functions).

For USB MSC disk transfers, you implement the functions I mentioned before on the USB files.

The sticking point is, your uC application cannot (should not) write into your files using FATFS while Windows has control of your drive, e.g. USB connected. All sorts of weird things can happen there! But so far it's been working for me.

Cheers,

Alberto

[Jtron.11]

Alberto,

Thank you. I am aware of the only one host access the SD card at the time.

[Piranha]

I would refactor that previous code into something like this.

static void usb_monitor_task(void* pvParameters)
{
	for (;;) {
		bool fNewState = HAL_GPIO_ReadPin(USB_Det_GPIO_Port, USB_Det_Pin);
 
		if (fNewState != dg_usb_state) {
			if (fNewState) {
				/* USB cable plugged in */
				MX_USB_DEVICE_Init();
				log_msg(LOG_INFO, (const char*)"USB Connected!");
				rmi_init();
			} else {
				/* USB cable unplugged */
				log_msg(LOG_INFO, (const char*)"USB Disconnected!");
				MX_USB_DEVICE_Deinit();
				rmi_deinit();
			}
			dg_usb_state = fNewState;
		}
 
		vTaskDelay(50);
	}
}

And even better would be to implement it with a vTaskNotifyGiveFromISR() call from EXTI interrupt and waiting on a ulTaskNotifyTake() in a task.

https://www.freertos.org/RTOS_Task_Notification_As_Binary_Semaphore.html

[AFahr.1]

Hi @Piranha​ ,

The reason why USB state is not bool is because at startup / reset the unit could be connected or disconnected, but that code you show above is pretty much what I had before.

For this application this approach works well, but for other event based interfaces (SPI, UART, DMA) I use a lot of vTaskNotifyGiveFromISR() and vTaskNotifyGive().

Thanks for your comments!

Cheers,

Alberto

[Jtron.11]

Alberto,

May I ask you do you measure the throughput "MSC is used for data transfer. We get a stable throughput of about 500 - 700 KBytes per second" ?

I am waiting for the hardware to come in order to implement the USB MSC + uSD card.

If my application only need to get the file from uSD card that store the logging from USB, I don't actually need the CDC VCOM like your application is that correct? I assume you need VCOM in your application because you actually want to send commands to your system. If you don't need to send in the commands, you are off with USB MSC is enough right?

[AFahr.1]

Hi JTron,

No, you don't need the VCOM interface if you're not planning to send any commands to the micro. You only need to implement with CubeMX the MSC interface on the USB. In that case, you get a USB drive and that's it.

We get about 700 kB/sec on raw data transfer between PC (Windows) and micro. Takes about 20 seconds or so to copy a 10MB file. We only use the Windows transfer window to gauge the transfer rate, that's good enough for us.

Cheers,

Alberto

[Piranha]

The dg_usb_state variable is initially 0 - the software always starts in a disconnected state. If the hardware is not connected, nothing happens. If the hardware is connected, the (fNewState != dg_usb_state) detects the change and connects the software. It doesn't matter at which moment the hardware is connected - before of after startup. All of the situations are covered and there is no need for a third state. :)

[AFahr.1]

I feel like you are right, and I did try that approach a few weeks ago, yet for some reason that didn't work on a particular edge case, and that's why I had to add the third state.

But basically I agree with you.

Cheers,

Alberto