Advice for speeding up USB writes?

[Chris Rice]

Hello, we are using an STM32F7 and are saving image data to USB, using mostly the BSP code (disk_write in usbh_diskio.c -> USBH_MSC_Write in usbh_msc.c and so on). However we have found it's quite slow, 6-7 seconds to save one 400x400 image.

I've started profiling and looking at the clocks (very complicated) to see if there is a bottleneck, or if this is something we will have to live with. Does anybody have any advice on improving our throughput? Thanks very much and happy new year.

[Mon2]

Hi. For starters, review this thread:

https://community.st.com/s/question/0D50X00009XkeLmSAJ/usb-msc-device-low-transfer-rate-stm32f7

Some good discussion from Clive & Dr. Tsuneo Chinzei.

[Chris Rice]

Thank you... however looks like it pertains to USB as device (transferring from PC to device?), not host (we are acting as host, saving to an attached USB stick), so there's no MSC_MEDIA_PACKET value for me to tweak. (Got my hopes way up for a second. :))

[Tesla DeLorean]

Are you hosting a Flash Drive?

Doing large media aligned writes helps in most mass-storage applications. For NAND arrays knowing the internal erase block size helps, but at this point is probably unhelpfully large. Allowing the "data" writes to be linear, and deferring/folding small writes to file system structures (FAT, DIR, etc) will provide a means to more optimal write speeds.

Many small f_write operations will be punishing, have a more strategic buffering scheme.

[Chris Rice]

Thanks Clive... I am hosting a flash drive. Not sure if this is germane but I actually make just two calls to f_write, one for the BMP header and one for the entire BMP image data (400x400x3=480,000bytes). The f_write function internally loops on this about 117 times, 4096 bytes per loop, taking about 0.04sec+ per loop.

[Mon2]

See here for more discussion:

https://community.st.com/s/question/0D50X00009XkYTfSAN/usbfs-msc-slow-writing-speed

and this interesting read and possible solution:

https://community.st.com/s/question/0D50X00009XkhrpSAB/stm32f107-and-usb-lib-mass-storage-device-speed-issue

[Chris Rice]

Thanks Mon2, I've skimmed those (Clive seems to have helped many people here :))... it actually looks like I am doing something in the *opposite* extreme, in that I am not buffering my image data *at all*, sending my entire 480000 bytes at once. From there f_write seems to split it up internally... I assume optimally? Perhaps not, I will look into how/why f_write chooses 4096 per loop. Or if you have any insight, thanks very much.

[Tesla DeLorean]

4KB presumably cluster size

Pretty sure this is closer to 32KB on the multi GB SD Cards in FAT32

As i recall FATFS chops down larger buffers, the inefficiencies come when you force it to read/write to deblock data into sectors where the content is partially changed.

The other thing to watch is DMA, the platform expects 4-byte (32-bit word) alignment.

[Chris Rice]

Thank you... apparently the code #defines a sector size of 512 and extracts a cluster size of 4 sectors from something called a "disk access window" in FATFS.win.d8[BPB_SecPerClus] (haven't begun to pull that thread).

Which do you think is the right way to modify this, changing my sector size or something within the FATFS structure? And I'm sorry for the naive question, but do you know a way to check a USB stick to get this info? (sector size, cluster size, etc).

[Mon2]

Sorry that I do not have any low level details of the root cause at this time but can only state that from our years of product development - we did find differences in the quality of the USB media / thumbdrives. Not all vendors are created equal. This may not be a factor but recalling that some (early) Kingston branded sticks worked well for our DOS based production cycles (using RuFUS tool) but then something changed and (recent) Kingston was on the public black list. The experience was that any PC we would use would not cold boot from the new Kingston drives - warm boot was ok but that takes away from our time to reflash each product we build. Changed to Lexar and have been happy. Boots everytime with RUFUS tool used for apply FreeDOS.

Perhaps you can test other brands of USB media? Have you tested with USB 3.0 capable drives?

Believe we have a STM32F7 kit of some sort in the lab (NOT Eval board but Nucleo) and do have a few USB bus analyzers (USB 2.0 and USB 3.1 Gen1). If if it helps, can try to set it up but as usual, limited free time when at work.

Clive - he is keeping this forum alive and very knowledgeable. Also, google Dr. Tsuneo Chinzei and check out his background and assorted patents - it is mind blowing the experience being shared from these seasoned developers.

[Chris Rice]

Thank you Mon2, I appreciate the insight. We have tried this across a variety of sticks and we also notice differences... in fact one type doesn't write at all and we have to investigate this too. But we seem to be slow across all thumb drives.

Clive2, do you agree with my take that I am not making the same mistake as those posts? Do you think f_write should be parcelling up my buffer properly, or is something I should tweak?

Thanks both of you.

[Tesla DeLorean]

FATFS does a good job of using data-in-place, but it doesn't handle memory alignment limitations of the architecture, CMx or STM32. These would need to be caught at the DISKIO level, or judital use of malloc() and bufferring on the user side.

[Chris Rice]

Agree on Clive2, he has helped me out a lot, pretty amazing how on the ball he is here.

[Mon2]

@Chris Rice​ , review this thread:

https://electronics.stackexchange.com/questions/394011/how-to-increase-write-speed-with-usb-high-speed-host-mode-with-stm32f4

and consider to source the referenced WaveShare board for some experimenting. Then you can compare the benchmarks and related solutions. This WaveShare board is based on an external Microchip USB PHY controller but appears to be faster from the posts.

https://www.waveshare.com/wiki/XCore407I

https://www.waveshare.com/