[F767ZI] Sampling and storing Measurement Data

[Armin L.]

Posted on February 16, 2018 at 01:03

Hello!

I want to start a project with the NUCLEO-F767ZI soon. In this project I want to measure some quantities with up to 100 kSamples/s and store the data.

Would it be physically possible to have the data either stored to an SD memory card (via SDMMC interface) or sent via Ethernet with up to 10 MB/s in realtime and still having some computing power to execute additional calculations on the MCU? Or are there restrictions on the achievable data rates?And which memory device or data transfer protocol would you suggest for that purpose in general?

Thanks in advance!

#data-transfer #ethernet #sdmmc

[Tesla DeLorean]

Posted on February 18, 2018 at 00:44

I think you'd have some headroom at 3 MBps

[Armin L.]

Posted on February 18, 2018 at 16:24

Most of the data will be 32-bit floating point numbers. First I wanted to process the ADC values, but I could also leave them as 12-bit integers and process them later, after transferring the data to a PC.

Is the form of the ADC values relevant?

[Tesla DeLorean]

Posted on February 18, 2018 at 17:01

Well you've doubled your data volume there, and burned time processing it.

Perhaps you'd better start experimenting, building the code to move the data to the files, and seeing what your subsystem can sustain in terms of raw capture and then processing and forms of data. If card capacity, and speed are critical focus on that, and let a 3GHz PC grind on the details later.

[John Craven]

Posted on February 18, 2018 at 18:48

767 has two SDMMC units,

you could almost double the rate again, running 2xADC units flipping buffers (or half buffers) with 2x SDIO based uSD cards.

what I can't say at this time without some reading, is when you would hit the limits of DMA1 and DMA2 with 4 streams going.

[John Craven]

Posted on February 18, 2018 at 18:03

I have been playing with uSD with and without freertos.

I quickly converted one on my test app that was logging some uart dma streams to log adc.

adc was setup as continuous circular dma to 32k sample uint16 buffer. 

half full adc interrupt sends message to sd task to write out first half of adc buffer.

complete adc interrupt sends message the sd task to write out second half of buffer.

the sdtask writes each buffer half using dma.

in no way was my test optimal, it was using 1bit Sdio, but I was easily able to get to 150ksamples per second.

at that rate, 300kB/s, the sd rate is still low compared to the sustained 2.3MB/s write rate I have got pushing the sd card.

test was on Nucleo-429zi board with waveshare SDIO breakout board.

if you went to 4bit SDIO (8-10MB/s write) and you used adc double dma buffering, I would think 500-1000ksamples/s would be easy achieved and reliable, probably a lot more.

Note Freertos probably slow things down in this simple case, but I already had it running and just flipped the template to ADC to see what it would do when I read your post.

[Tesla DeLorean]

Posted on February 18, 2018 at 20:41

In nominal configuration 4-bit SDIO on a F769I-DISCO 8.2 MBps Write, 10.6 MBps Read

[John Craven]

Posted on February 19, 2018 at 04:56

Thx. I am still struggling to get 4bit work with HAL.

I am getting a crcfail error in the mounting when CMD55 is executed.

Just started experimenting with workarounds proposed by others. But that's another topic.

I moved my test app to NUCLEO-767 and optimized a bit

No FreeRTOS, and 32 KSample adc buffer (uint_16), this gives SD writes of the half buffers that are 32KB blocks

I lowered the adc sampling down in steps, to increase the ADC rate,

At 15cycles sampling time, i got 920 KSample/s, 1.84MB/s to card. I sustained this for a 1GB file with errors.

Card only did 2.3MB/s just jamming static data to the card. Next sampling time is 3 cycles and would definitely push the card over the edge without using a timer based trigger pace the ADC.

My test used ADC1 and SDMMC1, both use DMA2 streams.

So i am thinking even with 1bit SDIO, using 2xADC and 2xSDMMC units on the 767, you could get close to 1.8 MSamples/s onto uSD continuously.

[John Craven]

Posted on February 19, 2018 at 17:55

Ifound a patch for 4bit SDIOthat works.

https://community.st.com/0D50X00009XkWt5SAF

I bumped the ADC1 sampling time to 3cycles, and read the processor temp sensor at 1660KSamples/s and streamed them to uSD. Checking the file it all seems good. Measurements are quite badwith that lowsettling time. But from a 'how fast can i sample and store' perspective of the original post, it shows what can be sampled and pushed to disk with the F767 (or similar).

[John Craven]

Posted on February 20, 2018 at 00:48

You said you wanted to measure some quantities at 100ks/s. It's a little loose.

How many channels at what rate per channel, and how long do you need to sample for?

Yes there is big difference between small cheap cards and larger hi performance cards.

Whats probably more important in you case, is finding a card were the intermittent latency pauses are short. Lots of short periods might be ok. But a single long one might cause a buffer overrun.

Like Clive said 8MB/s sustained write is good. But you also have to look to see the consistency of that rate, and the latency period when the card is doing house cleaning.

[Armin L.]

Posted on February 20, 2018 at 00:22

Does that mean that I have to use two SD cards? And I would have to split the data between them and try to fit them together at a PC afterwards?

And one additional question: Is there a difference between SD cards (big size) and µSD cards, apart from the size? Or are there limits in data rate and different protocols used?