[F767ZI] Sampling and storing Measurement Data

[Armin L.]

Posted on February 16, 2018 at 01:03

Hello!

I want to start a project with the NUCLEO-F767ZI soon. In this project I want to measure some quantities with up to 100 kSamples/s and store the data.

Would it be physically possible to have the data either stored to an SD memory card (via SDMMC interface) or sent via Ethernet with up to 10 MB/s in realtime and still having some computing power to execute additional calculations on the MCU? Or are there restrictions on the achievable data rates?

And which memory device or data transfer protocol would you suggest for that purpose in general?

Thanks in advance!

#data-transfer #ethernet #sdmmc

[Tesla DeLorean]

Posted on February 16, 2018 at 10:06

Seems like that would be pretty close to the bandwidth limit of those two interfaces.

[Armin L.]

Posted on February 17, 2018 at 18:20

I thought that the datasheet states about 50 or 100 MB/s as data rate limit. Or are those theoretical limits that cannot be achieved in real operation?

[Tesla DeLorean]

Posted on February 17, 2018 at 19:56

Don't confuse bits (Mbps) and bytes (MBps)

Ethernet 10/100 would be a maximum of 100Mbps , which is going to be something less than 12.5 MBps for practical purposes.

SD cards are highly variable in their performance, clocking the bus at 50 MHz(SDIO 4-bit) might deliver 13 MBps writes on a quality SDXC card using large aligned blocks. Reads likely closer to 20 MBps.

[John Craven]

Posted on February 17, 2018 at 23:51

sorry are we talking adc samples?

stored raw? no processing?

[Armin L.]

Posted on February 18, 2018 at 16:24

Most of the data will be 32-bit floating point numbers. First I wanted to process the ADC values, but I could also leave them as 12-bit integers and process them later, after transferring the data to a PC.

Is the form of the ADC values relevant?

[Tesla DeLorean]

Posted on February 18, 2018 at 17:01

Well you've doubled your data volume there, and burned time processing it.

Perhaps you'd better start experimenting, building the code to move the data to the files, and seeing what your subsystem can sustain in terms of raw capture and then processing and forms of data. If card capacity, and speed are critical focus on that, and let a 3GHz PC grind on the details later.

[John Craven]

Posted on February 18, 2018 at 18:48

767 has two SDMMC units,

you could almost double the rate again, running 2xADC units flipping buffers (or half buffers) with 2x SDIO based uSD cards.

what I can't say at this time without some reading, is when you would hit the limits of DMA1 and DMA2 with 4 streams going.

[Armin L.]

Posted on February 20, 2018 at 00:22

Does that mean that I have to use two SD cards? And I would have to split the data between them and try to fit them together at a PC afterwards?

And one additional question: Is there a difference between SD cards (big size) and µSD cards, apart from the size? Or are there limits in data rate and different protocols used?

[Tesla DeLorean]

Posted on February 20, 2018 at 00:42

Not sure a dual card setup is desirable.

Pretty sure ALL microSD cards are rated at 50 MHz for SPI clocking.

The capacity, make and model have a significant impact on the read/write speeds achievable. Got some 16GB cards which are significantly faster a writing than supposedly better/fast 32GB cards.

https://en.wikipedia.org/wiki/Secure_Digital

 

[John Craven]

Posted on February 20, 2018 at 00:48

You said you wanted to measure some quantities at 100ks/s. It's a little loose.

How many channels at what rate per channel, and how long do you need to sample for?

Yes there is big difference between small cheap cards and larger hi performance cards.

Whats probably more important in you case, is finding a card were the intermittent latency pauses are short. Lots of short periods might be ok. But a single long one might cause a buffer overrun.

Like Clive said 8MB/s sustained write is good. But you also have to look to see the consistency of that rate, and the latency period when the card is doing house cleaning.

[Armin L.]

Posted on February 20, 2018 at 01:07

Among the measurement quantities are only 2 or 3 ADCs. The rest will be floating point values, some of them a result of calculations of incremental encoders connected to the timer modules.

Maybe I can provide details about that tomorrow. However, if it does not work out, I have to decrease the sampling rate.

[Tesla DeLorean]

Posted on February 20, 2018 at 01:58

May be instead of explaining it you can work on the functionality and quantifying the rates at which you can generate data, and how you can modulate/vary that. ie not hard code specific rates and constants, but creating a frame work that has programability and flexibility.

Not sure there are people here that will implement your project. The broad goals specified so far seem achievable with the hardware. You should focus on the volume of data and how it is represent in memory.

[Armin L.]

Posted on February 22, 2018 at 01:18

Ok, thanks so far! I think I have to try and experiment a bit (like John suggested). Another question: Among the measured quantities are a few that do not necessarily have to be sampled with 100 kHz, maybe only 1 kHz. Would it be beneficial if I separate them from the fast-sampled quantities and transfer them via Ethernet? That would decrease the amount of data going to the SD card. Or would this cause additional issues?