low latency streaming audio over IP --- NUCLEO-H723ZG

[SWenn.1]

I am looking to stream audio over an ethernet connection to a cellular hot spot.  This will be low latency.  Can someone tell me what STM32 firmware library I should use and if this is an appropriate hardware for that task?

Thanks

[Andrew Neil]

---

---

Define "audio": how many channels? bit rate? bit depth?

Define "low latency".

When you say "ethernet", is that purely at the physical layer, or do you also need to be IP-based?

Especially with IP, it is a non-trivial exercise - doubly so with wireless links.

https://en.wikipedia.org/wiki/Audio_over_Ethernet

[SWenn.1]

Hello Andrew.

Great questions!

This will be up to about 10kHz 12kHz.

Low latency will be sub second.

IP-based.

Right now my vision (and please feel free to jump in here).

I have a NUCLEO-H723ZG on the way.  I am looking at the X-CUBE-OPUS and X-CUBE-ETHERNET libraries to integrate.  I 'hope' I can take A/D samples, create PCM for the OPUS and send to the ETHERNET library. Initially I want to get the pieces in place and do a proof of concept. On the final solution I am still out on the fence as whether to use UDP/Multicasting vs TCP to a website and hosting....I guess it comes down to reliability vs latency in that sense.

Thoughts?

Steve

[Andrew Neil]

---

@SWenn.1 wrote:

This will be up to about 10kHz 12kHz.

---

Is that your sample rate?

What bit-depth?

Mono or stereo?

---

@SWenn.1 wrote:

Low latency will be sub second.

---

eg, 950ms would be OK ?

Another issue to consider on IP networks is the allowable "jitter" on that ...

[SWenn.1]

Sample rate (whatever OPUS allows)...24 or 48khz 

bit depth....16 if possible, if not 12 and padded for PCM

mono....

if 950 is the best I can get then so be it...(if you have another lower latency path in the confines of that nucleo I mentioned I would love to hear it)

[LCE]

I'm doing that with the H723 / H733 / H735.

8 audio channels (I2S from the 2 SAIs) at 32 bit / 200 kHz (51.2 Mbit/s) via DMA to internal SRAM buffers to HyperRAM via DMA to ETH via TCP to PC host.

I use lwIP (some minor parts tuned a little) and self-made ETH-driver, no OS.

CPU at 400 MHz, no audio processing, CPU sleeping about 50%.

Works like a charm.

If you have a client that's taking data quickly and reliably (so just do not plug into "a network" but directly to host) you probably don't need extra RAM, and latency will be about 2 packet times (guessing: < 5 ms).

So how many audio channels do you have, just this one?
Anyway, at that sampling rate it shouldn't be a problem - BUT it will be lots of work.

Spoiler: CubeMX, and even many HAL functions will not do all the work for you.
In all of my "signal path" relevant code there's only HAL_GetTick() left from HAL. :D

[SWenn.1]

Just one audio channel.  I too was hoping to avoid OS.  That is really low latency!  Can you explain the 

" (so just do not plug into "a network" but directly to host)"....

My vision would be to send to a IP address of at say AWS and users could stream off of that...

[LCE]

> My vision would be to send to a IP address of at say AWS and users could stream off of that...

Live streaming to or from AWS? Anyway, I have no idea about that.

Sometimes Windows 10 alone has other stuff in mind, and the more nodes in the network, the more your latency goes up - and more than the inbuilt RAM might be needed (prolly not with 1 channel at 16b/48k).

So, my short latency is in a super small network: STM32 - PoE switch - PC.
I have seen latency go up in our company office network, depending on the traffic.
I have tweaked lwIP PCBs by adding DiffServ / QoS priorities, but I have no idea if that actually helped.
There's some more testing to do on my side...

[SWenn.1]

Thanks so much for the advice!  One more quick question.....Do you know if / where there is an AAC codec ST library?

[LCE]

Sorry, I don't.