livetune sysin

[mantwih]

Looking to add a 4 mic pdm array setup to livetune environment. Had questions on how deep into the environment we need to go for the changes. Is it just a matter of changing the # of channels for the sysIn, # of slots and active slots on the pdm interface? or do we need to drive deep as address the audio buffer as well. At one point I thought I read the audio buffer was allocated dynamically. We are just looking for a little guidance before we start changing things as there might be a simple way to achieve our goal vs us guessing and creating issues.

[JonathanC · Accepted solution]

Hello, 

From the DK schematic, PDM_SAI1_SD2 is connected to the audio connector, thus its feasible on the DK.
On the Designer FW side, you will need to modify to introduce a 4ch sys mic inputs. You can get inspiration from the existing code; here are few guidelines:

1. update UTIL_AUDIO_MAX_IN_CH_NB to 4
2. Then create an audio config with 4mic to be able to switch to it with the ac audio command in the terminal. This is in the file called stm32_audio_persist_config_setup.c (see example below)
3. You will probably need to modify stm32_audio_sai_pdm.c:
   1. complete the s_gpio_init/s_gpio_deinit with new IOs,
   2. also get inspiration from the code around UTIL_AUDIO_SAI_PDM_D2_USED but with a SlotActive that match your need. As explain, this code is a trick to work around one of the DK that has no connection on D1 , so 2 microphones only but on D2... here you'll have 2mics on D1 & 2mics on D2.

stm32_audio_persist_config_setup.c proposed update:

{
.pConfigName = "stval_mic008",
.audioMs = 8U,
.audioInFreq = 16000U,
.audioInCh = 4U, // Modif here
.audioInHwInterleave = 0U,
.audioOutFreq = 16000U,
.audioOutCh = 2U,
.audioOutResolution = UTIL_AUDIO_RESOLUTION_16B,

.audioInResolution = UTIL_AUDIO_RESOLUTION_16B,
.audioInHwResolution = UTIL_AUDIO_RESOLUTION_8B,
.audioInHwCh = 4UL, // Modif here
.audioInDeviceSelect = AUDIO_IN_DEVICE_DIGITAL_MIC1,
.audioInVolume = 100UL,
.audioOutVolume = 100UL,
.audioInDeviceOutFormat = 1UL,
.audioInUsePdm2Pcm = 1U,
.audioInUsePdm = 1U,
.ac_sysInMicBuffType = ABUFF_FORMAT_FIXED16,

.audioChainLowLatency = 0U,

.usbResolution = UTIL_AUDIO_RESOLUTION_16B,
.ac_sysUsbBuffType = ABUFF_FORMAT_FIXED16
},

[mantwih]

Thanks JonathanC appreciate the guidance. All made sense but right out of the gate we get a ram issue lol when changing UTIL_AUDIO_MAX_IN_CH_NB to 4. 1372 bytes. Is it possible to take away from the user heap?

[JonathanC]

sure,

just reduce HEAP size and it should fit. 
The heap size is set at maximum to give max capabilities to the LiveTune designer. You can take a chunk from there!

[mantwih]

do we need to change any of the audio chain modules? For example this sysin now has 4 channels do we need to somehow modify the pdm2pcm module to now work with 4 channels or is that done in the background on its own

[mantwih]

or is the trick to use the deinterleave module first then use the previous mentioned modules like pdm2pcm

[JonathanC]

Hello,

In your setup, microphones are connected to a hardware block that outputs PDM data (for example, SAI PDM). In this case, LiveTune Designer exposes two types of system inputs:

SysIn‑Microphones

• Provides PCM data.
• The Utilities/Audio software component performs the PDM‑to‑PCM conversion.
• UTIL_AUDIO_MAX_IN_CH_NB = 4 is propagated to the PDM2PCM conversion stage.

SysIn‑MicroPDM

• Provides raw PDM data from the same microphone(s).
• The Utilities/Audio software component forwards the data directly from the DMA, without conversion.

With the SysIn‑MicroPDM input, you can apply the PDM‑to‑PCM conversion directly in the LiveTune dataflow, either by using the pdm2pcm algorithm (equivalent to the one used in the driver) or a CIC filter. This can also be used to introduce delay on the PDM streams if required.
Additionally, acousticBf supports both PCM and PDM input streams; using PDM enables support for shorter microphone spacing.

When designing your dataflow in LiveTune, you are free to route the four system‑input channels as needed. For example, you can split them and send them to multiple router algorithms, create streams based on selected input channels, de‑interleave channels, and so on.

Jonathan

[mantwih]

So essential just changing the sysIn parameters and the rest should fall into place.

How do we overcome the 2048000 sampling frequency when using the pdm sysIn?

sampling frequency 2048000 is not in algo capabilities. This was also confirmed with the original supplied hex file for the designer. We attempted to use the resample but got the same error message.

[JonathanC]

The PDM stream is 1-bit high frequency.
You need to use pdm2pcm or cic filter to get a PCM data. 
Some routing blocks are able to work with PDM format like splitter, router, etc... 
But the algorithm called resample is working on PCM only, this is why it fails.
If after the PDM sysIn you use CiC or PDM2PCM it will work; don't forget to configure the output wire to 16kHz and 128 elements if working at 8ms (see below). Then it will work!

[mantwih]

Got it thanks. I will try changing the output wire. Sorry for the questions, helping out an audio engineer to get 4 mics running but I really don't have much context of how to use the tool. 

To go from 4 channels to 2 channels to resolve the input vs output buffer issues. The best method is to use the router?