cancel
Showing results for 
Search instead for 
Did you mean: 

processing audio in real time

audi
Associate II

In the next code I sample audio from the mic, play it to the headphones, then i process the data to get Mel-Spectrogram feature and then I feed the Mel feature into an AI model to recognize speech.

when I enable the functions that calc the mel and the AI model I get a distorted sound.

the time taking to calc the mel and the AI model is 451 ms.

 

int main(void)

{
uint32_t i;

*/
HAL_Init();

/* Configure the system clock to have a frequency of 120 MHz */
SystemClock_Config();

MX_CRC_Init();

/* Configure LED1 */
BSP_LED_Init(LED1);

Preprocessing_Init();
AI_Init();

/* Initialize DFSDM channels and filter for record */
DFSDM_Init();

/* Initialize playback */
Playback_Init();

 

/* Start DFSDM conversions */

if(HAL_OK != HAL_DFSDM_FilterRegularStart_DMA(&DfsdmRightFilterHandle, RightRecBuff, 8000))

{
Error_Handler();
}

if(HAL_OK != HAL_DFSDM_FilterRegularStart_DMA(&DfsdmLeftFilterHandle, LeftRecBuff, 8000))

{
Error_Handler();
}

/* Start loopback */
while(1)
{
if((DmaLeftRecHalfBuffCplt == 1) && (DmaRightRecHalfBuffCplt == 1))
{
/* Store values on Play buff */

for(i = 0; i < 4000; i++)

{
PlayBuff[2*i] = SaturaLH((LeftRecBuff[i] >> 4), -32768, 32767);
PlayBuff[(2*i)+1] = SaturaLH((RightRecBuff[i] >> 4), -32768, 32767);

}
if(PlaybackStarted == 0)
{
if(0 != audio_drv->Play(AUDIO_I2C_ADDRESS, (uint16_t *) &PlayBuff[0], 16000))

{
Error_Handler();
}

if(HAL_OK != HAL_SAI_Transmit_DMA(&SaiHandle, (uint8_t *) &PlayBuff[0], 16000))
{
Error_Handler();
}


PlaybackStarted = 1;
}
DmaLeftRecHalfBuffCplt = 0;
DmaRightRecHalfBuffCplt = 0;

}

if((DmaLeftRecBuffCplt == 1) && (DmaRightRecBuffCplt == 1))
{
/* Store values on Play buff */

for(i = 4000; i < 8000; i++)
{

PlayBuff[2*i] = SaturaLH((LeftRecBuff[i] >> 4), -32768, 32767);
PlayBuff[(2*i)+1] = SaturaLH((RightRecBuff[i] >> 4), -32768, 32767);

}
DmaLeftRecBuffCplt = 0;
DmaRightRecBuffCplt = 0;

}

AudioPreprocessing_Run(&PlayBuff[0], &pOutMel[0], 16000);
AI_Run(&pOutMel[0], aiOutData);

}
}

 

How can I fix it?

thank you

4 REPLIES 4
Andrew Neil
Evangelist III

In what way(s), exactly, is it "distorted" ?

 


@audi wrote:

when I enable the functions that calc the mel and the AI model I get a distorted sound.


Have you compared the output waveforms for the two cases?

 

Please see the posting tips for how to properly post source code:

https://community.st.com/t5/community-guidelines/how-to-write-your-question-to-maximize-your-chances-to-find-a/ta-p/575228

 

AScha.3
Chief III

+

Which cpu ?

How long sampling time ? + how many sampels ?

to process in ?which? time ?

If you feel a post has answered your question, please click "Accept as Solution".

I'm rising stm32L4R9I-DISC

How long sampling time ? -> right now the sample rate is 44.1KHz (but I wish to reduce it to 16KHz, meanwhile there is a bug when I'm reduce it to 16 - the audio is really distorted).

how many samples ? -> 16000 samples (8000 from each mic)

even if I replace the buffer to some external array (not the samples from the mic) but some array I copied to the script, I got audio with Eco and duplicate sound

 

 

audi
Associate II

the time processing the audio is 415 ms.
only the Mel-spectrogram process is 255 ms, and the AI model is 160 ms