2024-07-09 02:08 AM
In the next code I sample audio from the mic, play it to the headphones, then i process the data to get Mel-Spectrogram feature and then I feed the Mel feature into an AI model to recognize speech.
when I enable the functions that calc the mel and the AI model I get a distorted sound.
the time taking to calc the mel and the AI model is 451 ms.
int main(void)
{
uint32_t i;
*/
HAL_Init();
/* Configure the system clock to have a frequency of 120 MHz */
SystemClock_Config();
MX_CRC_Init();
/* Configure LED1 */
BSP_LED_Init(LED1);
Preprocessing_Init();
AI_Init();
/* Initialize DFSDM channels and filter for record */
DFSDM_Init();
/* Initialize playback */
Playback_Init();
/* Start DFSDM conversions */
if(HAL_OK != HAL_DFSDM_FilterRegularStart_DMA(&DfsdmRightFilterHandle, RightRecBuff, 8000))
{
Error_Handler();
}
if(HAL_OK != HAL_DFSDM_FilterRegularStart_DMA(&DfsdmLeftFilterHandle, LeftRecBuff, 8000))
{
Error_Handler();
}
/* Start loopback */
while(1)
{
if((DmaLeftRecHalfBuffCplt == 1) && (DmaRightRecHalfBuffCplt == 1))
{
/* Store values on Play buff */
for(i = 0; i < 4000; i++)
{
PlayBuff[2*i] = SaturaLH((LeftRecBuff[i] >> 4), -32768, 32767);
PlayBuff[(2*i)+1] = SaturaLH((RightRecBuff[i] >> 4), -32768, 32767);
}
if(PlaybackStarted == 0)
{
if(0 != audio_drv->Play(AUDIO_I2C_ADDRESS, (uint16_t *) &PlayBuff[0], 16000))
{
Error_Handler();
}
if(HAL_OK != HAL_SAI_Transmit_DMA(&SaiHandle, (uint8_t *) &PlayBuff[0], 16000))
{
Error_Handler();
}
PlaybackStarted = 1;
}
DmaLeftRecHalfBuffCplt = 0;
DmaRightRecHalfBuffCplt = 0;
}
if((DmaLeftRecBuffCplt == 1) && (DmaRightRecBuffCplt == 1))
{
/* Store values on Play buff */
for(i = 4000; i < 8000; i++)
{
PlayBuff[2*i] = SaturaLH((LeftRecBuff[i] >> 4), -32768, 32767);
PlayBuff[(2*i)+1] = SaturaLH((RightRecBuff[i] >> 4), -32768, 32767);
}
DmaLeftRecBuffCplt = 0;
DmaRightRecBuffCplt = 0;
}
AudioPreprocessing_Run(&PlayBuff[0], &pOutMel[0], 16000);
AI_Run(&pOutMel[0], aiOutData);
}
}
How can I fix it?
thank you
2024-07-09 02:17 AM - edited 2024-07-09 03:20 AM
In what way(s), exactly, is it "distorted" ?
@audi wrote:when I enable the functions that calc the mel and the AI model I get a distorted sound.
Have you compared the output waveforms for the two cases?
Please see the posting tips for how to properly post source code:
2024-07-09 02:50 AM
+
Which cpu ?
How long sampling time ? + how many sampels ?
to process in ?which? time ?
2024-07-09 04:03 AM
I'm rising stm32L4R9I-DISC
How long sampling time ? -> right now the sample rate is 44.1KHz (but I wish to reduce it to 16KHz, meanwhile there is a bug when I'm reduce it to 16 - the audio is really distorted).
how many samples ? -> 16000 samples (8000 from each mic)
even if I replace the buffer to some external array (not the samples from the mic) but some array I copied to the script, I got audio with Eco and duplicate sound
2024-07-09 04:06 AM
the time processing the audio is 415 ms.
only the Mel-spectrogram process is 255 ms, and the AI model is 160 ms