Floating-point Audio Classification inference in FP-AI-SENSING1

[MSant.11]

Reading the UM2524 user manual, I noted the following statements at page 14

 

The phrase "A training script for HAR is provided in the Utilities\AI_Resources\Training Scripts\HAR folder along with a Jupyter Notebook to explain all the steps taken." is obviously an oversight.

It seems anyway that the ASC (Audio Scene Classification) is deployed in the FP-AI-SENSING1 package as a fixed-point NN inference.

I have developed a simple two-class floating-point implementation.

Can I modify some files in the package, e.g. asc_processing.c, asc_processing.h and adapt the package to my inference? Do I need for some reason to quantize it to integer?

Do you have tips about such a challenge?

The UM2524 manual is available at https://www.st.com/resource/en/user_manual/um2524-getting-started-with-the-stm32cube-function-pack-for-ultralow-power-iot-nodes-for-ai-applications-based-on-audio-and-motion-sensing-stmicroelectronics.pdf

Thank you

M

[Laurent]

Hello M.

There is no particular challenge to adapt the current fixed point model to floating point . float is even ,in general ,simplifying the design. the code modification should not be extensive.

The main thing to take care of is the real time in case the model is too computational intensive . same for the memory footprint that will expand .

best regards

L

[ChipHead]

In order to generate NN model and C code with FP-AI-SENSING1 4.0.3 and STM32-CUBE AI 7.3.0

As you commented, we need to use X-CUBE AI 4.1.0 to convert the C source code to the network model for ASC. Otherwise, the source will not compile.

Recently, I changed the version from CUBE AI 4.1.0 to CUBE AI 7.3.0. When I use CUBE AI 7.3.0, it is impossible to compile the C source code generated using STM32 CUGE IDE.

1. Cube MX (CUBE AI 4.1.0) + FP-AI-SENSING1 (ver. 4.0.3): I can build the C source generated by STM32 IDE normally.

2. Cube MX (CUBE AI 7.3.0) + FP-AI-SENSING1 (ver. 4.0.3): I cannot build the C source generated by STM32 IDE normally

I wonder if there is a version of FP-AI-SENSING1 available for us to use CUBE AI 7.3.0 instead of CUBE AI 4.10. ​

* Log message with Number 2:

18:34:56 **** Build of configuration B-L475E-IOT01A for project B-L475E-IOT01A ****
 
make all 
 
arm-none-eabi-gcc "D:/e4ds/e4ds-stm-cube-ai/quest01/en.fp-ai-sensing1/STM32CubeFunctionPack_SENSING1_V4.0.3/Projects/B-L475E-IOT01A/Applications/SENSING1/Patch/usbd_audio_patch.c" -mcpu=cortex-m4 -std=c99 -g3 -DSTM32L475xx -DUSE_HAL_DRIVER -DUSE_STM32L475E_IOT01 -DPRIORITY_RTOS -DARM_MATH_CM4 -D__FPU_PRESENT=1 -c -O2 -ffunction-sections -Wall -fstack-usage -MMD -MP -MF"SENSING1/User/Patch/usbd_audio_patch.d" -MT"SENSING1/User/Patch/usbd_audio_patch.o" --specs=nano.specs -mfpu=fpv4-sp-d16 -mfloat-abi=hard -mthumb -o "SENSING1/User/Patch/usbd_audio_patch.o" @"SENSING1/User/Patch/usbd_audio_patch.c_includes.args"
 
arm-none-eabi-gcc "D:/e4ds/e4ds-stm-cube-ai/quest01/en.fp-ai-sensing1/STM32CubeFunctionPack_SENSING1_V4.0.3/Projects/B-L475E-IOT01A/Applications/SENSING1/Src/har_Postprocessing.c" -mcpu=cortex-m4 -std=c99 -g3 -DSTM32L475xx -DUSE_HAL_DRIVER -DUSE_STM32L475E_IOT01 -DPRIORITY_RTOS -DARM_MATH_CM4 -D__FPU_PRESENT=1 -c -O2 -ffunction-sections -Wall -fstack-usage -MMD -MP -MF"SENSING1/User/HAR/har_Postprocessing.d" -MT"SENSING1/User/HAR/har_Postprocessing.o" --specs=nano.specs -mfpu=fpv4-sp-d16 -mfloat-abi=hard -mthumb -o "SENSING1/User/HAR/har_Postprocessing.o" @"SENSING1/User/HAR/har_Postprocessing.c_includes.args"
 
D:/e4ds/e4ds-stm-cube-ai/quest01/en.fp-ai-sensing1/STM32CubeFunctionPack_SENSING1_V4.0.3/Projects/B-L475E-IOT01A/Applications/SENSING1/Src/har_Postprocessing.c: In function 'har_postProcInit':
 
D:/e4ds/e4ds-stm-cube-ai/quest01/en.fp-ai-sensing1/STM32CubeFunctionPack_SENSING1_V4.0.3/Projects/B-L475E-IOT01A/Applications/SENSING1/Src/har_Postprocessing.c:165:47: error: 'ai_buffer' {aka 'struct ai_buffer_'} has no member named 'channels'
 
 165 |  int size   = aiGetReport(algo)->outputs[0].channels;
 
   |                        ^
 
D:/e4ds/e4ds-stm-cube-ai/quest01/en.fp-ai-sensing1/STM32CubeFunctionPack_SENSING1_V4.0.3/Projects/B-L475E-IOT01A/Applications/SENSING1/Src/har_Postprocessing.c: In function 'har_postProc':
 
D:/e4ds/e4ds-stm-cube-ai/quest01/en.fp-ai-sensing1/STM32CubeFunctionPack_SENSING1_V4.0.3/Projects/B-L475E-IOT01A/Applications/SENSING1/Src/har_Postprocessing.c:193:47: error: 'ai_buffer' {aka 'struct ai_buffer_'} has no member named 'channels'
 
 193 |  int size   = aiGetReport(algo)->outputs[0].channels ;
 
   |                        ^
 
make[1]: *** [SENSING1/User/HAR/subdir.mk:46: SENSING1/User/HAR/har_Postprocessing.o] Error 1
 
make: *** [makefile:74: all] Error 2
 
"make all" terminated with exit code 2. Build might be incomplete.
 
 
 
18:35:00 Build Failed. 4 errors, 0 warnings. (took 4s.228ms)
 

* FP-AI-SENSING1, https://www.st.com/en/embedded-software/fp-ai-sensing1.html

* How to peform motion sesning on STM32L4 IOT node, https://wiki.st.com/stm32mcu/wiki/AI:How_to_perform_motion_sensing_on_STM32L4_IoTnode#Create_an_STM32Cube-AI_application_using_X-CUBE-AI

[Laurent]

Hello

Unfortunately, there is no version of FP-AI-SENSING1 compatible with latest 7.3.0. instead FP-AI-MONITOR1 is compatible with 7.2.0 but does not embed ASC model, it exposes Ultra Sound Classification if you are interested.

In my opinion, you need to do some hacking in any case, probably the quickest would be to adapt the code to 7.3.0 X-Cube-AI API. but it would still be a little time consuming.

Best regards

L

[ChipHead]

》 you need to do some hacking in any case, probably the quickest would be to adapt the code to 7.3.0 X-Cube-AI API. but it would still be a little time consuming.

@Laurent​ 

Thanks. To save my time, Can you recommend me related web-pages that describes how to customize (or hack) ​7.3.0 X-Cube-AI?

[Laurent]

Hello ,

There is no specific related web-pages specific to porting an x-cube-ai version to another. But I can give you some pointers and hints.

You have an example of a modern x-cube-ai (7.2) with FP-AI-MONITOR1 most of the code related to X-CUBE-AI integration can be found in:

X-CUBE-AI/App/aiApp.c/h

X-CUBE-AI/App/app_x-cube-ai.c/h

X-CUBE-AI/App/aiTestHelper.c/h

In FP-AI-SENSING1, the equivalent is to be found in /Applications/SENSING1/Src/ai_common.c

/Applications/SENSING1/Inc/ai_common.h

and the run time in the following directory (in both FPs)

Middlewares/ST/STM32_AI_Library/Lib

Middlewares/ST/STM32_AI_Library/Inc

Do not forget to link to the correct runtime librarie (7.3) in your favorite IDE

I hope this helps

L