Computer Vision and CNN Neural Models

[carlosdelfino]

greetings!

Continuing my studies on MCU STM32, I am dedicating myself to STM32N6.

I need to learn how to develop an application that in its first stage will monitor noise and classify it with a neural network type CNN. I have already made a simulator with Python, and a converter of the model's H5 file to tinyML, thus creating a header file (.h) with the neural network model. Now I need to learn how to put it on the stm32N6 and thus classify the noises captured with the STM32N6-DK microphone.

In the second stage, I need to develop a CV (Computer Vision) to identify the silhouette of an object and center it, the equipment will be installed on a rover with omni wheels that will move to position the audio pickup prepared in the first stage.

I looked at the examples that come with cubemx, but I didn't find anything related to the use of CV and NN.

Could you tell me where I can get more advanced examples for STM32N6 that use computer vision and neural networks, especially CNN?

Thanks for the help.

[Julian E. · Accepted solution]

Hello @carlosdelfino,

I would suggest you looking at the AI Getting started for N6:

STM32N6-AI - AI software ecosystem for STM32N6 with Neural-ART accelerator - STMicroelectronics

You will find example of complete application including audio event detection, image classification and object detection.

These are standalone application that are documented, but you can also take a look at the st model zoo:

GitHub - STMicroelectronics/stm32ai-modelzoo-services: AI Model Zoo services for STM32 devices

The model zoo contains scripts to retrain, quantize and deploy models for multiple use cases.

In the case of deployment, the application used are the one from the getting started. So its a different way of using it.

It is in my opinion easier to use at first.

Have a good day,

Julian

[carlosdelfino]

Thanks Julian, I will definitely study this material
About CNN, I made a simulator in Python and achieved the results I expected
for this stage of the project, but now I need to convert the CNN model
files in H5 format to tinyML, I even made a script in Python that generates
the .h file with the binary array, but is it right? here's my question
hehehehe

[Julian E.]

Hello @carlosdelfino,

To use ST tools related to AI, the supported format is h5 or onnx for not quantize models or tflite and onnx qdq for quantized model.

In your case, I would suggest quantizing your model to tflite. (or not quantize it if it is small enough or if you loose too much accuracy after quantization).

Regarding version of onnx, keras and tensorflow supported, please look at the documentation:

https://stedgeai-dc.st.com/assets/embedded-docs/index.html

(you will find a "toolbox" page for each framework that describes the supported version)

note: this online tool could also be useful for you, I let you take a look:

https://stedgeai-dc.st.com/session 

Have a good day,

Julian

[carlosdelfino]

Thank you, it's a long and intriguing road ahead.