How to replace AI model in STM32N6570-DK ImageClassification sample for standalone operation

[masatake]

Body:

I would like to replace the AI model in the STM32N6570-DK sample project "STM32N6-GettingStarted-ImageClassification" and run it in standalone mode.

Purpose:
My goal is simply to try running different AI models, not for development purposes. I would like to confirm operation by powering the board ON/OFF without using a debugger.

Environment:

• Board: STM32N6570-DK
• IDE: STM32CubeIDE 2.2.0
• STM32CubeAI Studio

What I want to do:
I am currently trying to generate C code using STM32CubeAI Studio from the following AI model provided by STMicroelectronics, and replace the files in the sample project:

https://github.com/STMicroelectronics/stm32ai-modelzoo/blob/main/image_classification/mobilenetv2/Public_pretrainedmodel_public_dataset/ImageNet/mobilenetv2_a035_128/mobilenetv2_a035_128_int8.tflite

I am also interested in whether it is possible to run third-party AI models in the same way, such as models from the following repository:

https://github.com/PINTO0309/PINTO_model_zoo

Questions:

1. What is the correct procedure to replace the C code generated by STM32CubeAI Studio into the sample project and run it in standalone mode (power ON/OFF)?
2. Is it also possible to run third-party models that are not provided by ST in the same way?

Thank you in advance for your help.

[yessine]

Hello ​@masatake 

I don't know about STM32CubeAIstudio

but I can ensure that you can do it using STM32modelzoo

 

I already tried it and managed to deploy multiple custom image classification models  that I created using notebooks on the STM32N6DK in standalone mode

I advice you to learn about stm32modelzoo and try to install it .it may take few tries to manage to install it succesfully but once done you will be able to deploy  any ai model you want

it also uses the getting started with image classification model like you want

 

just note that wahtever method you use, your AI model should respect some conditions to be compatible with STM32N6 for example it should be TFLITE and int8 or fp32 as output

etc..…

 

 

BR

Yessine

[Alissia1240]

You should be able to replace the model, but it may require more than just copying the new generated C files. The input size, data format, output layers, and preprocessing must match the sample project. You will also need to update the model initialization and inference code if the generated function names are different.

third-party models may work as well, provided STM32CubeAI Studio supports all their layers and operators and the model fits within the board’s memory. After rebuilding the project, flash the complete firmware to the board and make sure the boot settings are correct. Then you should be able to test it by powering the board off and on without using the debugger.

[masatake]

Thank you for your helpful advice. I successfully deployed the sample project and it works fine.

However, when I try to deploy my own model using STM32ModelZoo, the LCD shows nothing after flashing.

During deployment, I noticed this warning in the log:

"series" value is not coherent.. stm32n6 != stm32n6npu

I used the following configuration:

yaml

hardware_setup:
  serie: STM32N6
  board: STM32N6570-DK

The deployment completed successfully with [INFO] : Deployment complete. but the board shows nothing on LCD after switching to boot from flash mode.

Could you please advise what is causing this warning and how to fix it?

[masatake]

Also, I tried with an ONNX model but got the same result. I noticed that the warning says "stm32n6 != stm32n6npu". The STM32N6570-DK has NPU, so should I use "STM32N6NPU" instead of "STM32N6" for the serie parameter? When I change to STM32N6NPU, I get this error:"Options for cfg.deployment.hardware_setup.serie and cfg.deployment.IDE not supported yet! Only options are respectively STM32H7 and GCC."How can I properly target the NPU on STM32N6570-DK?

[masatake]

I have tried multiple versions of CubeIDE (1.17.0, 1.18.1, 1.19.0, 2.2.0) and STEdgeAI (3.0 and 4.0), but LCD still remains black.Every deployment shows this warning:"series" value is not coherent.. stm32n6 != stm32n6npuIs this warning causing the problem? How can I fix this warning?

[yessine]

Hello ​@masatake 

it not about using STM32N6NPU instead of STM32N6

first of all

did you do a power cycle to the board after switchng boot jumper

also please try a tflite model and a template  from config file example folders

to make sure you are using a  correct yaml file

-finally keep using cube ide 1.17.0 and stedgeai 4.0

they are most compatible and i  already use them so they work for sure

BR

Yessine

[masatake]

Thank you for your advice.

I have tried following your suggestions:

• Using CubeIDE 1.17.0 + STEdgeAI 4.0
• Using TFLite model
• Using the template YAML from config_file_examples folder
• Power cycling the board after switching boot jumper to the left

But the LCD still remains black after all these attempts.

Could you please give me more specific steps to debug this issue?

For example:

• Is there any way to check if the firmware was flashed correctly?
• Is there any log output I should check?

Thank you very much.

[yessine]

Hello ​@masatake 
-are you sure the camera is not broken?
-at this point please provide some images of the N6DK  and captions from cmd
-just to make suRe your hardware is ok please run dcmipp application for STM32N6 cube firmware 
https://github.com/STMicroelectronics/STM32CubeN6/tree/main/Projects/STM32N6570-DK/Applications/DCMIPP/DCMIPP_ContinuousMode

BR
Yessine

[masatake]

Thank you for your reply.

The hardware is working fine. I confirmed this by flashing the original sample hex file (STM32N6570-DK_GettingStarted_ImageClassification.hex from the Binary folder), and the LCD shows the camera image correctly.The problem only occurs when I deploy using ModelZoo with my own model.Could you please advise what else I should check?

[yessine]

Hello ​@masatake 
Okay, great!

At least now we have identified the root cause of your issue.

Since your hardware is working fine and the ST ModelZoo examples are also working properly, the issue is most likely related to your custom AI model, which may not be fully compatible with STM32 ModelZoo or the STM32N6.

I have encountered similar issues before.

I still don't know the exact conditions that an AI model needs to meet in order to be fully compatible with STM32 ModelZoo and STM32N6, but here is what I have found so far. These are the conditions I made sure to respect when creating my own models so that they could run successfully on the board.

STM32N6 / ModelZoo compatibility conditions

1. The resulting model must be a valid .tflite model.
2. The model must have a static, well-defined input shape.
3. The model must not require dynamic input dimensions.
4. The model must use TFLite operators supported by ST Edge AI.
5. The model must be successfully parsed by ST Edge AI.
6. The model's input layout must be compatible with the ModelZoo configuration.
7. The model's input data type must match the deployment configuration.
8. The model's output data type must match the deployment configuration.
9. The number of output classes must match the expected label configuration.
10. The label mapping must be consistent with the model's output.

 Things to avoid

•  Using Float32 input/output when requesting INT8 inference
• Mixed or ambiguous quantization
•  Unsupported TFLite operators
•  Dynamic input shapes
•  An incorrect number of output classes
•  Incorrect label mapping
•  Using a representative dataset with preprocessing that differs from the preprocessing used during inference
•  Feeding the STM32N6 a model with an unexpected input data type

   

This is what I have been following for my own models, and so far it has allowed me to deploy them successfully on the STM32N6.

However, there may be additional ModelZoo-specific requirements that I am not aware of yet.

BR
Yessine

[masatake]

Thank you for your advice.

I confirmed that my hardware is working fine by running the DCMIPP_ContinuousMode example successfully - LCD shows camera image correctly.However, when deploying with ModelZoo, LCD still remains black.I have tried:- Multiple CubeIDE versions (1.17.0, 1.18.1, 1.19.0, 2.2.0)- STEdgeAI 3.0 and 4.0- Multiple models including ST's official pre-trained modelsCould you please share your exact YAML configuration and model that you successfully deployed on STM32N6570-DK?Thank you very much.BR