How to run model inference for classification using test images on PC ?

[carlos_b]

Hi,
I've been trying to set up an experiment that deploys the MobileNet v2 model from the STM32AI Model Zoo (mobilenetv2_a035_128_fft_int8.tflite) to do some classification of images using my own test dataset; so that's a set of discrete images, not real-time camera data. I want to use a Python script on a PC to send each test image, one at a time, to the MCU via serial (USB C), have the model run inference, and then return the classification results to the PC. This will repeat in a loop so all images are processed and the Python script will then calculate final metrics, based on all the individual image classification results.

I already have some Python code and can send an image as a tensor to the MCU, but I am struggling to correctly return, or interpret, the classification results. I.e., using the model trained on the Flowers dataset, I expect a classification result consisting of 5 float values (one for each class in the dataset) to be returned. But the values I get back don't seem right. E.g., for a daisy image I get these probability results:
0: daisy 0.332031,
1: dandelion 0.332031,
4: tulips 0.332031,
2: roses 0.000000,
3: sunflowers 0.000000

I think my problem/task is similar to this post: https://community.st.com/t5/edge-ai/inferencing-image-files-from-pc-on-stm32n6570-dk-target/td-p/838807, but I am using the STM32H7 MCU on a NUCLEO-H753ZI board. I followed some of the links suggested in that thread (https://stedgeai-dc.st.com/assets/embedded-docs/stneuralart_getting_started.html, https://stedgeai-dc.st.com/assets/embedded-docs/stm32_on_target_validation.html) and have a few questions:

1. Can I use the built-in ST Edge AI validate command to do this work instead, so that it sends images from a test dataset one at a time and returns classification results to the PC from the model on the MCU?

2. Do the test dataset images need any preprocessing before being sent to the MCU for inference? I.e., it looks like I need to convert the images to a NumPy array format, is that right? What about scaling and normalising the images to make sure they match the properties of the images used to train the model in the first place?

3. If I do use the built-in validate command, do I get back a classification result per image, or just a confusion matrix of overall classification performance? If it is the second one, is there a way to get classification-result-per-image output back to the PC?

Any help would be greatly appreciated.

[Julian E.]

Hi @carlos_b,

Sorry for the late answer.

As of today, you can download STM32Cube AI Studio (that replace X Cube AI):

https://community.st.com/t5/edge-ai/meet-stm32cube-ai-studio-the-new-go-to-tool-for-embedded-ai/td-p/880369

With this you can import input data as npz file or csv file (1) and get the output (2):

 

For your second question:

your model needs to receive the same kind of data you use to train it.

• In the case of a real application, you need to implement all the preprocessing you may have applied in training, to make sure that the input from the sensor is what the model wants.
• In the case of validation with the tool, you don't implement any C code, so you need to provide data in files that is already preprocessed.

For your third question, if you do a validate, you get the confusion matrix in the CLI but I am pretty sure you have the real output in the generated files after the command in core. (in st_ai_output or st_ai_ws).

If you use the STM32Cube AI Studio, you directly have them as shown earlier.

You could also be interested by the AI runner:

https://stedgeai-dc.st.com/assets/embedded-docs/ai_runner_python_module.html 

Basically, it allows to interact with the validation application on your board. Useful to create script for example to compare your target runtime and onnx or tensorflow lite runtime etc..

To use the AI runner, you need a "validation app" that we provided in X Cube AI. But X Cube AI is now replaced by STM32Cube AI Studio (a desktop app) and this is not currently available.

This will be added soon, in the next release. We will also do a tutorial for that soon.

Have a good day,

Julian