Inferencing image files from PC on STM32N6570-DK target

[pmiracle]

Hey,

I have created a simple computer vision NN model and successfully deployed it on my STM32N6570-DK (original and quantized in the zip file).

I want to "push" image files (to "emulate" the camera) from my computer to the board to run inference of the images on the board and get classification results and other information if possible.

What is the quickest way to do that? 

Thanks in advance.

[Julian E.]

Hello @pmiracle,

If you just want to test the results you would get on a set of images, you can use the validation scripts to run inference on random images (or your images on a npz format) and get a report on the errors, exec time etc.

More on that here:

https://stedgeai-dc.st.com/assets/embedded-docs/stneuralart_getting_started.html

Else, you can 

• Write a small firmware task on the STM32 that:
• Waits for an image over UART (USB-CDC if you want faster transfer).
• Receives the bytes into a buffer in RAM.
• Preprocesses them (resize, normalize, etc.).
• Feeds them into the NN input tensor.
• Sends back inference results (class ID, probabilities, etc.) over UART.

This is often the quickest and easiest if you just want to “push” test images from PC → board.

On PC side: use python pyserial:

import serial
import numpy as np
ser = serial.Serial("COM5", 115200)

img = np.load("test_image.npy").astype(np.uint8)  # or read jpg/png and preprocess
ser.write(img.tobytes())

out = ser.readline()
print("Result:", out)

You can look at the following resources to help you with that (or just use them as they are):

Getting started: STM32N6-AI | Software - STMicroelectronics

We have getting started for image classification, object detection etc. With the STM32N6570-DK, they contain the full camera pipeline, model inference and display output.

You can maybe also look at our ST Model zoo Services, that make a use of these getting started by also provide training, quantization, benchmarking, evaluation and deployment scripts.

GitHub - STMicroelectronics/stm32ai-modelzoo-services: AI Model Zoo services for STM32 devices

Have a good day,

Julian

[pmiracle]

Hey Julian, Thanks a lot for the quick reply and the elaborated answer.

When trying to use the 

stedgeai validate --target stm32n6 --mode target -d serial:921600 ...

I am getting an error - 

E200(ValidationError): TARGET: Unable to bind the ST.AI runtime with "network" c-model: []
 E801(HwIOError): Invalid firmware - COM3:921600

The loaded firmware is the image classification example from - 

https://github.com/STMicroelectronics/stm32ai-modelzoo-services/tree/f9212f7d7af9bba30c4c6c879690b7d9d0031714/image_classification

Using the config deployment file - 

https://github.com/STMicroelectronics/stm32ai-modelzoo-services/blob/f9212f7d7af9bba30c4c6c879690b7d9d0031714/image_classification/src/config_file_examples/deployment_n6_config.yaml

This makes sense to me, because in the documentation that you sent here

https://stedgeai-dc.st.com/assets/embedded-docs/stneuralart_getting_started.html#ref_getting_started_evaluate

It says to look at the CLI documentation here

https://stedgeai-dc.st.com/assets/embedded-docs/command_line_interface.html#ref_validation_on_target_overview

That directs to here

https://stedgeai-dc.st.com/assets/embedded-docs/stm32_on_target_validation.html

That directs to the file; Phew... a lot of redirecting :) 

https://www.st.com/resource/en/user_manual/dm00570145.pdf

Where it shows this - 

And I am wondering if such validation application already exists or I need to create one myself from the bottom up using your (Awesome) tutorial here that implements what you said - 

• Write a small firmware task on the STM32 that:
• Waits for an image over UART (USB-CDC if you want faster transfer).
• Receives the bytes into a buffer in RAM.
• Preprocesses them (resize, normalize, etc.).
• Feeds them into the NN input tensor.
• Sends back inference results (class ID, probabilities, etc.) over UART.

My current effort is on checking the feasibility of using my model on the STM32N6 (In terms of performance) and I am not particularly looking into coding a firmware if this can be avoided.

Also, I came across this AiRunner package  - does this has anything to do with this effort?

Thanks in advance and have a great week.

[Julian E.]

Hello @pmiracle,

This is indeed a lot of redirecting :)

We are aware of this complexity, and we are slowly working on it.

Back to your issue:

My current effort is on checking the feasibility of using my model on the STM32N6 (In terms of performance) and I am not particularly looking into coding a firmware if this can be avoided.

Model zoo would have been more useful if you wanted to try retraining a model we propose on your dataset and test it live using the deployment script, that automatically modify the getting started.

In this case, I would suggest to just use the validation script.

The error you get is due to not using the validation firmware before running the command. 

You need to go to your local installation folder of the ST Edge AI Core

For context, the ST Edge AI Core is the "model converter to C code" behind most ST AI Tools.

Then you need to go to /scripts/N6_scripts.

Then edit the config.json file, here is mine:

{
	"compiler_type": "gcc",
	// "gdb_server_path": "C:\\ST\\STM32CubeIDE_1.17.0\\STM32CubeIDE\\plugins\\com.st.stm32cube.ide.mcu.externaltools.stlink-gdb-server.win32_2.2.100.202501151542\\tools\\bin",
	// "gcc_binary_path": "C:\\ST\\STM32CubeIDE_1.17.0\\STM32CubeIDE\\plugins\\com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.13.3.rel1.win32_1.0.0.202411081344\\tools\\bin",
	"iar_binary_path": "C:\\Users\\foobar\\TOOLS\\IAR\\IAR9.30.1\\common\\bin\\",
	"objcopy_binary_path": "C:\\ST\\STM32CubeIDE_1.18.1\\STM32CubeIDE\\plugins\\com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.13.3.rel1.win32_1.0.0.202411081344\\tools\\bin\\arm-none-eabi-objcopy.exe",
	// "cubeProgrammerCLI_binary_path": "C:\\Program Files\\STMicroelectronics\\STM32Cube\\STM32CubeProgrammer\\bin\\STM32_Programmer_CLI.exe",
	"cubeide_path":"C:\\ST\\STM32CubeIDE_1.18.1\\STM32CubeIDE"
	// "make_binary_path":"C:\\ST\\STM32CubeIDE_1.17.0\\STM32CubeIDE\\plugins\\com.st.stm32cube.ide.mcu.externaltools.make.win32_2.2.0.202409170845\\tools\\bin\\make.exe"
}

I think that only the compiler type and cubeide_path are needed.

I don't remember what is by default in this folder, but the next step is to copy your model here, open a CLI and do a generate command: stedgeai.exe generate --model my_model --target stm32n6 --st-neural-art

Then you can run the python script n6_loader.py with your board plugged in dev mode. This will flash the validation firmware with your model.

Finally run the validate command.

If you have error of missing files in any of these steps, they are probably here: 2.2\Utilities\windows

Just copy them to the validation script folder.

Everything I explained here is written here: 

https://stedgeai-dc.st.com/assets/embedded-docs/stneuralart_getting_started.html

Concerning the AI runner. I am not familiar with it, but my understanding is that it is a way to create scripts on top using the validation firmware. A more advanced version of what I described here.

Also, if you follow what I just written, the validation script will run 10 inferences of random images. If you want you can create npz files containing your image and labels and add argument to the validate command to use them.

The command:

stedgeai validate -m mobilenet_v2_0.35_224_fft_int8.tflite --target stm32n6 --mode target -d serial:921600 \
     -vi input_20_images.npy

You can ask chat gpt to create you python code to load images, reshape them as your model input size and save them as npz format.

You can also do it for labels and use the -vo the command.

I don't remember if you get the inference time with this. If you need to have it, you can use the dev cloud and the benchmark step to run 10 random inferences on a real n6 board with your model:

Home - ST Edge AI Developer Cloud

I hope it helps. Let me know if you're stuck anywhere.

Have a good day,

Julian

[pmiracle]

Hey,

I have managed to flash the validation firmware using the n6_loader instructions.
Also here.

When trying to use the validate command after deployment of my own model using the image classification deployment configuration it fails due to:

Running the ST.AI c-model (AI RUNNER)...(name=network, mode=TARGET)
                                                                                                                                                            
 Proto-buffer driver v2.0 (msg v3.1) (Serial driver v1.0 - COM3:921600) ['network']

  Summary 'network' - ['network']
  ----------------------------------------------------------------------------------------------------------------------------------------------
  I[1/1] 'Input_0_out_0'         :   uint8[1,224,224,3], 150528 Bytes, QLinear(0.018658448,114,uint8), activations
  O[1/1] 'Dequantize_28_out_0'   :   f32[1,10], 40 Bytes, activations
  n_nodes                        :   3
  activations                    :   602112
  compile_datetime               :   Sep 16 2025 14:21:52                                                                                                   
  ----------------------------------------------------------------------------------------------------------------------------------------------
  protocol                       :   Proto-buffer driver v2.0 (msg v3.1) (Serial driver v1.0 - COM3:921600)
  tools                          :   ST Neural ART (LL_ATON api) v1.1.1
  runtime lib                    :   atonn-v1.1.1-14-ge619e860 (optimized SW lib v10.1.0-ae536891 GCC)
  capabilities                   :   IO_ONLY, PER_LAYER, PER_LAYER_WITH_DATA
  device.desc                    :   stm32 family - 0x486 - STM32N6xx @800/400MHz
  device.attrs                   :   fpu,core_icache,core_dcache,npu_cache=1,mcu_freq=800MHz,noc_freq=400MHz,npu_freq=1000MHz,nic_freq=900MHz
  ----------------------------------------------------------------------------------------------------------------------------------------------
 Warning: C-network signature checking has been skipped


E200(ValidationError): Invalid data type 'int8', expected 'uint8', can be not quantized.
 Possible reason: '--input-data-type [int8, uint8]' option has been used to deploy the model.

Altough the input .npy array is uint8 (attached). Any ideas?

BTW: 

Regarding the AiRunner - It seems like it is the backend for the "running on the device" methodology through python (instead of validate --target):
Following the code of image-classification stm32ai_main.py --> uses evaluate.py script when evaluation mode is chosen --> uses ai_runner.py.

[Julian E.]

Hello @pmiracle,

I think it is because you used the "--input-data-type uint8" in the generate, but not in the validate.

Here is what I did:

1. stedgeai.exe generate --model ./models/color_classifier.quant.onnx --input-data-type uint8 --output-data-type uint8 --target stm32n6 --st-neural-art
2. python n6_loader.py
3. stedgeai validate -m ./models/color_classifier.quant.onnx --target stm32n6 --input-data-type uint8 --output-data-type uint8 --mode target -d serial:921600

In the folder /n6_script.

If you have npz, add the --vi option.

Also note that for the test I also used the "output-data-type", make sure to use it/ or not correctly, based on what you are doing.

Have a good day,

Julian