How to deploy a custom PyTorch AI model on the STM32N6570?

[llcc]

I want to deploy a PyTorch AI model on the STM32N6570. I have already trained the model and converted it to an ONNX file. I’m looking for a quantization, compilation, and deployment example or tutorial. The speech enhancement example in the STM32 AI Model Zoo Services doesn't match my use case. Although the input shape is the same, my model does not require any pre-processing. I only want to feed input data into the model and observe the output. Also, I would like to know if it's possible to perform deployment without providing a dataset.

[Matthieu]

Hello,

You can refer to the ST Edge AI Core documentation to have details on the quantization including some snippets examples:

https://stm32ai-cs.st.com/assets/embedded-docs/quantization.html

You will also find a Jupyter notebook to quantize and benchmark onnx computer vision models: https://github.com/STMicroelectronics/stm32ai-modelzoo-services/blob/main/tutorials/notebooks/stm32ai_quantize_onnx_benchmark.ipynb

For direct deployment with the model zoo scripts, you need to match the use case as the deployment is considering the pre and post processing for the given use case.

However, you can adapt the Getting Started package to your need. It is also possible to work in standalone mode for deployment of a new model even if more manual intervention is generally required than going through the deployment service of the model zoo.

For the audio getting started package, you can refer to https://github.com/STMicroelectronics/stm32ai-modelzoo-services/tree/main/application_code/audio/STM32N6#manual-deployment

Best Regards

[llcc]

python D:\stm32edgeai\stedgeai\2.1\scripts\N6_scripts\n6_loader.py

I deployed the model following this tutorial https://stm32ai-cs.st.com/assets/embedded-docs/stneuralart_getting_started.html , and the command executed successfully afterward.

05/26/2025 03:09:42 PM  __main__ -- Preparing compiler GCC
05/26/2025 03:09:42 PM  __main__ -- Setting a breakpoint in main.c at line 125 (before the infinite loop)
05/26/2025 03:09:42 PM  __main__ -- Copying network.c to project: -> D:\stm32edgeai\stedgeai\2.1\Projects\STM32N6570-DK\Applications\NPU_Validation\X-CUBE-AI\App\network.c
05/26/2025 03:09:42 PM  __main__ -- Extracting information from the c-file
05/26/2025 03:09:42 PM  __main__ -- Converting memory files in results/<model>/generation/ to Intel-hex with proper offsets
05/26/2025 03:09:42 PM  __main__ -- arm-none-eabi-objcopy.exe --change-addresses 0x71000000 -Ibinary -Oihex network_atonbuf.xSPI2.raw network_atonbuf.xSPI2.hex
05/26/2025 03:09:43 PM  __main__ -- Resetting the board...
05/26/2025 03:09:44 PM  __main__ -- Flashing memory xSPI2 -- 14.625 kB
05/26/2025 03:09:46 PM  __main__ -- Building project (conf= N6-DK)
05/26/2025 03:09:47 PM  __main__ -- Loading internal memories & Running the program
05/26/2025 03:09:50 PM  __main__ -- Start operation achieved successfully

However, when I run the following command, it keeps throwing an error.

python D:/stm32edgeai/stedgeai/2.1/scripts/ai_runner/examples/checker.py ^
More?   -d serial:COM9:921600 ^
More?   --input-file D:/stm32aibushu/input_data.npy ^
More?   --perf-only -b 32

Creating AiRunner session with `Namespace(desc='serial:COM9:921600', batch=32, name=None, input_file='D:/stm32aibushu/input_data.npy', perf_only=True, io_only=False, with_data=False, show_tensors=False, verbosity=0, debug=False)`
ERR: No c-model available, use the --desc/-d option to specifiy a valid path/descriptor
 E801(HwIOError): Invalid firmware - COM9:921600

Would you happen to know how to fix this issue? Thanks!

[SlothGrill]

Hello
Is there a reason why you specify the com-port number ? 

# This should be enough if you do not have multiple ST-link connected, as a first step.
-d serial:921600

This may be the issue as the "n6_loader" does not seem to show other issues.

If this does not solve the issue, could you also post your "n6_loader.log" so we can check if there is an issue with other steps... 

[llcc]

My "n6_loader.log" is the following.

D:\stm32cubeprogrammer\bin\STM32_Programmer_CLI.exe -q -c port=SWD mode=powerdown freq=2000 ap=1

      -------------------------------------------------------------------
                       STM32CubeProgrammer v2.19.0                  
      -------------------------------------------------------------------

Error: No debug probe detected.
D:\stm32cubeprogrammer\bin\STM32_Programmer_CLI.exe -q -c port=SWD mode=hotplug ap=1 --extload D:\stm32cubeprogrammer\bin\ExternalLoader\MX66UW1G45G_STM32N6570-DK.stldr --download D:\pycharmcode\bushu\st_ai_output\network_atonbuf.xSPI2.hex --verify

      -------------------------------------------------------------------
                       STM32CubeProgrammer v2.19.0                  
      -------------------------------------------------------------------

Error: No debug probe detected.
D:\stm32cubeprogrammer\bin\STM32_Programmer_CLI.exe -q -c port=SWD mode=powerdown freq=2000 ap=1

      -------------------------------------------------------------------
                       STM32CubeProgrammer v2.19.0                  
      -------------------------------------------------------------------

ST-LINK SN  : 002700393433511930343835
ST-LINK FW  : V3J15M6
Board       : STM32N6570-DK
Voltage     : 3.29V
Error: Unable to get core ID
Error: Cannot connect to access port 1! 
 If you are trying to connect to a device with TrustZone enabled please try to connect with HotPlug mode. 
 If you are trying to connect to a device which supports Debug Authentication with certificate or password, please open your device using it.

D:\stm32cubeprogrammer\bin\STM32_Programmer_CLI.exe -q -c port=SWD mode=hotplug ap=1 --extload D:\stm32cubeprogrammer\bin\ExternalLoader\MX66UW1G45G_STM32N6570-DK.stldr --download D:\pycharmcode\bushu\st_ai_output\network_atonbuf.xSPI2.hex --verify

      -------------------------------------------------------------------
                       STM32CubeProgrammer v2.19.0                  
      -------------------------------------------------------------------

ST-LINK SN  : 002700393433511930343835
ST-LINK FW  : V3J15M6
Board       : STM32N6570-DK
Voltage     : 3.29V
SWD freq    : 8000 KHz
Connect mode: Hot Plug
Reset mode  : Software reset
Device ID   : 0x486
Revision ID : Rev B
Revision ID : Rev B
Device name : STM32N6xx
Device type : MCU
Device CPU  : Cortex-M55
BL Version  : --

Opening and parsing file: network_atonbuf.xSPI2.hex


Memory Programming ...
  File          : network_atonbuf.xSPI2.hex
  Size          : 14.28 KB 
  Address       : 0x71000000 



Erasing memory corresponding to sector 0:
Erasing external memory sector 256
Download in Progress:


File download complete
Time elapsed during download operation: 00:00:00.966



Verifying ...




Download verified successfully 


D:/Stm32Cubbeide/STM32CubeIDE_1.16.0/STM32CubeIDE/plugins/com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.12.3.rel1.win32_1.0.200.202406191623/tools/bin/arm-none-eabi-gdb.exe -batch --command=D:/stm32edgeai/stedgeai/2.1/Projects/STM32N6570-DK/Applications/NPU_Validation/armgcc/n6_commands.gdb D:/stm32edgeai/stedgeai/2.1/Projects/STM32N6570-DK/Applications/NPU_Validation/armgcc/build/Project.elf

0x18003a1a in ?? ()
Loading section .isr_vector, size 0x34c lma 0x34000000
Loading section .text, size 0x25264 lma 0x34000350
Loading section .gnu.sgstubs, size 0x20 lma 0x340255c0
Loading section .rodata, size 0x38b4 lma 0x340255e0
Loading section .ARM, size 0x8 lma 0x34028e94
Loading section .init_array, size 0x4 lma 0x34028e9c
Loading section .fini_array, size 0x4 lma 0x34028ea0
Loading section .data, size 0xfc lma 0x34028ea4
Start address 0x34010cb8, load size 167824
Transfer rate: 127 KB/sec, 9872 bytes/write.

*****
**Finished User setup phase. Setting BPX at end of init...
*****
Temporary breakpoint 1 at 0x34000a76: file ../../../Applications/NPU_Validation/Core/Src/main.c, line 125.

Temporary breakpoint 1, main () at ../../../Applications/NPU_Validation/Core/Src/main.c:125
125	  aiValidationInit();
[Inferior 1 (Remote target) detached]
D:\stm32cubeprogrammer\bin\STM32_Programmer_CLI.exe -q -c port=SWD mode=powerdown freq=2000 ap=1

      -------------------------------------------------------------------
                       STM32CubeProgrammer v2.19.0                  
      -------------------------------------------------------------------

ST-LINK SN  : 002700393433511930343835
ST-LINK FW  : V3J15M6
Board       : STM32N6570-DK
Voltage     : 3.29V
Error: Unable to get core ID
Error: Cannot connect to access port 1! 
 If you are trying to connect to a device with TrustZone enabled please try to connect with HotPlug mode. 
 If you are trying to connect to a device which supports Debug Authentication with certificate or password, please open your device using it.

D:\stm32cubeprogrammer\bin\STM32_Programmer_CLI.exe -q -c port=SWD mode=hotplug ap=1 --extload D:\stm32cubeprogrammer\bin\ExternalLoader\MX66UW1G45G_STM32N6570-DK.stldr --download D:\pycharmcode\bushu\st_ai_output\network_atonbuf.xSPI2.hex --verify

      -------------------------------------------------------------------
                       STM32CubeProgrammer v2.19.0                  
      -------------------------------------------------------------------

ST-LINK SN  : 002700393433511930343835
ST-LINK FW  : V3J15M6
Board       : STM32N6570-DK
Voltage     : 3.29V
SWD freq    : 8000 KHz
Connect mode: Hot Plug
Reset mode  : Software reset
Device ID   : 0x486
Revision ID : Rev B
Revision ID : Rev B
Device name : STM32N6xx
Device type : MCU
Device CPU  : Cortex-M55
BL Version  : --

Opening and parsing file: network_atonbuf.xSPI2.hex


Memory Programming ...
  File          : network_atonbuf.xSPI2.hex
  Size          : 14.28 KB 
  Address       : 0x71000000 



Erasing memory corresponding to sector 0:
Erasing external memory sector 256
Download in Progress:


File download complete
Time elapsed during download operation: 00:00:00.955



Verifying ...




Download verified successfully 


D:/Stm32Cubbeide/STM32CubeIDE_1.16.0/STM32CubeIDE/plugins/com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.12.3.rel1.win32_1.0.200.202406191623/tools/bin/arm-none-eabi-gdb.exe -batch --command=D:/stm32edgeai/stedgeai/2.1/Projects/STM32N6570-DK/Applications/NPU_Validation/armgcc/n6_commands.gdb D:/stm32edgeai/stedgeai/2.1/Projects/STM32N6570-DK/Applications/NPU_Validation/armgcc/build/Project.elf

0x18003a1a in ?? ()
Loading section .isr_vector, size 0x34c lma 0x34000000
Loading section .text, size 0x25264 lma 0x34000350
Loading section .gnu.sgstubs, size 0x20 lma 0x340255c0
Loading section .rodata, size 0x38b4 lma 0x340255e0
Loading section .ARM, size 0x8 lma 0x34028e94
Loading section .init_array, size 0x4 lma 0x34028e9c
Loading section .fini_array, size 0x4 lma 0x34028ea0
Loading section .data, size 0xfc lma 0x34028ea4
Start address 0x34010cb8, load size 167824
Transfer rate: 123 KB/sec, 9872 bytes/write.

*****
**Finished User setup phase. Setting BPX at end of init...
*****
Temporary breakpoint 1 at 0x34000a76: file ../../../Applications/NPU_Validation/Core/Src/main.c, line 125.

Temporary breakpoint 1, main () at ../../../Applications/NPU_Validation/Core/Src/main.c:125
125	  aiValidationInit();
[Inferior 1 (Remote target) detached]

View more

When I run the following command, it hangs at this step without any response or further output.Thanks!

(stmaitorchn) D:\pycharmcode\bushu>python D:/stm32edgeai/stedgeai/2.1/scripts/ai_runner/examples/checker.py -d serial:921600 --perf-only -b 32 -i D:/pycharmcode/bushu/input_data.npy
Creating AiRunner session with `Namespace(desc='serial:921600', batch=32, name=None, input_file='D:/pycharmcode/bushu/input_data.npy', perf_only=True, io_only=False, with_data=False, show_tensors=False, verbosity=0, debug=False)`

[SlothGrill]

Okay thanks :)

So the "loader" seems to do its job.

Can you then try to launch this easier command, and provide the output of it:

python D:/stm32edgeai/stedgeai/2.1/scripts/ai_runner/examples/ai_checker.py -d serial:921600 --debug

Thanks.

+ could you check the UART baudrate in the sourcecode to be sure it says 921600 ? Thanks

[llcc]

(stmaitorchn) D:\stm32aibushu>python D:/stm32edgeai/stedgeai/2.1/scripts/ai_runner/examples/ai_checker.py -d serial:921600 --debug
Running ai_checker v1.3.0..

Opening st.ai runtime "serial:921600" (AiRunner v3.2)..
[D:AiRunner] creating "AiRunner" object (v3.2)
[D:AiRunner] resolving('desc' parameter: 'serial:921600')..
[D:AiRunner] creating <stm_ai_runner.pb_mgr_drv.AiPbMsg object at 0x000001837C4F8A90> (v2.0)
[D:AiRunner] creating <stm_ai_runner.serial_hw_drv.SerialHwDriver object at 0x000001837C4F86D0>
[D:AiRunner] 'desc' parameter for 'AiPbMsg' driver: '921600'
[D:AiRunner] connecting..
[D:AiRunner] baud=921600 device=None timeout=0.1 io_hw_log=None
[D:AiRunner] 0: {'type': 'serial', 'device': 'COM9', 'desc': 'STMicroelectronics STLink Virtual COM Port (COM9)', 'hwid': 'USB VID:PID=0483:3754 SER=002700393433511930343835 LOCATION=1-1:x.1'}
[D:AiRunner] is_alive() STM32 - read timeout 609.0ms/500ms
[D:AiRunner] 1: {'type': 'serial', 'device': 'COM3', 'desc': '蓝牙链接上的标准串行 (COM3)', 'hwid': 'BTHENUM\\{00001101-0000-1000-8000-00805F9B34FB}_LOCALMFG&0000\\7&1C6CEAEF&0&000000000000_00000000'}

[llcc]

Hello! Is it because my development board is in dev mode and both boot0 and boot1 are set to the right, causing this issue?

[Julian E.]

Hello @llcc,

No, the board needs to be in dev mode (both pins to the right).

Can you please make sure to use a usb c to usb c cable. then put both pin to the right and unplug/replug the board to the pc. Make sure your cable is data.

In order, you need to to a generate of your model, then the n6_loader.py and finally the ai_checker.py as explained here:

https://stedgeai-dc.st.com/assets/embedded-docs/stneuralart_getting_started.html#getting-started 

Could you check the troubleshooting points we listed here and tell me if anything helped:
https://stedgeai-dc.st.com/assets/embedded-docs/stneuralart_getting_started.html#ref_troubleshoot

Have a good day,

[llcc]

Thanks! The issue has been resolved. However, I noticed this warning:

"Do not disconnect the board until the end of the experimentation: the application is stored in RAM and not in external flash (power-cycling the board erases the firmware). After a power-cycle, be sure to run this step before running a new evaluation with the AI Runner."

I would like to ask: how can I deploy the model to the STM32N6570-DK so that it can run normally even after a reboot, without being erased?