ST Edge AI 4.0.1: generated STM32N6 network does not match its reference on the NPU (validate --mode target: cos 0.091, snr -5.87 dB)
- 0 replies
- 34 views
Tool: ST Edge AI Core v4.0.1-20581 (7ed50de05), target stm32n6
Neural-ART compiler: 1.1.3-275
Runtime: ST Neural ART (LL_ATON api) v1.1.3, atonn-v1.1.3-275-g6770925d
Board: STM32N6570-DK, device 0x486, Cortex-M55
Host: Windows 11, STM32CubeProgrammer 2.23.0, gnu-tools-for-stm32 14.3.1
WHAT HAPPENS
A 23x60 -> 30x5 INT8 TCN, generated for stm32n6 with the Neural-ART compiler
and run on the NPU, returns an output that is not the INT8 graph's.
This is not an application bug. "stedgeai validate --mode target", run against
your own NPU_Validation project with your own random stimulus, reports the
board's output as essentially uncorrelated with the reference model the
network was generated from:
acc=n.a. rmse=2.543027878 mae=1.976331592 l2r=1.100639820
mean=-0.346161 std=2.520198 snr=-5.872663 nse=-2.889206 cos=0.091053
cos 0.091 is no correlation. snr -5.87 dB is noise above signal. nse -2.889 is
worse than answering with the mean of the reference. l2r 1.10 is an error
larger than the quantity it is an error in.
Two runs, with and without --full, gave these figures to nine decimals, so the
wrong output is deterministic rather than noisy.
Reference and board also differ in spread, not only in value:
min max mean std
reference (ONNX, host) -3.922 +3.444 -0.105 1.289
c-model (board, NPU) -5.979 +6.218 +0.241 2.298
HOW IT WAS GENERATED
stedgeai generate --model model_qdq_int8.onnx --target stm32n6 \
--st-neural-art default@user_neuralart_STM32N6570-DK.json \
--input-data-type int8 --output-data-type float32 \
--inputs-ch-position chfirst
with stm32n6570_tcn.mpool as the memory pool (weights in xSPI2 at 0x74000000,
activations in AXISRAM3-6). The compiler ran with
-O3 --cache-maintenance --Oauto-sched --native-float
--enable-virtual-mem-pools --Omax-ca-pipe 4 --Ocache-opt --Os
--enable-epoch-controller
and split the graph into 51 epochs: 2 pure software, 16 hybrid, 33 pure
hardware in 15 epoch-controller blobs. The hybrid epochs are SpaceToDepth /
DepthToSpace pairs around the dilated convolutions.
HOW TO REPRODUCE
1. stedgeai generate as above, with the attached model_qdq_int8.onnx and
stm32n6570_tcn.mpool.
2. Load Projects/STM32N6570-DK/Applications/NPU_Validation (build conf N6-DK)
with the generated network.c, using scripts/N6_scripts/n6_loader.py. The
board must be in development boot (BOOT1 at 1-3).
3. stedgeai validate --model model_qdq_int8.onnx --target stm32n6
--mode target --desc serial:<COM>:921600
--val-json <out>/network_c_info.json
The attached st_target_validation_report.txt is what step 3 produced here.
WHAT WE RULED OUT BEFORE POSTING
Our application dumps one inference stage by stage over UDP and compares each
stage with the same graph under onnxruntime. Over four dumps on a running rig:
the board's constants against the model's ... identical, bit for bit
window -> int8 input tensor ............... 0 of 1380 elements differ
int8 tensor -> network output ............. FAILS: 120-138 output levels
network output -> dequantised values ...... max 2.06e-07
From the same dumps:
- The board's output lies on the same quantisation grid as the reference, step
0.047830, to within 7e-4 of an integer multiple, so the output scale is
right. But the board occupies grid levels -125..+120 where the correct
answer occupies -18..+37.
- The error is 81.5% a fixed pattern (|mean error| RMS 2.23 against |error|
RMS 2.47), smooth along the step axis, up to 111 output levels. It is not
the head bias (correlation 0.117) and not the network's response to a
constant input.
- Errors between dumps correlate +0.52..+0.90, highest for the dumps whose
inputs are closest: the wrong function is deterministic.
Also checked on the board: the weight blob is CRC-32 verified against the
generated package at every start; its address is the one the mpool assigns;
the activation arena (AXISRAM3-6) is MPU non-cacheable, so it is never held in
the M55 D-cache; and the runtime reports the clocks the board is configured
for (mcu_freq=800MHz, noc_freq=400MHz, npu_freq=1000MHz, nic_freq=900MHz,
npu_cache=1).
QUESTIONS
1. Does this generation reproduce at ST with the same tool version, and does
validate --mode target fail the same way?
2. Does any of --enable-epoch-controller, --Ocache-opt, --cache-maintenance or
--Omax-ca-pipe 4 change the result? The hybrid SpaceToDepth/DepthToSpace
epochs around the dilated convolutions are the part of this graph we cannot
check from outside.
3. Is there a per-layer target validation that would name the first diverging
epoch? The runtime reports PER_LAYER and PER_LAYER_WITH_DATA among its
capabilities, and --full gave per-node timings but no per-node comparison.
ATTACHED
model_qdq_int8.onnx .............. the input model (QDQ INT8, onnxruntime)
stm32n6570_tcn.mpool ............. the memory pool used for generation
network_generate_report.txt ...... generation report, incl. the epoch split
network_c_info.json .............. for validate --val-json
st_target_validation_report.txt .. the failing run of your own target validation
modeldump_1.bin .................. one inference captured on the board
(Paths in the reports were replaced with C:\USER\... ; nothing else was
changed.)