issue with NPU operation on Discovery N657

[Franzi.Edo]

STM32N657 / STEdgeAI 4.0.1 - Non-deterministic inference results with a minimal STAI wrapper

Hello,

I am currently evaluating STEdgeAI 4.0.1 on an STM32N657 before integrating AI support into my RTOS (uKOS-X).

My objective is not to use the generated CubeMX application. Instead, I want to build a small self-contained library that encapsulates all ST Edge AI resources and exposes only a minimal API to the application.

The wrapper API is intentionally very small:

gan_npu_status_t npu_init(void);
gan_npu_status_t npu_putInput(const void *input);
gan_npu_status_t npu_inference(void);
gan_npu_status_t npu_getOutput(void *output);

The application never accesses any STAI object directly.

This is only a first validation step, therefore I intentionally use synchronous execution (STAI_MODE_SYNC) and polling only. Interrupt-driven execution will be implemented later.

Configuration

• STM32N657

• STEdgeAI 4.0.1

• Neural-ART generated network

• Single network instance

• ST-generated sources left completely unchanged

• Custom wrapper only

• HyperRAM used for activations

• OctoFlash used for weights

• D-Cache disabled

• Same network instance reused for successive inferences

• No asynchronous execution

• No RTOS interaction during inference

The wrapper simply performs:

stai_runtime_init();
stai_network_init();

...

memcpy(input_buffer, user_input, input_size);

stai_network_run(network, STAI_MODE_SYNC);

memcpy(user_output, output_buffer, output_size);

No additional processing is performed.

Problem

Running exactly the same inference twice with exactly the same input does not produce identical results.

Example:

Inference #1
Output[0] = 0x55

Inference #2
Output[0] = 0x61

The complete output buffer differs.

The difference already exists inside the STAI internal output buffer, before my wrapper copies the data back to the application.

Debugging performed

The following points have been verified.

Input buffer

The internal STAI input pointer remains identical between runs.

Example:

Address : 0x91000000

The first byte is identical.

A checksum computed over the complete input tensor is identical before every inference.

Therefore the network always receives exactly the same input data.

Output buffer

The internal STAI output pointer also remains identical.

Example:

Address : 0x91002000

However, its contents differ after each inference although the input is unchanged.

Wrapper

The wrapper was carefully verified.

There is now exactly one call to:

stai_network_run(network, STAI_MODE_SYNC);

per inference.

Weights

stai_network_get_weights() returns valid pointers.

The generated network does not use user-provided weight buffers.

Input/output management

The generated code reports:

• no user input allocation required

• no user output allocation required

Therefore the wrapper simply obtains the internal pointers using:

stai_network_get_inputs()
stai_network_get_outputs()

and copies the user buffers into them.

HyperRAM

As an experiment, the complete HyperRAM activation area was cleared before every inference.

This did not change the behaviour.

API usage

The wrapper follows the API sequence shown in the ST examples:

• runtime init

• network init

• get input pointer

• copy input

• synchronous run

• copy output

No undocumented API is used.

Question

At this point I have demonstrated that:

• identical input is presented to the network;

• identical internal input address is used;

• identical internal output address is used;

• stai_network_run() returns STAI_SUCCESS;

• the output already differs inside the STAI internal output buffer.

Has anyone observed similar non-deterministic behaviour with STEdgeAI 4.0.1 on STM32N657?

Is there any known configuration requirement (NPU, HyperRAM, XSPI, MPU, RIF, CACHEAXI, etc.) that could produce this behaviour while still returning STAI_SUCCESS?

Any suggestions would be greatly appreciated.

Thank you.

[Franzi.Edo]

Hi everyone,

I finally found the root cause of the issue.

The problem was not related to the generated network, HyperRAM, quantization or the STAI runtime itself.

I integrated a STEdgeAI-generated network into my own RTOS (uKOS-X), where all low-level hardware initialization (clocks, CACHEAXI, external memories, etc.) is already performed by the operating system.

The missing step was simply the initialization of the ST CACHEAXI software module:

#include "npu_cache.h"

npu_cache_enable();
stai_runtime_init();
stai_network_init(s_network);

Although the CACHEAXI hardware was already configured, the Neural-ART runtime expects npu_cache_enable() to initialize its internal software handle before performing any cache maintenance operations.

After adding this call, the network became fully deterministic:

• identical inputs always produce identical outputs;

• activation memory checksums are identical across multiple runs.

I hope this helps anyone integrating STEdgeAI into an existing RTOS instead of using the generated standalone application.

Many thanks to the ST team for their support during the investigation.