Discrepancy in Predictions Between Laptop and STM Microcontroller for 1D CNN Model

[khushbu_parmar]

I am working on deploying a 1D CNN model on an STM microcontroller using STM32CubeMX with the X-CUBE-AI package to convert the TensorFlow Lite model into C code.

While testing the model, I noticed significant discrepancies in predictions:

• On Laptop: Using the quantized model for inference gives reasonable predictions.
• On Microcontroller: Using the same test data, the predictions differ substantially when validating the model on hardware.

Steps Taken:

1. Ensured the test data was preprocessed identically for both laptop and microcontroller predictions.
2. Used X-CUBE-AI to generate C code from the quantized TFLite model.
3. Deployed the generated C code on the microcontroller for validation.

Questions:

1. What could cause such prediction differences between the laptop and microcontroller?
2. Could this issue be related to quantization or numerical precision differences during deployment?
3. Are there any additional steps or configurations in X-CUBE-AI to ensure consistent performance between platforms?

[fauvarque.daniel]

The STM32Cube.AI (i.e. X-CUBE-AI) library used on the target has it's own implementation of the neural network kernels so you may see different results between the results when executed in Python versus the target.

In the X-Cube-AI tool  you have 2 validate buttons, one to validate on the desktop using the same library as the one used on the target and one validate on target.

The results of the validate on desktop and validate on target should not differ as it is basically the same library. On the target we have some more optimizations based on the processor you use.

If you see a bad COS (less that 0.98) when you validate the model with real data then it is likely to be a bug in the implementation of the kernels that we should look at.

Could you share your model and ideally some input/output data so we can reproduce the issue ?

Thanks in advance

Regards

[khushbu_parmar]

Thank you for your response. I’d like to clarify my issue further:

• I am using a quantized .tflite model for inference. On my laptop, the predictions from this model (using TensorFlow Lite) are reasonable and consistent.
• When I use X-CUBE-AI’s Validate on Desktop functionality to test the same model and input data, the predictions differ significantly from those obtained via TensorFlow Lite.
• The same issue persists when validating on the target STM32 hardware (Validate on Target).

Steps I’ve taken:

1. Ensured consistent preprocessing for input data across platforms.
2. Verified that the quantized .tflite model is correctly converted using X-CUBE-AI.
3. Cross-checked the scaling and quantization parameters.

Given these discrepancies, I suspect there may be differences in how TensorFlow Lite and STM32Cube.AI implement quantized operations. Could you please confirm whether this is expected behavior or suggest further steps to investigate?

Thank you for your assistance.

Best regards,
Khushbu

[fauvarque.daniel]

Yes I confirm that the implementation of the kernels are different.

If you see a significant accuracy drop using X-CUBE-AI then it may be an issue we have to work on.

Regards