NN quantization: wrong IO buffer format in code generation

[bardetad]

Hi,

I'm trying the quantization of Keras (.h5) pretrained neural networks with some different quantization schemes.

Using the 'stm32ai.exe' command I run the following command:

$ stm32ai.exe quantize -q <pathtoconfig.json>

Then I drag and drop the quantized <my_nn_name>.h5 and <my_nn_name_Q>.json output files in X-Cube-AI menu in CubeMX and generate my code.

In the generated file <my_nn_name>.h, there are #defines for IO buffer formats. I believe they depend on the schemes chosen in config.json file and the quantization process.

The code generation seems to work fine, except for the 'ss/sa' for 'Integer arithmetic'.

For lines of interest, I get:

...
#define AI_GENERICNETWORK_IN_NUM       (1)
#define AI_GENERICNETWORK_IN { \
  AI_BUFFER_OBJ_INIT(AI_BUFFER_FORMAT_U8, 512, 1, 2, 1, NULL), \
}
...
#define AI_GENERICNETWORK_OUT_NUM      (1)
#define AI_GENERICNETWORK_OUT { \
  AI_BUFFER_OBJ_INIT(AI_BUFFER_FORMAT_U8, 1, 1, 6, 1, NULL), \
}
...

In this case the call of "ai_platform_network_process()" throws an error of type "AI_ERROR_INVALID_INPUT" or something like that.

But if I replace "AI_BUFFER_FORMAT_U8" by "AI_BUFFER_FORMAT_S8", the NN forward works fine.

Plus it is the same problem using TF Lite quantization, since it is also based on 'ss/sa' scheme according to documentation.

So I believe it might be a wrong generation of code for these #defines (only for the 'ss/sa' scheme)?

Or maybe am I doing something wrong?

Thanks,

Adrien B

[jean-michel.d · Accepted solution]

​Hi,

Nothing is wrong on your side. There is currently an issue in the code generator in this case,  generated define is not correct.

To avoid this situation, you can use the ai_<network_get_info() function to initialize correctly the parameters used to call ai_<my_nn_name>_run(hdl, ai_in, ai_out).

ai_network_report report;

ai_<my_nn_name>_get_info(hdl, &report);

ai_in[0] = report.inputs[0];

Br,

Jean-Michel

[bardetad]

Ok. Thanks a lot !

Best,

Adrien