ST Edge AI Developer Tool: Onnx IR version error, Matlab

[silviag]

Hello everyone,

I am trying to use the ST Edge AI Developer Tool and I am encountering the following error when I try to quantize my model (a RNN with an input layer, 3 GRU layers and a Fully Connected layer):

 2024-12-11 07:25:35.451880: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-12-11 07:25:35.479788: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. ONNX Runtime Model Optimization Failed! Consider rerun with option `--skip_optimization'. Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/onnxruntime/quantization/shape_inference.py", line 91, in quant_pre_process _ = onnxruntime.InferenceSession(input_model_path, sess_option, providers=["CPUExecutionProvider"]) File "/usr/local/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 347, in __init__ self._create_inference_session(providers, provider_options, disabled_optimizers) File "/usr/local/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 384, in _create_inference_session sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model) onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /tmp/quantization-service/3e006691-c53a-4aca-bf0e-804efdb92544/RNN_Final_FT.onnx failed:/onnxruntime_src/onnxruntime/core/graph/model.cc:147 onnxruntime::Model::Model(onnx::ModelProto&&, const PathString&, const IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 9, max supported IR version: 8

My network is exported from MATLAB 2024b in a ONNX file. Does anyone know how to fix this error?

Thank you in advance!
Silvia

[Julian E.]

Hello @silviag ,

A user had a similar issue, here is what he said in the end:  

I found a simple fix - I installed Matlab R2024a and did the export with it (ir version = 7) - that works great.

The thread: Solved: Which Intermediate Representation Version needed -... - STMicroelectronics Community 

Have a good day,

Julian

[silviag · Accepted solution]

Hello @Julian E. 

Thanks for the help. I can confirm that the solution suggested here Solved: Which Intermediate Representation Version needed -... - STMicroelectronics Community works perfectly also in my case. Thanks again!!

Have a good day,
Silvia

[silviag]

Hello again @Julian E. 

If I quantize without a representative data set, the quantization is successful. But if I try to quantize loading my data set, I obtain the following error: "Initializer gru_1_Gather1Indices appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py." that is seemed to be repeated for the entire structure of the model (see below).

Do you have suggestions?

Thanks,
Silvia

Executing with: {'model': '/tmp/quantization-service/3512eb56-41a5-4055-a2ee-82ec3ea3b21c/RNN_Final_FT_01.onnx', 'data': '/tmp/quantization-service/3512eb56-41a5-4055-a2ee-82ec3ea3b21c/dataForQuantization.npz', 'disable_per_channel': False}
Preprocess the model to infer shapes of each tensor
Quantizing model...
[ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: sequenceinput Got: 1 Expected: 3 Please fix either the inputs or the model.

2024-12-11 09:45:15.150815: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-11 09:45:15.178400: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-11 09:45:16.353965760 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_1_Gather1Indices appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.353993394 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_1_ConcatConst1 appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.353999607 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_1_initial_h appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354005795 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_1_W appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354012106 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_1_R appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354017159 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_1_B appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354022133 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_2_Gather1Indices appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354027241 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_2_ConcatConst1 appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354032237 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_2_initial_h appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354037797 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_2_W appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354043986 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_2_R appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354049898 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_2_B appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354055581 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_3_Gather1Indices appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354060986 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_3_ConcatConst1 appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354066310 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_3_initial_h appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354070950 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_3_W appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354075668 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_3_R appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
2024-12-11 09:45:16.354081578 [W:onnxruntime:, graph.cc:1231 Graph] Initializer gru_3_B appears in graph inputs and will not be treated as constant value/weight. This may prevent some of the graph optimizations, like const folding. Move it out of graph inputs if there is no need to override it, by either re-generating the model with latest exporter/converter or with the tool onnxruntime/tools/python/remove_initializer_from_input.py.
....