Encoder only transformer on STM32

[MK06]

I am trying to deploy encoder only transformer model on STM32 by converting pytorch model to onnx format first and then using STM toolchain (STM32 CubeAI). However, I am constantly getting this error when I analyzed the model:

TOOL ERROR: cannot reshape array of size 0 into shape ()

I tried to debug my model but couldn't find any issue in it. Any guidance will be appreciated.

Thank you.

[abcc123]

Which type of chip are you using? I'm also deploying the transformer model on the STM32 microcontroller recently.

[MK06]

Currently working with STM32U575ZIT6Q 

[Julian E. · Accepted solution]

Hi @MK06,

We support some layers required for transformers models, but not all. This depends on how your model is made.

Can you make sure that your model uses only support onnx nodes: ONNX toolbox support

You can also look at this:

https://www.youtube.com/watch?v=q_T6aDoQrgA 

The NPU was developed for Convolutions and is not really adapted for transformers. We do support some layers being part of transformers model, and we are working on future product to improve our solutions regarding LSTM/RNN and transformers.

have a good day,

Julian