On-board fine tuning/training of a model

[amirkfir]

Hey!

I have a ResNet model that I have deployed on my NUCLEO-N657X0-Q, and I want to run a training step on the board itself. I'm fine even with training just the last 1-2 layers of my model on a single sample, so that the memory on the board will suffice. Is the NPU itself capable of it? is there a assisting tool for on-board training? Is it more likely to succeed by using the ARM cpu for that?

Thank you,

Amir

[Julian E.]

Hi @amirkfir,

The ST Edge AI Core does not support on board training.

The only similar solution I know of is the on board training available for anomaly detection model through NanoEdge AI Studio.

You can maybe take a look at it, depending on your use case it could be an alternative for you.

Bringing your own model and training it on board is not supported nor part of the road map as it is the first time I see such request.

Would you mind sharing a bit of context, why are you interested in such matter? I don't see why you would need to retrain a model.

Have a good day,

Julian

[amirkfir]

Thank you for the reply @Julian E.,

I am trying to implement a model that is capable of "test time training" (there were several articles about it), where the idea is to make a single training step on a the deployed model to adjust it to data distribution drift that you didn't trained on and that kills your performances (e.g a specific noise like a water drop on the lens, weird lighting etc.).  

I will look at the anomaly detection code and see if I can take some parts from it.

Generally speaking, do you think it is possible to access the NPU's sections of the RAM with the CPU in order to extract intermediate results for training? 

Thanks again for all your help,

Amir

[Julian E.]

Hi @amirkfir,

The tool I recommended does not generate libraries using the NPU and you also don't have access to the source code of these models. 

This is one way to use AI within ST, more focus on embedded engineer who don't want to dev a model.

The other way being the ST Edge AI core and deploying a model.

It is not possible to access the NPU's sections of the RAM to extract intermediate results.

The possibility to retrain a model is something we study but it is still R&D at this point.

So sadly, retraining is not currently possible.

Have a good day,

Julian