Command Line Interface for TFLM

[Jeff Teng]

Hi everyone,

For the command line interface of stm32ai, it has three commands of analyze, validate, and generate. Therefore, I can use the command to generate network and data C-files. However, for the command line interface of stm32tflm, it can only analyze RAM and Flash usage of the input model.

My question is how to use command line interface to generate network and data C-files based on TFLM runtime? And these generated files can be used for the following compilation process like stm32ai. Besides, how can I use the tag to on/off the usage of CMSIS-NN?

Thank you very much for your reply!

Best regards,

Jeff

[fauvarque.daniel]

The code generation for the tensorflow lite runtime is performed by the X-Cube-AI STM32CubeMX plugin itself.

it consist of copying all the tensorflow lite micro runtime sources, i.e. the content of Middleware/tensorflow in the pack, in the project and transforming the tflite file into a c file

The C file is an array (g_tflm_network_model_data) that contains a hexadecimal dump of the tflite file and "g_tflm_network_model_data_len" that contains the length

The plugin always generates with the usage of the CMSIS-NN, if you need another generation option you'll have to generate the project from the tensorflow git

Regards

Daniel

[Jeff Teng]

Hi Daniel,

Thank you for your reply. Could you provide more instructions about how to use command line interface to generate the C-files?

Besides, based on your suggestions, if I use tensorflow git to generate the C-files, are the generated C-files still compatible with X-Cube-AI? Because X-Cube-AI provides a user-friendly interface for model validation, I want to keep using it.

Thank you for your help.

Best regards,

Jeff

[fauvarque.daniel]

We don't have command line to fully create the entire project, it is done in the UI.

What you can do is do the project once with X-Cube-AI, select the Validation application and generate the code. It will generate a project compatible with the stm32ai command line for validation on the target

You can also do automatic validation on the target with the UI using the TFLite runtime.

When the project has been generated once, you can manually update the network c and h files with a new network if you don't want to use the UI anymore.

Regards

Daniel

[Jeff Teng]

Hi Daniel,

Thank you for your reply. Sorry for a stupid question, how can I use tensorflow git to generate the network c and h files? Or will you recommend other tools like MBed?

If I use the Makefile in tensorflow git, it seems that it will directly generate a final bin file?

Another small question: is the CMSIS-NN related to the generated c and h files, which means it must be provided when generating c and h files? Or it is used when compiling the final bin file?

Thank you for your help.

Best regards,

Jeff

[fauvarque.daniel]

In the git there is a script to generate a runtime with all the files, in tensorflow 2.5 it was (with cmsis_nn enabled)

python tensorflow/lite/micro/tools/project_generation/create_tflm_tree.py DESTINATION --makefile_options "OPTIMIZED_KERNEL_DIR=cmsis_nn DISABLE_DOWNLOADS=true�?

I didn't check with more recent versions

[Jeff Teng]

Hi Daniel,

Thank you for your instructions. I can successfully generate the folder now and I guess it is the same as the folder in STM32CubeIDE under Middlewares/tensorflow. Is it right?

So if I want to generate model files without using CMSIS-NN, I can generate another runtime without setting OPTIMIZED_KERNEL_DIR?

For the final question, based on this generated runtime, how can I use it to transform .tflite file to its network.c and network_tflite_data.h files, like X-Cube-AI does?

Thank you very much for your assistance.

Best regards,

Jeff

[fauvarque.daniel]

The transformation of the tflite file is done "by hand", you can write a python script, a bash shell or a C program on the PC.

g_tflm_network_model_data is an hexacdecimal dump of the file

g_tflm_network_model_data_len is the length of the array

and TFLM_NETWORK_TENSOR_AREA_SIZE if you use it is the output of the stm32tlfm command Ram line

[jean-michel.d]

Hi Jeff,

In attachment, you can find a python script allowing to generate c/h file compliant with those generated by X-CUBE-AI.

Note that as mentioned by Daniel, we need to provide the expected size of the tensor area (-ts option in KB) and if you use the tfm_c.cc/tflm_c.h wrapper, list of the registered operators should be updated or you can set the TFLM_RUNTIME_USE_ALL_OPERATORS C-define.

/* tfm_cc.c file */
#if defined(TFLM_RUNTIME_USE_ALL_OPERATORS) && TFLM_RUNTIME_USE_ALL_OPERATORS == 1
  static tflite::AllOpsResolver _resolver;
#else
    static tflite::MicroMutableOpResolver<3> _resolver;
    _resolver.AddAveragePool2D();
    _resolver.AddConv2D();
    _resolver.AddDepthwiseConv2D();
 #endif

Other way to replace the CMS-NN-based operator by the non-optimized version in the generated source-tree case-by-case:

For example, to replace the Middlewares\tensorflow\tensorflow\lite\micro\kernels\cmsis_nn\add.cc operator by the default version:

• copy the add.cc file from the original repo source tree (tfv2.5) in Middlewares\tensorflow\tensorflow\lite\micro\kernels\
• remove the Middlewares\tensorflow\tensorflow\lite\micro\kernels\cmsis_nn\add.cc (or exclude this file from the build system)

br,

Jean-Michel

[Jeff Teng]

Hi Daniel and Jean-Michel,

Thanks very much for your clear description. It helps me a lot.

Best regards,

Jeff