What do i need to do to be able to use a RNN that uses a .pkl vocab/mapping?

[lyannen]

Hello,

im currently working on utilizing a Keras RNN (seq2seq model) on the STM32N6570-DK to generate text from a certain input.
The inference of the .h5 model on my laptop works with a .pkl file that has mapping of the input to the output. When converting the model to tflite the .pkl isnt included.

So my question ist, what do i need to do, to convert the .pkl mappings to make use of it when im doing the inference of the model with the generated C-Code from stedgeai?

I wasnt able to find a lot of information on this topic so far, so im thankful, if anyone would be able to help me in this case!

Kind regards!

[Julian E. · Accepted solution]

Hello @lyannen,

I am not really familiar with pkl file, but here is what I understood after a bit of research.

You should extract the mapping from your pkl file, and then export these mappings as flat arrays or C-style dictionaries (directly write them into a .h file), something like:

(this code is AI generated)

import pickle

with open("mymapping.pkl", "rb") as f:
    mappings = pickle.load(f)

char_to_index = mappings["char_to_index"]
index_to_char = mappings["index_to_char"]


# Export as C arrays
with open("char_map.h", "w") as f:
    f.write("const char* index_to_char[] = {\n")
    for i in range(len(index_to_char)):
        f.write(f'  "{index_to_char[i]}",\n')
    f.write("};\n")

    f.write("const int char_to_index[128] = {\n")  # assuming ASCII range
    for i in range(128):
        char = chr(i)
        idx = char_to_index.get(char, -1)
        f.write(f"  {idx},  // '{char}'\n")
    f.write("};\n")

So that you can:

• Before feeding input into the model: convert your text to int indices using char_to_index.
• After inference: convert model outputs back to characters using index_to_char.

If you use X Cube AI and the Application template. In the main.c, in the Cube_MX_AI_Run() (the name could be different, I don't remember exactly). You have a pre and post process functions. You should edit these template function to do your processing here.

I don't know how you manage tokenizing; you may also have to re-implement minimal tokenizer logic in embedded C.

Did you try to convert your model with the st edge ai core? I know that seq model could face issue during the conversion. Did it work for you?

Have a good day,

Julian

[lyannen]

Hello @Julian E. ,

thanks for your quick reply! The information is very helpful and i will try this approach once i was able to convert the model to C-code.

Regarding your question if i managed the C-code conversion.
Im currently working on that problem as i used a couple of unsupported layers in my seq2seq model:
NOT IMPLEMENTED: Unsupported layer types: WHILE, FlexTensorListReserve, FlexTensorListStack, FlexTensorListFromTensor, stopping.

If youve got any advice on how to achieve that or some information on what would be the same layer functionwise but that is supported by stedgeai i would be very thankful. For now im trying to change my model to supported layers without losing the functionality.

Kind regards!

[Julian E.]

@lyannen,

As of today, I think that seq2seq is difficult to do.

As you saw, we still need to work on adding a lot of layers required for such models.

I don't really have any advice sadly. You should, as you are doing, try to find models/edit models that use supported layers.

You can maybe ask ChatGPT for a list of simple seq2seq model architecture that tries to respect this list of supported layers: https://stedgeai-dc.st.com/assets/embedded-docs/supported_ops_keras.html 

We don't have any example on our side regarding seq2seq, so know that you kind of are outside of what we currently support. So, if you don't find suitable model, I don't think that I will be able to help you other than telling you to wait for the support of your required layers.

Would you mind sharing the use case behind the use of seq2seq model (in private message if you want).

We define the roadmap based on clients need and I don't recall any users working with seq2seq model.

Have a good day,

Julian

[lyannen]

Thanks for your nice message, i will hit you up with a direct message later today and try to explain the need for it!
Kind regards!