How to integrate TouchGFX with an AI model from Modelzoo Services and Camera Middleware on the STM32N6570-DK

[TatianaP]

Hi! 

I have already trained my AI model using the STM32 Model Zoo Services, and the project is fully running on the STM32N6570-DK board. The model has been successfully deployed, and the object detection is working correctly with the camera.

In parallel, I have also developed the graphical user interface using TouchGFX as a separate project. At this point, my goal is to integrate both projects into a single application: the TouchGFX interface and the AI inference application.

I have read your article about integrating TouchGFX with the Camera Middleware, which was very helpful.

However, I could not find information about integrating TouchGFX with an AI application generated from the STM32 Model Zoo Services.

What I would like to achieve is the following workflow: the application starts with the TouchGFX interface, and when the user presses a button, the camera is initialized, the AI inference pipeline starts, and the model begins detecting objects in real time.

Could you please advise on the recommended approach for integrating a TouchGFX application with an AI project generated by the STM32 Model Zoo Services? Are there any examples, guidelines, or best practices for combining these two applications on the STM32N6570-DK?

[TatianaP]

Please, anyone? I really need this information for my project at my job

[mƎALLEm]

Hello ​@TatianaP ,

The other post with the same content has been removed.

In next time please don’t duplicate threads. You can continue in one thread by posting reminder posts. Your post will be seen at the top queue of the community home page.

Thank you for your understanding.

[JohanAstrup]

Hello ​@TatianaP.

First of all, I should mention that I am not very familiar with STM32 Model Zoo. However, could you be more specific about what is missing from the guide you are referring to? Are you looking for more information on how to do this the other way around? In other words, instead of basing an AI project on the TouchGFX Board Setup available in TouchGFX Designer, do you want to integrate TouchGFX into an existing AI project from STM32 Model Zoo?

If so, I would recommend reading this section of the TouchGFX documentation: https://support.touchgfx.com/docs/development/touchgfx-hal-development/touchgfx-al-development-introduction.

Best regards,
Johan

[TatianaP]

Thank you for your reply ​@JohanAstrup 

Yes, my goal is actually the opposite of the article (https://community.st.com/stm32%2Dmcus%2D60/how%2Dto%2Dintegrate%2Dtouchgfx%2Dand%2Dcamera%2Dmiddleware%2Don%2Dan%2Dstm32n6570%2Ddk%2D159856)

I already have a fully working STM32 Model Zoo Object Detection project running on the STM32N6570-DK. The camera, ISP, NPU, inference and post-processing are all working correctly.

I also have a separate TouchGFX project with my complete user interface.

What I would like to achieve is to integrate TouchGFX into the existing STM32 Model Zoo project, instead of starting from a TouchGFX project and adding the camera middleware.

The main difficulty is that the Model Zoo project already initializes many components (camera pipeline, NPU, AI runtime, post-processing, display pipeline, ThreadX, etc.), so migrating all of that into a TouchGFX-generated project seems considerably more complex than integrating TouchGFX into the existing AI project.

I also noticed that the Object Detection example already uses both LTDC layers:

• Layer 1 for the camera preview.
• Layer 2 for the detection overlay (bounding boxes and labels).

Because of this, I am not sure what would be the recommended architecture for integrating TouchGFX.

Would ST recommend:

1. Starting from the STM32 Model Zoo project and integrating TouchGFX into it?
2. Starting from the TouchGFX Board Setup and migrating all the AI components?
3. Another architecture that is recommended for STM32N6?

My goal is to keep the TouchGFX GUI active (menus, buttons, navigation) while running the Object Detection demo.

[TatianaP]

I also have this doubt ​@JohanAstrup , in an STM32N6 application combining TouchGFX and STM32 Model Zoo Object Detection, should the AI application continue drawing the bounding boxes directly into the LTDC overlay framebuffer (as Display_NetworkOutput() does), or is the recommended architecture to pass the detection results to TouchGFX and let TouchGFX perform all the rendering?

[JohanAstrup]

I agree that TouchGFX sounds easier to integrate into your Model Zoo project than doing it the other way around, based on your description. If you follow this procedure from the link I shared in the previous reply, you get TouchGFX enabled in your project.

However, the STM32N6 is an advanced chip with many capabilities, so I would recommend using the TouchGFX Board Setups (TBSs) available in TouchGFX Designer as reference when integrating TouchGFX. Settings like e.g. the MPU configuration are important.

As I see it, you have multiple options, but the cleanest approach would be to pass the camera feed to TouchGFX using a dynamic bitmap. You could then also draw the bounding boxes in TouchGFX. Whether you choose to do this is up to you. It would result in a cleaner project, but since you already have the other solution where the overlay exists in a different layer, you would likely be done faster by keeping that approach. That said, it should be fairly simple to pass the information and draw the boxes in TouchGFX.

Best regards,
Johan