No video stream for Python/C++ OD demo with COCO SSD MobileNet v1 TFLite on STM32MP157D-EV1

[ggreg]

Hi,

I'm attempting to test some Object Detection demo models from the X-LINUX-AI expansion package on the STM32MP157D-EV1 eval kit with a live camera video stream. However, I'm currently having what seems like a streaming problem with both the C++ or Python apps (refered to as "Object Detection COCO SSD v1"). When launching those apps on the device, the app window opens, but the stream never starts.

Hence, to investigate the issue, I performed some manual GStreamer testing with command lines. It seems that with the CSI camera part of the kit, the stream will not start when the required data transfer rate is too high. For example, the stream will start with the following parameters: "video/x-raw, width=720, height=480, framerate=(fraction)15/1", but not with 30 fps with the same resolution. When we reduce the resolution to "width=320, height=240", we can reach 30 fps. Furthermore, when using a USB third party camera, the streaming limitation does no occur.

Please find a few questions below to help troubleshooting:

- I see on the meta-st-x-linux-ai GitHub that the demo package was tested on the STM32MP157F-EV1 eval kit with its built-in camera module. Could you confirm that this package is also compatible with STM32MP157D-EV1?

- Is there a data rate limitation on the DCMI interface that would explain why the CSI camera cannot work at higher rates than the above threshold?

- Any insight on the next troubleshooting steps would be helpful.

Thanks.

[Olivier GALLIEN]

Hi @ggreg ,

You mention CSI kit, but EV1 don't accept natively CSI.  Can you please specify the camera module you are using ? 

Please can you also details SW version you are using. 

Thanks 

Olivier 

[ggreg]

Hi Olivier, 

Sorry, I meant DCMI. I'm using the built-in camera module part of the eval kit (MB1379).

Regarding SW, my flash package is: en.flash-stm32mp1-openstlinux-6.1-yocto-mickledore-mp1-v23.06.21.tar.gz

And the AI package: http://extra.packages.openstlinux.st.com/AI/5.0/pool/config/a/apt-openstlinux-ai/apt-openstlinux-ai_1.0_armhf.deb

Let me know if more details on the SW are required.

Thanks,

Mathieu

[Olivier GALLIEN · Accepted solution]

Hi @ggreg 

Clear so this confirm our main assumption. 

You probably face known bug in this version around DMA for DCMI. 

Please see https://wiki.st.com/stm32mpu/wiki/STM32_MPU_OpenSTLinux_release_note#Minor_release_updates 

And note the related change integrated in 5.0.1 .. 

I recommend then to switch on 5.0.2 openstlinux-6.1-yocto-mickledore-mp1-v23.11.15. 

Looking forward your confirmation it fix the issue you have. 

Olivier 

[ggreg]

Hi Olivier,

I tried to update to 5.0.2 using the "repo" method, but bumped into a few issues:

• At the "bitbake st-image-weston" step (which took around 2 hours to run) the following error occured:

ERROR: Task (/.../Documents/STM32MP1_Flash/Distribution-Package/layers/openembedded-core/meta/recipes-support/boost/boost_1.81.0.bb:do_compile) failed with exit code '1'

This error could be related to a RAM depletion on host PC according to https://community.toradex.com/t/boost-do-compile-error/16872/2. However, my PC is brand new and has 16 GiB of RAM (?)...

• Ignoring this error; At the flashing step, the .tvs files were nowhere to be found in the image folder, so I copied the ones from the starter packages in the "flashlayout_st-image-weston" folder to test. However, when running below command, I have following error:

STM32_Programmer_CLI -c port=usb1 -w flashlayout_st-image-weston/optee/FlashLayout_sdcard_stm32mp157d-ev1-optee.tsv

Error: File does not exist: .../Distribution-Package/build-openstlinuxweston-stm32mp1/tmp-glibc/deploy/images/stm32mp1/flashlayout_st-image-weston/optee/arm-trusted-firmware/tf-a-stm32mp157d-ev1-usb.stm32
Error: TSV flashing service failed
 The missing file does not exist in the Starter-Package.

Bottom line is I did not have much luck to test the 5.0.2 patch today. Would you recommend to go with the GitHub method instead of the "repo" method to update the package?

[ggreg]

If it can help, here is a log of a second build attempt with "bitbake st-image-weston" command. The same error with boost_1.81.0.bb occured.

I will now try to install the:

penstlinux-6.1-yocto-mickledore-mp1-v23.06.21

Distribution package instead of minor release v23.11.15 to see if it helps.

Thanks.

[Olivier GALLIEN]

Hi @ggreg ,

Such kind of error may come from HOST system not properly configured. 

Did you follow PC prerequisites - stm32mpu and install all required packages ?

Else did you build default config as is or added the X-LINUX-AI ? 

Olivier

[ggreg]

Hi Olivier,

The problem occurred when running the bitbake command line to generate the flash files as is (i.e. without the X-LINUX-AI).

Here is an update on both the bitbake build problem and the OD demo app problem:
- I made sure that all the prerequisites were installed by rerunning the commands. I think everything was good already as no new library was installed. As I suspected some RAM issues to cause the "boost" error during the build process, I increased the swap memory from 2 GB to 8 GB, which brought the total memory to 24 GB (16 GB RAM + 8 GB swap). During the build process (with "bitbake st-image-weston" command), I noticed that the total memory usage went up to 19 GB at some point. This time though, the script was able to complete successfully without the "boost" error, which points out to insufficient memory as the RC. FYI I'm using Ubuntu 20.04.

- After flashing the v23.11.15 minor update and reinstalling X-LINUX-AI, I was able to confirm that this patch indeed fixes the original streaming of this post.  I was able to run the OD demo both with Python and C++ versions. Thus, I will mark your previous answer as the valid solution for this issue.

Thanks for your support,

Mathieu