object detection using USB UVC camera in stm32n6570-dk

[Thiha2025]

I have STM32N6570-DK board and need to use very small sized camera ~1x1mm 30 fps with USB UVC interface for object detection applications. As the camera does not have dcmi/csi-2 interface, I am exploring how to use USB HS Host driver and UVC class driver. Questions:

1. Is it possible to get real-time inference on UVC video stream(20-30fps) as in an example ST      provides? I have tried the example and inferencing speed is very fast.

2. Are there any example codes I can refer to for USB host and vuc class driver? 

3. or should I try find the video bridging hardware converter(UVC to DCMI parallel/MIPI CSI-2)?

4. Even if I can find CSI-2 converter, firmware development will still be very challenging for CSI interface. Is it worth trying out? DCMI parallel will be much easier to implement in coding. However, small camera does not come with parallel interface. Are my assumptions correct?

Any help/guidance/clarification would be greatly appreciated!

Thank you

[FBL]

Hi @Thiha2025 

For host application, you can refer to this example.

For example, the Yolov8n (256x256x3) model achieves an inference time of 35.6 ms per frame (about 28 inferences per second), and TinyYolov2 (224x224x3) achieves 31.4 ms per frame (about 32 inferences per second) under optimal conditions (Cortex-M55 at 600 MHz, NPU at 800 MHz, VDDCORE at 0.81 V). Check this wiki: AI:STM32Cube.AI model performances - stm32mcu

Here is another example firmware with trained model TinyYolov2.

Processing a USB UVC video stream will depend on the combined time for video acquisition, pre-processing, inference, and any post-processing or display steps.