STM32MPU25 object detection model failing to run python scripts

[akr1]

Hi

I am using STM32MPU25 to evaluate object detection, when i use stai_mpu_object_detection binary to run default ssd_mobilenet_v2 models , i can see object detection works.

when i run python scripts to launch application( example : launch_bin_object_detection_testdata.sh) i see below error.

root@stm32mp25-discounknownunknown:/usr/local/x-linux-ai/object-detection# sh launch_python_object_detection.sh
stai_mpu wrapper used :
machine used = stm32mp2x with GPU/NPU
user : weston
-sh: /usr/local/x-linux-ai/object-detection/stai_mpu_object_detection.py: Permission denied

i assume it may be due to weston user and root user.

so i copied all objection detection samples to /home/weston and gave appropriate permissions and change user to weston and launch application, i see below errors.

^^^^^^^^^^^^^^^^^^^^^
File "/home/weston/test/Application/ssd_mobilenet_pp.py", line 129, in get_results
return locations, classes, scores
^^^^^^^^^
UnboundLocalError: cannot access local variable 'locations' where it is not associated with a value
Traceback (most recent call last):
File "/home/weston/test//Application/object_detection.py", line 336, in new_sample
self.app.nn_result_locations, self.app.nn_result_classes, self.app.nn_result_scores = self.nn.get_results()
^^^^^^^^^^^^^^^^^^^^^
File "/home/weston/test/Application/ssd_mobilenet_pp.py", line 129, in get_results
return locations, classes, scores

can you point to environment where i can evaluate object detection models ( i followed procedure at https://github.com/STMicroelectronics/stm32ai-modelzoo-services/tree/main/object_detection  to create object detection model  with custom data set and deploy into stm32mpu25)

Thanks

[akr1]

Hi,

Attached models zip file.

Thanks

[Julian E.]

Hello @akr1,

The dev team gave me feedback; here is what should be done:

1. You need to create a .txt with your classes, for example "labels_custom.txt".
   The class "background" should be first.
2. Change your model's name for it to include "ssd_mobilenet_v2", for example "ssd_mobilenet_v2_custom.nb". 
   The scripts look for the kind of model used to apply the right preprocessing.

Then in  launch_python_object_detection.sh, edit the model and label:

• $OBJ_DETEC_MODEL  --> ssd_mobilenet_v2_custom.nb 
• $OBJ_DETEC_MODEL_LABEL.txt --> custom_label.txt

This should help.

In case you had already done that, please share some examples images for us to test.

Seeing your classes, we did not really understand what the classes were and how to test them.

Have a good day,

Julian

[akr1 · Accepted solution]

Adding background class started detecting objects properly in stm32mpu257.

couple of questions i have 

1) while running launch_python_object_detection.sh  i see below error

while running  File "/home/weston/object-detection//Application/object_detection.py", line 679, in drawing
y0 = int(self.app.nn_result_locations[0][i][1] * preview_height)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: index 3 is out of bounds for axis 0 with size 3
Traceback (most recent call last):
File "/home/weston/object-detection//Application/object_detection.py", line 679, in drawing
y0 = int(self.app.nn_result_locations[0][i][1] * preview_height)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: index 3 is out of bounds for axis 0 with size 3
Traceback (most recent call last):
File "/home/weston/object-detection//Application/object_detection.py", line 679, in drawing
y0 = int(self.app.nn_result_locations[0][i][1] * preview_height)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: index 3 is out of bounds for axis 0 with size 3 

2) does tiny yolo work in stm32mpu25 for object detection , if it works can you point to README or any documentation ?

[Julian E.]

Hi  @akr1,

I am glad it helped.

Concerning your error, could you print the following shapes please:

• self.app.nn_result_classes[0]
• self.app.nn_result_locations[0]
• self.app.nn_result_scores[0]

Then regarding the use of tiny yolo on MP2, unfortunately, we don't have documentation regarding that.
I was told that they choose to support the ssd mobilenet because of its better performances on the MP2.

If you want to use a tiny yolo, you need to create a tiny_yolo_pp.py that respect the architecture of the ssd_mobilenet_pp.py.
It should contain all the information for the function set input, get input and the pre/post processing.

Have a good day,

Julian

[akr1]

Hi,

find below debug prints.

IndexError: index 0 is out of bounds for axis 0 with size 0
Traceback (most recent call last):
File "/home/weston/object-detection//Application/object_detection.py", line 337, in new_sample
print("self.app.nn_result_classes is",self.app.nn_result_classes[0]);
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: index 0 is out of bounds for axis 0 with size 0
Traceback (most recent call last):
File "/home/weston/object-detection//Application/object_detection.py", line 337, in new_sample
print("self.app.nn_result_classes is",self.app.nn_result_classes[0]);
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: index 0 is out of bounds for axis 0 with size 0
self.app.nn_result_classes is [3]
self.app.nn_result_locations is [[0.22344506 0.5551508 0.5069047 0.74899846]]
self.app.nn_result_scores is [0.796875]
self.app.nn_result_classes is [3]
self.app.nn_result_locations is [[0.24382773 0.5551508 0.5171083 0.74899846]]
self.app.nn_result_scores is [0.7734375]
self.app.nn_result_classes is [3]
self.app.nn_result_locations is [[0.24382773 0.5551508 0.52728736 0.74899846]]
self.app.nn_result_scores is [0.796875]
self.app.nn_result_classes is [3]
self.app.nn_result_locations is [[0.24382773 0.5551508 0.5171083 0.74899846]]
self.app.nn_result_scores is [0.9375]
self.app.nn_result_classes is [2 3]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.92673284]
[0.29108444 0.5799015 0.4676358 0.6948684 ]]
self.app.nn_result_scores is [0.99609375 0.94921875]
self.app.nn_result_classes is [2 3]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.92673284]
[0.29882812 0.58935547 0.45989212 0.69077754]]
self.app.nn_result_scores is [0.99609375 0.94921875]
self.app.nn_result_classes is [2 3]
self.app.nn_result_locations is [[0.3244378 0.7524409 0.46970147 0.9267583 ]
[0.27736875 0.58956534 0.484375 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.8125 ]
self.app.nn_result_classes is [2 3]
self.app.nn_result_locations is [[0.3149414 0.7426758 0.47942322 0.92673284]
[0.29108444 0.58935547 0.45989212 0.6981311 ]]
self.app.nn_result_scores is [0.99609375 0.8125 ]
self.app.nn_result_classes is [2 3]
self.app.nn_result_locations is [[0.3244378 0.7524409 0.46970147 0.9267583 ]
[0.28334075 0.5876558 0.4676358 0.6948684 ]]
self.app.nn_result_scores is [0.99609375 0.94921875]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.3244378 0.7524409 0.46970147 0.9267583 ]
[0.27736875 0.5891456 0.47336367 0.68882704]
[0.484375 0.60546875 0.6731271 0.69984484]]
self.app.nn_result_scores is [0.99609375 0.8125 0.796875 ]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.92673284]
[0.27736875 0.58384657 0.484375 0.69412607]
[0.4949794 0.60546875 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.80859375]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.92673284]
[0.29882812 0.58935547 0.45989212 0.6981311 ]
[0.4949794 0.60546875 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.80859375]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.3244378 0.74512094 0.4623691 0.9267583 ]
[0.29108444 0.58935547 0.45989212 0.6981311 ]
[0.4949794 0.60016656 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.796875 ]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.92673284]
[0.28334075 0.5799015 0.45989212 0.6948684 ]
[0.4949794 0.60546875 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.80859375]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.33177456 0.74512094 0.4623691 0.9267583 ]
[0.28334075 0.5799015 0.45989212 0.6948684 ]
[0.4949794 0.60546875 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.796875 ]
self.app.nn_result_classes is [2 3]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.92673284]
[0.29108444 0.58935547 0.45989212 0.69077754]]
self.app.nn_result_scores is [0.99609375 0.94921875]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.91694224]
[0.29882812 0.58935547 0.45214844 0.6981311 ]
[0.4949794 0.60016656 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.80859375]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.3244378 0.74512094 0.4623691 0.9267583 ]
[0.29108444 0.5799015 0.45989212 0.6948684 ]
[0.4949794 0.60546875 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.80859375]
self.app.nn_result_classes is [2 3]
self.app.nn_result_locations is [[0.33177456 0.74512094 0.4623691 0.9267583 ]
[0.29108444 0.58935547 0.45989212 0.6981311 ]]
self.app.nn_result_scores is [0.99609375 0.94921875]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.92673284]
[0.28334075 0.5799015 0.4676358 0.6948684 ]
[0.4949794 0.60546875 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.9453125 ]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.91694224]
[0.28334075 0.5876558 0.45989212 0.6948684 ]
[0.4949794 0.60546875 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.9453125 ]
self.app.nn_result_classes is [2 3]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.91694224]
[0.29108444 0.58935547 0.45214844 0.69077754]]
self.app.nn_result_scores is [0.99609375 0.94921875]
self.app.nn_result_classes is [2 3 4]
self.app.nn_result_locations is [[0.324732 0.7426758 0.46962672 0.91694224]
[0.28334075 0.5799015 0.45989212 0.6948684 ]
[0.4949794 0.60016656 0.6731271 0.6945458 ]]
self.app.nn_result_scores is [0.99609375 0.94921875 0.80859375]

Thanks

[Julian E.]

Hi @akr1,

It seems that the application is running as it should on the last prints.

At the beginning, there is an error in the case where no object is detected and the function to draw the boxes don't have anything to do.

You should be able to fix it adding this code before the loop that does the drawing (around line 673)

num_detections = len(self.app.nn_result_scores)
if num_detections == 0:
    return True  # No detections to draw

Have a good day,

Julian

[akr1]

Thanks it works.