Not able to deploy Instance Segmentation Model on N6.

[athern27]

Hi everyone, I am trying to deploy a yolov8seg instance segmentation my STM32N6570-DK board, but I am facing some issues. I am getting this error.

(st_zoo) kartikkhandewal@atl-hpzg14-99:~/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src$ python3 stm32ai_main.py --config-name deployment_n6_yolov8n_seg_config-custom.yaml 
2025-04-28 16:10:37.307414: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-12.6/lib64:
2025-04-28 16:10:37.307440: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
[WARNING] The usable GPU memory is unlimited.
Please consider setting the 'gpu_memory_limit' attribute in the 'general' section of your configuration file.
[INFO] : Running `deployment` operation mode
[INFO] : Found 80 classes in label file ../datasets/coco_classes.txt
[INFO] : The random seed for this simulation is 123
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
[INFO] : Generating C header file for Getting Started...
out_shape2 = [   1   37 1344]
len(out_shape2) = 3
Error executing job with overrides: []
Traceback (most recent call last):
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/stm32ai_main.py", line 135, in <module>
    main()
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/main.py", line 94, in decorated_main
    _run_hydra(
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
    raise ex
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
    return func()
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/stm32ai_main.py", line 122, in main
    process_mode(mode=cfg.operation_mode, configs=cfg)
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/stm32ai_main.py", line 65, in process_mode
    deploy(cfg=configs)
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/../deployment/deploy.py", line 66, in deploy
    gen_h_user_file_n6(config=cfg, quantized_model_path=model_path)
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/./utils/gen_h_file.py", line 133, in gen_h_user_file_n6
    n_masks = out_shape2[3]
IndexError: index 3 is out of bounds for axis 0 with size 3

View more

Steps to reproduce-

1. First I trained my model using yolov8n-seg model using this script. Followed this tutorial.

from ultralytics import YOLO

# Load a model
model = YOLO("yolov8n-seg.pt")  # Note the corrected model file name

# Train the model with adjusted batch size and learning rate
train_results = model.train(
    data="/home/kartik/yolov8/datasets/segment/floor-detection/data.yaml",
    project="yolo8n-seg",
    epochs=200,  # number of training epochs
    imgsz=256,  # training image size
    device=0,  # device to run on
    batch=64,  # batch size
    lr0=1e-6, 
    lrf=1e-4,
    weight_decay=0.0005,
    warmup_epochs=5,
    multi_scale=False
)

2. After this I exported the model in 'saved_model' format using this script. I followed this tutorial.

from ultralytics import YOLO
model = YOLO('best.pt')
# Export the model in TFLite format
model.export(format='saved_model', int8=True, imgsz=[256,256], data="/home/kartikkhandewal/stm32packages/quantization_ultralytics/datasets/floor_segmentation/data.yaml")	

3. After this I deployed my model. Followed this tutorial. and I selected the integer_quant.tflite model as mentioned here.
My deployment_n6_yolov8n_seg_config-custom.yaml is attached

python3 stm32ai_main.py --config-name deployment_n6_yolov8n_seg_config-custom.yaml

Note: This model only has 1 class. Also I was able to deploy pretrained st model (coco dataset) from here. Also I tried to debug the issue so I added print comments and this is what i got.
On my model

out_shape2 = [   1   37 1344]
len(out_shape2) = 3

On ST pretrained model

out_shape2 = [ 1 64 64 32]
len(out_shape2) = 4

Kindly help.
All the files will be in this drive. 

OS: Ubuntu 22.04

[athern27]

Any Updates regarding this?
Also I tried it with yolov8-seg.pt model (official) and it is still giving me error.

[athern27]

https://github.com/STMicroelectronics/stm32ai-modelzoo-services/issues/7

Created this issue

[athern27 · Accepted solution]

Solved

The issue was related to mismatched TensorFlow and ONNX dependencies. I am sharing the environments I used during model development and deployment:

• Training Environment:
  ultra_train_env — Python 3.11

• Quantization Environment:
  ultra_quant_env — Python 3.10

Note: You may encounter installation errors while setting up these environments manually. Please resolve them as they appear.

Also, for reasons unclear to me, the training environment did not produce a correctly quantized model, and conversely, the quantization environment failed to train the model properly. So it's important to keep these steps separate and use the environments specifically for their intended purpose.

[Julian E.]

Hello @athern27,

Sorry for not responding to your question. I must have forgot about it.

Concerning the issue regarding environment, I would suggest to strictly follow the instructions from model zoo here:

GitHub - STMicroelectronics/stm32ai-modelzoo-services: AI Model Zoo services for STM32 devices

It is written to use python 3.10 and I think it could be the reason you have errors.

I did some test also in 3.11 and got troubles.

Have a good day,

Julian