Not able to deploy Instance Segmentation Model on N6.

[athern27]

Hi everyone, I am trying to deploy a yolov8seg instance segmentation my STM32N6570-DK board, but I am facing some issues. I am getting this error.

(st_zoo) kartikkhandewal@atl-hpzg14-99:~/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src$ python3 stm32ai_main.py --config-name deployment_n6_yolov8n_seg_config-custom.yaml 
2025-04-28 16:10:37.307414: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-12.6/lib64:
2025-04-28 16:10:37.307440: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
[WARNING] The usable GPU memory is unlimited.
Please consider setting the 'gpu_memory_limit' attribute in the 'general' section of your configuration file.
[INFO] : Running `deployment` operation mode
[INFO] : Found 80 classes in label file ../datasets/coco_classes.txt
[INFO] : The random seed for this simulation is 123
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
[INFO] : Generating C header file for Getting Started...
out_shape2 = [   1   37 1344]
len(out_shape2) = 3
Error executing job with overrides: []
Traceback (most recent call last):
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/stm32ai_main.py", line 135, in <module>
    main()
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/main.py", line 94, in decorated_main
    _run_hydra(
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
    raise ex
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
    return func()
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/st_zoo/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/stm32ai_main.py", line 122, in main
    process_mode(mode=cfg.operation_mode, configs=cfg)
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/stm32ai_main.py", line 65, in process_mode
    deploy(cfg=configs)
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/../deployment/deploy.py", line 66, in deploy
    gen_h_user_file_n6(config=cfg, quantized_model_path=model_path)
  File "/home/kartikkhandewal/stm32packages/stm32ai-modelzoo-services/instance_segmentation/src/./utils/gen_h_file.py", line 133, in gen_h_user_file_n6
    n_masks = out_shape2[3]
IndexError: index 3 is out of bounds for axis 0 with size 3

View more

Steps to reproduce-

1. First I trained my model using yolov8n-seg model using this script. Followed this tutorial.

from ultralytics import YOLO

# Load a model
model = YOLO("yolov8n-seg.pt")  # Note the corrected model file name

# Train the model with adjusted batch size and learning rate
train_results = model.train(
    data="/home/kartik/yolov8/datasets/segment/floor-detection/data.yaml",
    project="yolo8n-seg",
    epochs=200,  # number of training epochs
    imgsz=256,  # training image size
    device=0,  # device to run on
    batch=64,  # batch size
    lr0=1e-6, 
    lrf=1e-4,
    weight_decay=0.0005,
    warmup_epochs=5,
    multi_scale=False
)

2. After this I exported the model in 'saved_model' format using this script. I followed this tutorial.

from ultralytics import YOLO
model = YOLO('best.pt')
# Export the model in TFLite format
model.export(format='saved_model', int8=True, imgsz=[256,256], data="/home/kartikkhandewal/stm32packages/quantization_ultralytics/datasets/floor_segmentation/data.yaml")	

3. After this I deployed my model. Followed this tutorial. and I selected the integer_quant.tflite model as mentioned here.
My deployment_n6_yolov8n_seg_config-custom.yaml is attached

python3 stm32ai_main.py --config-name deployment_n6_yolov8n_seg_config-custom.yaml

Note: This model only has 1 class. Also I was able to deploy pretrained st model (coco dataset) from here. Also I tried to debug the issue so I added print comments and this is what i got.
On my model

out_shape2 = [   1   37 1344]
len(out_shape2) = 3

On ST pretrained model

out_shape2 = [ 1 64 64 32]
len(out_shape2) = 4

Kindly help.
All the files will be in this drive. 

OS: Ubuntu 22.04

[athern27]

Any Updates regarding this?
Also I tried it with yolov8-seg.pt model (official) and it is still giving me error.

[athern27]

https://github.com/STMicroelectronics/stm32ai-modelzoo-services/issues/7

Created this issue