Data Format Output by YOLOv8n_float Model Running on Cube.AI via STM32N6570-DK

[实在太懒于是不想取名]

Dear ST staff, Recently, while using STM32CubeMX with CubeAI to deploy the yolov8n_192_quant_pc_uf_od_coco-person.tflite model from object_detection onto the STM32N6570-DK development board.

        Subsequently, following CubeAI's configuration, I programmed the network_atonbuf.xSPI2.bin file into the 0x71000000 memory location

It is worth noting that this model is of type float, not int8 quantisation (as I understand it), and both member pointers in

LL_Buffer_InfoTypeDef's

const float *scale; /**< */

const int16_t *offset;

are NULL. This should indicate that the output requires no normalisation conversion.

void MX_X_CUBE_AI_Process(void)
{
    /* USER CODE BEGIN 6 */
    LL_ATON_RT_RetValues_t ll_aton_rt_ret = LL_ATON_RT_DONE;
    const LL_Buffer_InfoTypeDef * ibuffersInfos = NN_Interface_Default.input_buffers_info();
    const LL_Buffer_InfoTypeDef * obuffersInfos = NN_Interface_Default.output_buffers_info();
    buffer_in = (uint8_t *)LL_Buffer_addr_start(&ibuffersInfos[0]);
    buffer_out = (uint8_t *)LL_Buffer_addr_start(&obuffersInfos[0]);
    printf("buffer_in addr=%p\n", buffer_in);
    printf("buffer_in addr mod4=%ld\n", ((uintptr_t)buffer_in)%4);
    printf("buffer_in addr mod8=%ld\n", ((uintptr_t)buffer_in)%8);
    HAL_LTDC_SetAddress(&hltdc, buffer_in, 1);
    HAL_DCMIPP_CSI_PIPE_Start(&hdcmipp, DCMIPP_PIPE2, DCMIPP_VIRTUAL_CHANNEL0 , buffer_in, DCMIPP_MODE_SNAPSHOT);
    HAL_Delay(5);
    LL_ATON_RT_RuntimeInit();

    // run 1 inferences
    for (int inferenceNb = 0; inferenceNb<1; ++inferenceNb) {
        /* ------------- */
        /* - Inference - */
        /* ------------- */
        /* Pre-process and fill the input buffer */
        //_pre_process(buffer_in);

        /* Perform the inference */
        LL_ATON_RT_Init_Network(&NN_Instance_Default);
        do {
            ll_aton_rt_ret = LL_ATON_RT_RunEpochBlock(&NN_Instance_Default);
            if (ll_aton_rt_ret == LL_ATON_RT_WFE) {
                LL_ATON_OSAL_WFE();
            }
        } while (ll_aton_rt_ret != LL_ATON_RT_DONE);

        /* Post-process the output buffer */

        /* ------------------ 替换这里的 Post-process 区块 ------------------ */
        {
			#define CONF_THRESHOLD 0.3f   // 根据模型调试设定

        	int channels  = (int)obuffersInfos->mem_shape[1]; // 5
        	int num_boxes = (int)obuffersInfos->mem_shape[2]; // 756
        	int total_elems = channels * num_boxes;

        	uint8_t *tmp = (uint8_t *)buffer_out;
        	int nn_out_len_bytes = channels * num_boxes * sizeof(float);
        	SCB_InvalidateDCache_by_Addr((uint32_t *)tmp, nn_out_len_bytes);
        	for(int i = 0;i<756;i++)
        	{
        		printf("A:%.2f\r\n",((float*)buffer_out)[i+4*num_boxes]);
        	}

        }



        LL_ATON_RT_DeInit_Network(&NN_Instance_Default);
        /* -------------------- */
        /* - End of Inference - */
        /* -------------------- */
    }
    LL_ATON_RT_RuntimeDeInit();
    /* USER CODE END 6 */
}

    I have attempted numerous methods to parse YOLOv8n data, but frequently encounter anomalous values, particularly when seeking configuration details within the [1x5x756] structure, where many values are disproportionately large.

Is there any relevant ST analysis data documentation available for reference?

[Julian E. · Accepted solution]

Hi @实在太懒于是不想取名,

First of all, the model is quantized in int8, it is just that the input and output are float32 (converted to or from).

Then in your case, I think you should compare the C model output and the original tflite output.

To do so, in a first step, you should use the "validation on target" from X Cube AI or do it manually: 

https://stedgeai-dc.st.com/documentation 

You want to look at the COS, if close to 1 then at least your output the one expected.

Then, maybe you don't print them correctly. To know that I suggest you use the AI Runner to get the output of the C model and compare them to the tflite model

Documentation: https://stedgeai-dc.st.com/assets/embedded-docs/ai_runner_python_module.html

The AI runner can be found here: \ST\STEdgeAI\2.2\scripts\ai_runner

• First, you need to install the requirements: pip install -r requierement.txt.
• Then you need to load the validation firmware with the n6_loader.py (which assumes you did the generate)
• Open a git bash (in ai_runner folder) and run:
  export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
• Then run: python ./example/checker.py -d serial:921600

By default, the AI checker do…

But you can edit it to:

• Load your dataset (npz)
• Save up the outputs

To do so, open checker.py

• Comment the line 62 and replace it to load your dataset:
  inputs = np.load(“./dataset.npy)
• Save ‘outputs’ after the line 64 with something like: np.save("./ output.npy", outputs)
• Comment line 66

Run again the checker.py and you should get the output saved.

Finally, in another python script or notebook, you can:

• Load your dataset
• Use the onnx runtime to get your original model outputs (or load them if you already have them
• Load the N6 output
• Run any tests you would like

Have a good day,

Julian

[实在太懒于是不想取名]

Thank you for your reply. I believe I have identified the root cause of the issue: I had not configured the NPU option in RIF. After enabling it, all outputs now fall between 0 and 1.