STM32N6 DCMIPP: NNPipe works in continuous mode but not in snapshot mode

[Sergey Burenkov]

I’m trying to run a fork of the STM32N6-GettingStarted-ObjectDetection project for the STM32N6570-DK on my own board.

My board has the same clock configuration, memory interface, and LCD interface as the STM32N6570-DK. However, instead of a CSI camera, I’m using an 8-bit parallel camera interface.

The frames come from a custom FPGA source with the following parameters:

• 75 MHz clock at 8 bit width bus
• YUV422
• 1280 × 768
• 30 fps
• Embedded synchronization ITU656

My changes to the original project are:

• Added GPIO initialization for the 8-bit parallel camera interface.
• Changed the DCMIPP configuration from CSI to parallel mode:

const DCMIPP_EmbeddedSyncCodesTypeDef syncСodes = {
    .FrameStartCode = 0xAB,  /* frame SAV*/
    .LineStartCode  = 0x80,  /* line SAV */
    .LineEndCode    = 0x9D,  /* frame EAV */
    .FrameEndCode   = 0xB6   /* line EAV */
};

const DCMIPP_ParallelConfTypeDef pParallelConfig = {
    .Format = DCMIPP_FORMAT_YUV422,
    .VSPolarity = DCMIPP_VSPOLARITY_LOW,
    .HSPolarity = DCMIPP_HSPOLARITY_LOW,
    .PCKPolarity = DCMIPP_PCKPOLARITY_RISING,
    .ExtendedDataMode = DCMIPP_INTERFACE_8BITS,
    .SynchroMode = DCMIPP_SYNCHRO_EMBEDDED,
    .SynchroCodes = syncСodes,
    .SwapBits = DCMIPP_SWAPBITS_DISABLE,
    .SwapCycles = DCMIPP_SWAPCYCLES_ENABLE
};

/* Set parallel mode */
ret = HAL_DCMIPP_PARALLEL_SetConfig(&hcamera_dcmipp, &pParallelConfig);
assert(ret == HAL_OK);

• Using 

  HAL_DCMIPP_PIPE_Start(&hcamera_dcmipp, pipe, (uint32_t)pbuff, mode);
  

  instead of

  HAL_DCMIPP_CSI_PIPE_Start(&hcamera_dcmipp, pipe, DCMIPP_VIRTUAL_CHANNEL0, (uint32_t)pbuff, mode);

  for pipe start

• Commented out the CSI clock configuration.
• Added YUV-to-RGB conversion to the ISP:

DCMIPP_ColorConversionConfTypeDef yuv_conv_config = {0};

yuv_conv_config.ClampOutputSamples = ENABLE;
yuv_conv_config.OutputSamplesType = DCMIPP_CLAMP_RGB;

yuv_conv_config.RR = 409;
yuv_conv_config.RG = 298;
yuv_conv_config.RB = 0;
yuv_conv_config.RA = -223;

yuv_conv_config.GR = -208;
yuv_conv_config.GG = 298;
yuv_conv_config.GB = -100;
yuv_conv_config.GA = 135;

yuv_conv_config.BR = 0;
yuv_conv_config.BG = 598;
yuv_conv_config.BB = 517;
yuv_conv_config.BA = -277;

ret = HAL_DCMIPP_PIPE_SetYUVConversionConfig(
    &hcamera_dcmipp,
    DCMIPP_PIPE1,
    &yuv_conv_config);

if (ret != HAL_OK)
{
    return CMW_ERROR_PERIPH_FAILURE;
}

ret = HAL_DCMIPP_PIPE_EnableYUVConversion(
    &hcamera_dcmipp,
    DCMIPP_PIPE1);

if (ret != HAL_OK)
{
    return CMW_ERROR_PERIPH_FAILURE;
}

With these changes, I can successfully display the video on the LCD after starting:

CameraPipeline_DisplayPipe_Start(lcd_bg_buffer, CMW_MODE_CONTINUOUS);

So the DCMIPP configuration seems to be working correctly, at least in continuous display mode.

However, I have a problem with the NNPipe.

When I run:

CameraPipeline_NNPipe_Start(nn_in, CMW_MODE_SNAPSHOT);

while (cameraFrameReceived == 0) {};
cameraFrameReceived = 0;

SCB_InvalidateDCache_by_Addr(nn_in, nn_in_len); // debug

inside the main loop, cameraFrameReceived becomes 1, so the frame-received callback appears to be triggered.

However, when I inspect the nn_in buffer, only the first 4 bytes have changed to 0x00FFFFFF. The rest of the buffer contains stale/garbage data and does not change between frames.

Interestingly, if I start the NNPipe in continuous mode before entering the main loop:

CameraPipeline_NNPipe_Start(nn_in, CMW_MODE_CONTINUOUS);

and commented out this section inside main loop:

//CameraPipeline_NNPipe_Start(nn_in, CMW_MODE_SNAPSHOT);

I get valid frames in the nn_in buffer. I can export the buffer contents from the memory browser, convert them to BMP, and the resulting image looks correct.

So the DCMIPP/ISP pipeline appears to capture and process the frames correctly in continuous mode, but the same pipeline does not seem to update the buffer correctly in snapshot mode.

What am I missing?

Could this be related to:

• DCMIPP Pipe configuration?
• DMA/cache coherency?
• Сlock configuration?
• The difference between CMW_MODE_CONTINUOUS and CMW_MODE_SNAPSHOT?
• Some required reconfiguration or restart of the NNPipe between snapshots?

Any suggestions on where to look would be greatly appreciated.

[KDJEM.1]

Hello ​@Sergey Burenkov and welcome to the community;

 

I recommend you to start with an available example like as DCMIPP_SnapshotDecimationMode and get inspired to develop your own application.

This example shows how to use the DCMIPP IP in Snapshot Mode as a Camera Serial Interface and it is based on the STM32Cube HAL API that can be used to build any firmware application. 

 

I hope this help you.

Thank you.

Kaouthar