STM32N647 Maximum Pixel Through DCMIPP and ISP

[cameradeveloper]

Dear Community, 

I am researching how to use a 50Mpx MIPI CSI-2 2 LANE RAW 10 sensor with stm32N647.
The idea is not to use the full sensor, instead using analogue/digital crop and windowing on the sensor to capture a specific part of the image and downsize it to a smaller size before entering the ISP of the MCU. The sensor doesn't have scaling.
So after all steps I am left with a frame size that varies from 11.4 to 2.4 Mpx depending on cases.
From there the image needs to be downsized to a buffer with a maximum size of 1024x768 px for the display.
The question is whether the scaling of the ISP of the STM32N647 can handle downsizing and at what cost. The ISP is rated at 5Mpx 30fps but specifies it is possible above, the datasheet is vague about this.

Thanks for the help.

[KDJEM.1 · Accepted solution]

Hello @cameradeveloper and welcome to the community;

The Pipe 1 is typically used for the main needs (such as video encode, software analysis, neural network), it allows to downsize the input, convert its color and dump to multiple pixel formats, including multi-planar ones. It can adapt the luminosity (gamma conversion) as well as auto-exposure and black level calibration based on its own embedded image processing functions, handling camera sensor without internal ISP.

This figure shows the DCMIPP main functions, namely the integrated parallel interface, the abutted CSI-2 host, the ISP functions in Pipe1, the differentiated applicative post-processing pipelines with their crop, downsize, and pixel formatting.

For main features about downsize and crop and ratios,  I recommend you to look at reference manual section 39
Digital camera interface pixel pipeline (DCMIPP).

I hope this answer you request.

Thank you.

Kaouthar