STM32MP257F + IMX335: std::bad_alloc when configuring 640x640 secondary libcamera stream
Hello ST Team,
I am working on an AI-based Driver Monitoring System on an STM32MP257F board using an IMX335 MIPI CSI camera, libcamera,
GStreamer and STAI-MPU.
I am facing a memory allocation / segmentation fault issue when using a custom STAI-MPU application.
------------------------------------------------------------
HARDWARE
------------------------------------------------------------
Board:
STM32MP257F
Camera:
Sony IMX335 MIPI CSI
Camera interface:
CSI / DCMIPP
Display:
Weston / Wayland
Camera source:
LIBCAMERA
------------------------------------------------------------
SOFTWARE
------------------------------------------------------------
libcamera:
0.3.0+dirty
Python:
3.12
The exact OpenSTLinux/X-LINUX-AI version and other system information can be provided if required.
------------------------------------------------------------
KNOWN-GOOD ST APPLICATION
------------------------------------------------------------
I tested the official ST face-recognition example:
https://github.com/STMicroelectronics/meta-st-x-linux-ai/tree/main/recipes-samples/face-recognition
The official application works correctly on the same STM32MP257F board and the same IMX335 camera.
The working application reports:
camera framerate: 30
camera frame width: 760
camera frame height: 568
camera source: LIBCAMERA
libcamera configures:
(0) 760x568-RGB565
(1) 128x128-BGR888
The application runs successfully.
------------------------------------------------------------
CUSTOM AI APPLICATION
------------------------------------------------------------
I replaced the AI models with my own proprietary STAI-MPU models.
The important model characteristics are:
Model 1:
Input: 640x640x3
INT8
STAI-MPU .nb format
Model 2:
Input: 192x192x3
INT8
STAI-MPU .nb format
Both models successfully load through STAI-MPU.
The first model reports approximately:
m_num_inputs = 1
m_input_height = 640
m_input_width = 640
m_input_channels = 3
m_sizeInBytes = 1228800
The second model reports:
m_num_inputs = 1
m_input_height = 192
m_input_width = 192
m_input_channels = 3
m_sizeInBytes = 110592
I am sharing the tensor dimensions and memory requirements above.
------------------------------------------------------------
FAILURE
------------------------------------------------------------
When the custom application is started, libcamera configures:
(0) 760x568-RGB565
(1) 640x640-BGR888
Immediately afterwards the application terminates with:
terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc
The kernel reports:
__vm_enough_memory: pid: ..., comm: queue2:src, not enough memory for the allocation
This makes me suspect that the second 640x640 camera stream, GStreamer queueing, DCMIPP buffers, or libcamera buffer allocation may be causing excessive memory allocation.
------------------------------------------------------------
PYTHON EXPERIMENT
------------------------------------------------------------
I also tried implementing the camera + STAI-MPU pipeline in Python 3.12.
In that implementation, libcamera negotiated:
(0) 800x600-BGR888
(1) 1280x1080-R8
The application then reported: appsink receiving 640x478 RGB
and immediately afterwards: Segmentation fault (core dumped)
This stream configuration is different from the official ST face-recognition application.
------------------------------------------------------------
CAMERA WARNINGS
------------------------------------------------------------
The following warnings are reported by libcamera:
'imx335 0-001a': Recommended V4L2 control 0x009a0922 not supported
'imx335 0-001a': The sensor kernel driver needs to be fixed
'imx335 0-001a': Failed to retrieve the camera location
'imx335 0-001a': Rotation control not available, default to 0 degrees
However, these warnings also appear when running the official ST face-recognition application, which works correctly.
Therefore, I am currently focusing on the difference in camera stream configuration and memory allocation.
------------------------------------------------------------
IMPORTANT COMPARISON
------------------------------------------------------------
Official ST Custom application
Camera IMX335 IMX335
Resolution 760x568 760x568
Source LIBCAMERA LIBCAMERA
Stream 0 760x568 RGB565 760x568 RGB565
Stream 1 128x128 BGR888 640x640 BGR888
Result Works std::bad_alloc
------------------------------------------------------------
QUESTIONS
------------------------------------------------------------
1. Is a 640x640 BGR888 secondary libcamera stream supported on STM32MP257F with the IMX335?
2. Is there a recommended maximum resolution for a secondary DCMIPP/libcamera stream?
3. Could the following error:
__vm_enough_memory
comm: queue2:src
not enough memory for the allocation
be caused by GStreamer queue buffers or libcamera/DCMIPP
buffer allocation?
4. Are there specific buffer-count or queue-size settings that should be used for a 640x640 AI stream?
5. Is 640x640 BGR888 expected to require significantly more contiguous/video memory than the 128x128 BGR888 stream used
by the official example?
6. For a custom STAI-MPU model with a 640x640 input, what is the recommended architecture?
Option A:
IMX335
|
+-- 760x568 camera stream
|
+-- 640x640 AI stream
|
+-- STAI-MPU
or:
IMX335
|
+-- 760x568 camera stream
|
+-- software resize/letterbox
|
+-- 640x640
|
+-- STAI-MPU
7. Is there an official ST recommendation for using custom 640x640 STAI-MPU models with libcamera on STM32MP257F?
8. Could the difference between the working ST application and my application be related to DMABUF, buffer stride, DCMIPP
alignment, or GStreamer memory handling?
I can provide additional system information, camera format information, GStreamer pipeline information, dmesg output and application source code if required.
Thank you.
