STM32N6 H264 Video Encoder ASIC datasheet / manual

[exarian]

Hello folks,

I am investigating the Video Encoder on the STM32N6 and have been working through the reference project (VENC_SDCard_ThreadX).

The user manual (RM0486) is very vague on detailing the Video Encoder, and lists over 100 pages of registers which appear to be referencing other Software Registers (VENC_SWREGx):

There are approximately 500 of registers like these, and it doesn't really say much. 

Looking through the `asicRegisterDesc[]` structure in the Middleware has been useful to try and see what the registers are used for, but still is nowhere near the amount of detail that is needed.

Any further info would be greatly appreciated.

Kindest regards,

[KDJEM.1 · Accepted solution]

Hello @exarian;

I reported the description issue in the reference manual internally.

I advise you to look at Hantro.VC8000NanoE.V50x.SW.Integration.Guide-v1.02-20200708.pdf document may help you.

I recommend you to run an example like VENC_JPEG_Encoding can help you to understand the VENC.

Internal ticket number: 209269 (This is an internal tracking number and is not accessible or usable by customers)

Thank you.

Kaouthar

[exarian]

Thank you @KDJEM.1 

That is a great reference to understand the EWL better, I appreciate it.

I have been working with the VENC_SDCard_ThreadX demo, and have managed to encode a video stream, using a test pattern sequence as well as the DCMI.

[Will_Robertson]

Hi Kaouthar @KDJEM.1 and @exarian ,

Thank you very much for this conversation and the link to the H264 encoder documentation and example.

We're planning a new open source wildlife camera system for conservation of Europe's most endangered dormouse, bat and squirrel species which combines the STM32N6 series MCUs and ST image sensors so it's very valuable for us to learn more about the H264 encoder on the STM32N6 series.

>I reported the description issue in the reference manual internally.
>Internal ticket number: 209269 (This is an internal tracking number and is not accessible or usable by customers)

I was wondering if there was any update on this?

Thank you very much,

Will

[KDJEM.1]

Hello @Will_Robertson ,

The 209269 ticket is under investigation.

I recommend you to look at these resources may help you:

Introduction to Hardware Video Encoding with STM32 - stm32mcu

H264 Hardware Handshake encoding - stm32mcu

H264 power consumption setup - stm32mcu

Thank you.

Kaouthar

[exarian]

Hi @Will_Robertson ,

At the outset when I started the VENC was a mystery, as the N6 datasheet is basically referring another document, and does not really give any value. 

Getting into the development however was not a pain, and the ST reference was a great starting point to test the H264. I thankfully didn't need to dive into the register level to debug, as things just worked quite well.

To test the H264 independently, I would just generate a test pattern. Firstly a coloured frame, then something more dynamic like a moving coloured box to see what the H264 does. I was very impressed when I went through that testing.

I started with the VENC_SDCard example and worked from there. 

Based on your resolution, camera and compression needed you may need to tweak the H264 settings such as the Encoder Level.

I wish you luck on your development. I am guessing you'll eventually be including some AI in your project, to detect dormouse versus bat, that is when the true fun will start!

Thank you @KDJEM.1 for the further assistance and help.

[Will_Robertson]

Hello Kaouthar @KDJEM.1,

Thank you very much - I hadn't found those documents and having them is an enormous help - I'm studying them in detail now. :)

Thank you very much for all your hwlp!

Will

[Will_Robertson]

Hi @exarian ,

Thank you very much.

This is the first time that I've worked with an MCU this advanced and the first time that I've worked on an MCU with a vision pipeline so I'm fairly intimidated about learning to configure it all.

>Getting into the development however was not a pain, and the ST reference was a great starting point to test the H264. I thankfully didn't need to dive into the register level to debug, as things just worked quite well.

That's great!

>I started with the VENC_SDCard example and worked from there. 

That sounds a good starting point - we were looking at starting from the examples for the ST BrightSense image sensors - 

>I wish you luck on your development. I am guessing you'll eventually be including some AI in your project, to detect dormouse versus bat, that is when the true fun will start!

Yes - we're training CNNs at the moment with video from the Belgian field study

https://new-homes-for-old-friends.cairnwater.com/quick-summary-new-homes-for-old-friends-belgium/

then planning to optimise them for the NPU on the STM32N6x7 series MCUs - we're hoping that by only doing a mammal / bird / no animal classification it will be possible to keep the CNN small enough and fast enough to run on an embedded NPU.

Kind regards,

Will

[KDJEM.1]

Hello @Will_Robertson ;

Are you unable to open these links?

Introduction to Hardware Video Encoding with STM32 - stm32mcu

H264 Hardware Handshake encoding - stm32mcu

H264 power consumption setup - stm32mcu

I will let you know when the description is available in the reference manual.

Thank you.

Kaouthar

[Will_Robertson]

Hi Kaouthar @KDJEM.1 

Thank you very much - yes, we're using the documentation in these links at the moment and the VENC_SDCard example:

https://github.com/STMicroelectronics/STM32CubeN6/tree/main/Projects/STM32N6570-DK/Applications/VENC/VENC_SDCard

It would definitely be great to know when the description in the reference manual (internal 209269) is ready so that we can study that as well as soon as it's ready.

Thank you very much for your help!

Will