I am developing a product which has an STM32H743 device. I am designing the firmware for the first PCBA prototype. The product receives 30fps video signal from an 720i analog video decoder through DCMI (every second bytes are stored, all lines, so 360x240 resolution frames are captured) to external SDRAM. The stm32 does an image conversion and sending the converted data from internal RAM to a second processing unit through parallel bus. The second processing unit displays the data. The parallel bus is a 16bit interface, with 10MHz clock (20Mbyte data rate with ~44% duty cycle – 44% reading, 56% idle). And in the during operation, the device captures two channel audio signals as well.
The ideal operation flow is the following:
- DCMI interface uses DMA1 Stream0 (very high priority, fifo), dma in double buffer mode. DMA moves data from DCMI to FMC external SDRAM. It generates two interrupt signal, half transfer complete and full transfer complete. When full transfer completed the transfer is restarted. With 30FPS I am getting half transfer and full transfer signals, so this seems like working properly.
- When half transfer completed flag is set, the CPU runs a conversion function lets call ycrycbToRGB16() which converts the ycrycb video format to rgb 16bit format. This function finishes before the full transfer signal are set (<15ms execution time)
- When full transfer completed, and ycrycbToRGB16() executed which is true, then we give a signal to external processor. It can start reading out the converted data, through the parallel interface (bus enable high). The parallel interface is implemented, at the STM32 side, with a timer capture input and full utilization of GPIO port B. The TIM5 input capture, captures the parallel bus clock signal and triggers a dma transfer (DMA2 Stream0, TIM5_CH1, fifo) to copy data from internal SDRAM to GPIOB port. When all data is read, the external processor signalize this on a GPIO input and goes to idle.
- ADC capture is running in the background on two channels. Slow channels are used, clock operates on 10MHz maximum, 48ksps,scan conversion mode, DMA circular mode (DMA2 Stream1, the DMA transfer cycle is ~15kHz), this data is also send concatenated with the video capture data.
The whole process must be in synchronization, if there is a pending something or too much calculation time, frames will be skipped and the ideal operation will be not kept.
And the issues that I observing:
- The ideal timing requirements are met only if D-CACHE is enabled. In this case everything executed properly the data acquisition runs smoothly without issue. However, the image data contains artifacts. And it is because the DMA2 copies the internal SDRAM content to GPIOB while the CPU still has data in D-CACHE and it did not store the data to internal SRAM, in fact it took approx. 60ms to update all the frame in sram. Here is an example image, with a dummy signal where the color bars changing colors at each vsync event. E.g. the red bar becomes pink and pink bar becomes red. It can be seen some pixel data remains at previous color but most of them are updated and was read out successfully with the parallel bus.
- If I disable the D-CACHE, the performance drops too much, and the timing requirements of the ideal operation is not met. E.g. the calculation time of the ycrycbToRGB16() function increases from ~7ms to 20ms which is not acceptable. With D-Cache the algo finished sooner, but the data was not there in the internal sram. So either solution is not ok.
Here is another test frame:
The image does not contain any artifact, but the capture rate dropped to 15FPS instead of 30FPS because of the slowing down to access to SDRAM. - If I increase the clock rate of the parallel bus, from 10MHz to 20MHz the timings are “crashed”. It seems stm32 is missing clocks from the external controller and therefore the data transmission breaks. The clock signal looks like this on the proto (with 20MHz clock):
All the parallel bus lines has a 120ohm series resistor now.
Some additional info:
- The STM32 configured to 480MHz SYSCLK from a 16MHz HSE crystal
- The FMC is configured to 240MHz FMC clock (SDRAM common clock 2HCLK – 120MHz, CAS latency 3 clock – 80MHz)
- The TIM5 configured with no preclear and autoload of 2
- each frame consists of ~320kB data, in double buffer mode it is 640kB data
I think I have some memory bandwidth issues. There must be some wait cycles when the CPU tries to write to internal SDRAM. Or when DCMI/DMA1 writes to external SDRAM and the TIM5 triggers data transfer with DMA2 to GPIO port happens in the same time causes latency.
I am thinking about the following options:
- Write code for MDMA or BDMA or DMAMUX or DMA2D controller for more advanced memory action. Especially considering a different domain rams+dma (D1, D2, D3)
- using mixed memory usage of D-CACHE + non cached sram. E.g. non cache for parallel bus data
- using d-cache but handling cases for sdram preparation before dma transfer e.g. clear/invalidate d-cache
- maximize the external SDRAM clock to 200MHz (as the SRAM maximum) and lower the SYSCLK
- using different memory layout e.g. smaller internal sram buffers for dcmi and writing data to external sdram with dma when the internal buffer is filled
Is there anything else to improve the acquisition? Do you see any issue with the concept of the ideal operation implementation?
Thanks the input. Makes lot of sense. Indeed, I am not using OS and I have an always running while loop. I will try to add some sleep, it can fit there.
For series resistor, first I used 27R and I changed to 120R that I read from other forums. Basically from datasheet I saw 5pF input capacitance, so with 120R the cut frequency is pretty high still.
For all GPIOB the output speed is very high as they are configured as digital output. For clock input, I am not sure if I can set high speed for timer input. In GPIO config of cubeide I see this:
I do not know if it make sense to change the "Maximum output speed". The signal is input. Will check this.
And I have this timer config:
