I am developing a product which has an STM32H743 device. I am designing the firmware for the first PCBA prototype. The product receives 30fps video signal from an 720i analog video decoder through DCMI (every second bytes are stored, all lines, so 360x240 resolution frames are captured) to external SDRAM. The stm32 does an image conversion and sending the converted data from internal RAM to a second processing unit through parallel bus. The second processing unit displays the data. The parallel bus is a 16bit interface, with 10MHz clock (20Mbyte data rate with ~44% duty cycle – 44% reading, 56% idle). And in the during operation, the device captures two channel audio signals as well.
The ideal operation flow is the following:
- DCMI interface uses DMA1 Stream0 (very high priority, fifo), dma in double buffer mode. DMA moves data from DCMI to FMC external SDRAM. It generates two interrupt signal, half transfer complete and full transfer complete. When full transfer completed the transfer is restarted. With 30FPS I am getting half transfer and full transfer signals, so this seems like working properly.
- When half transfer completed flag is set, the CPU runs a conversion function lets call ycrycbToRGB16() which converts the ycrycb video format to rgb 16bit format. This function finishes before the full transfer signal are set (<15ms execution time)
- When full transfer completed, and ycrycbToRGB16() executed which is true, then we give a signal to external processor. It can start reading out the converted data, through the parallel interface (bus enable high). The parallel interface is implemented, at the STM32 side, with a timer capture input and full utilization of GPIO port B. The TIM5 input capture, captures the parallel bus clock signal and triggers a dma transfer (DMA2 Stream0, TIM5_CH1, fifo) to copy data from internal SDRAM to GPIOB port. When all data is read, the external processor signalize this on a GPIO input and goes to idle.
- ADC capture is running in the background on two channels. Slow channels are used, clock operates on 10MHz maximum, 48ksps,scan conversion mode, DMA circular mode (DMA2 Stream1, the DMA transfer cycle is ~15kHz), this data is also send concatenated with the video capture data.
The whole process must be in synchronization, if there is a pending something or too much calculation time, frames will be skipped and the ideal operation will be not kept.
And the issues that I observing:
- The ideal timing requirements are met only if D-CACHE is enabled. In this case everything executed properly the data acquisition runs smoothly without issue. However, the image data contains artifacts. And it is because the DMA2 copies the internal SDRAM content to GPIOB while the CPU still has data in D-CACHE and it did not store the data to internal SRAM, in fact it took approx. 60ms to update all the frame in sram. Here is an example image, with a dummy signal where the color bars changing colors at each vsync event. E.g. the red bar becomes pink and pink bar becomes red. It can be seen some pixel data remains at previous color but most of them are updated and was read out successfully with the parallel bus.
- If I disable the D-CACHE, the performance drops too much, and the timing requirements of the ideal operation is not met. E.g. the calculation time of the ycrycbToRGB16() function increases from ~7ms to 20ms which is not acceptable. With D-Cache the algo finished sooner, but the data was not there in the internal sram. So either solution is not ok.
Here is another test frame:
The image does not contain any artifact, but the capture rate dropped to 15FPS instead of 30FPS because of the slowing down to access to SDRAM. - If I increase the clock rate of the parallel bus, from 10MHz to 20MHz the timings are “crashed”. It seems stm32 is missing clocks from the external controller and therefore the data transmission breaks. The clock signal looks like this on the proto (with 20MHz clock):
All the parallel bus lines has a 120ohm series resistor now.
Some additional info:
- The STM32 configured to 480MHz SYSCLK from a 16MHz HSE crystal
- The FMC is configured to 240MHz FMC clock (SDRAM common clock 2HCLK – 120MHz, CAS latency 3 clock – 80MHz)
- The TIM5 configured with no preclear and autoload of 2
- each frame consists of ~320kB data, in double buffer mode it is 640kB data
I think I have some memory bandwidth issues. There must be some wait cycles when the CPU tries to write to internal SDRAM. Or when DCMI/DMA1 writes to external SDRAM and the TIM5 triggers data transfer with DMA2 to GPIO port happens in the same time causes latency.
I am thinking about the following options:
- Write code for MDMA or BDMA or DMAMUX or DMA2D controller for more advanced memory action. Especially considering a different domain rams+dma (D1, D2, D3)
- using mixed memory usage of D-CACHE + non cached sram. E.g. non cache for parallel bus data
- using d-cache but handling cases for sdram preparation before dma transfer e.g. clear/invalidate d-cache
- maximize the external SDRAM clock to 200MHz (as the SRAM maximum) and lower the SYSCLK
- using different memory layout e.g. smaller internal sram buffers for dcmi and writing data to external sdram with dma when the internal buffer is filled
Is there anything else to improve the acquisition? Do you see any issue with the concept of the ideal operation implementation?
Seems like the issue is not related to the series resistor, but I replaced it to 10R. So here is the thing, I could manage to increase the parallel bus speed to 25MHz from 10Mhz, but it works only, if the audio capture is disabled. Without audio capture, the video stream runs smoothly. When I enable the audio capture, the DMA stream for parallel out is corrupted somehow... Both peripheral uses the DMA2 (Stream0 and Stream1). The DMA streams are configured as:
The TIM5_CH1 is the parallel interface clock input, it is configured for very high priority. The analog capture is on low priority. On the following picture you can see a "normal operation". CH9 is the interesting on, it shows the interrupt of the DMA2 Stream0, on CH5 you can see the parallel reading clock. Between two DMA interrupts we have ~1.895ms time.
And here it is the corrupted one:
The third transfer does not finish properly, and the last dma interrupt occours at ~2.95ms instead of 1.89ms. But the first arrived in time. And you can see the impulse train on the paralel clock are "doubled" in each reading cycle, because the MCU did not signalized the end of the parallel communication and so the external controller started a new reading cycle.
I checked plenty of things in the code already but I feel like I lost in the woods...It seems like the DMA2 Stream0 and Stream1 got into trouble somehow. Or there is some bandwidth issues on the internal buses. If I change back the paralell read speed to 10MHz then everything works properly.
Any idea to look for?
