Long rendering time when moving two images

[Marco.R]

Hi @Martin KJELDSEN​ 

On a screen I want to move two large images (one has transparency). I've implemented it like in your birdgame example (with the moveto(x,y) method.

Now I recognized that the animation is very slow and the rendering time is very long (>30ms). I don't know why this take so long. I measured it with the RENDER_TIME GPIO and also in SW. I see that the TouchGFX Library is active for more than 30ms.

Do you have any idea where the issue come from? Below you will find the implementation

Thank you

Best regards

Marco

#include <gui/containers/container_moving_waves.hpp>
#include <BitmapDatabase.hpp>
#include <touchgfx/Color.hpp>
 
container_moving_waves::container_moving_waves(): m_waveForeground(), 
 m_waveBackground(), 
 m_animationState(AnimationState::AnimationRunning),
 m_tickCounter(0U),
 m_tickinterval(1U) {}
 
void container_moving_waves::initialize()
{
 container_moving_wavesBase::initialize();
 initializeLayer(m_waveBackground, BITMAP_WAVE_BACKGROUND_ID, 255U, 1U, -1);
 initializeLayer(m_waveForeground, BITMAP_WAVE_FOREGROUND_ID, 204U, 1U, 1);
 
 m_boxWave.setPosition(0, 217, 800, 33);
 m_boxWave.setColor(touchgfx::Color::getColorFrom24BitRGB(112, 148, 197));
 
 add(m_boxWave);
}
 
void container_moving_waves::startAnimation() {
 m_animationState = AnimationState::AnimationRunning;
}
 
void container_moving_waves::stopAnimation() {
 m_animationState = AnimationState::NoAnimation;
}
 
void container_moving_waves::handleTickEvent() {
 m_tickCounter++;
 
 if ((m_tickCounter % m_tickinterval) != 0U) {
 return;
 }
 
 if (m_animationState == AnimationState::AnimationRunning) {
 moveLayer(m_waveForeground, m_tickCounter);
 moveLayer(m_waveBackground, m_tickCounter);
 }
}
 
void container_moving_waves::initializeLayer(Layer& layer, const BitmapId bmp, uint8_t alpha, const uint32_t animationUpdateSpeed, const int32_t animationWidth)
{
 layer.image0.setBitmap(Bitmap(bmp));
 layer.image1.setBitmap(Bitmap(bmp));
 
 layer.image0.setXY(0, 0U);
 if (animationWidth < 0) {
 layer.image1.setXY(layer.image0.getRect().right(), 0U);
 }
 else {
 layer.image1.setXY(layer.image0.getRect().x - layer.image1.getWidth(), 0U);
 }
 
 layer.image0.setAlpha(alpha);
 layer.image1.setAlpha(alpha);
 
 add(layer.image0);
 add(layer.image1);
 
 layer.animationUpdateSpeed = animationUpdateSpeed;
 layer.animationWidth = animationWidth;
}
 
void container_moving_waves::moveLayer(Layer& layer, const uint32_t tickCount) {
 if ((tickCount % layer.animationUpdateSpeed) == 0U) {
 layer.image0.moveTo(layer.image0.getX() + layer.animationWidth, layer.image0.getY());
 layer.image1.moveTo(layer.image1.getX() + layer.animationWidth, layer.image1.getY());
 
 if (layer.animationWidth < 0) {
 //when moving left
 if (layer.image0.getRect().right() < 0) {
 layer.image0.moveTo(layer.image1.getRect().right(), layer.image0.getY());
 }
 
 if (layer.image1.getRect().right() < 0) {
 layer.image1.moveTo(layer.image0.getRect().right(), layer.image1.getY());
 }
 }
 else {
 //when moving right
 if (layer.image0.getRect().x > layer.image0.getWidth()) {
 layer.image0.moveTo(layer.image1.getRect().x - layer.image0.getWidth(), layer.image0.getY());
 }
 
 if (layer.image1.getRect().x > layer.image1.getWidth()) {
 layer.image1.moveTo(layer.image0.getRect().x - layer.image1.getWidth(), layer.image1.getY());
 }
 }
 }
}

[Martin KJELDSEN]

Hi Marco,

I'd have to know more about your hardware setup. i think. How did you create this project? Is ChromART enabled? If so moving data from flash to framebuffer should only impact your render time minimally. What is the render time for a static frame?

Can you double check settings?

/Martin

[Marco.R]

Hi @Martin KJELDSEN​ 

Thank you for your reply. Here are some more information

Hardware-Setup:

• STM32F767 with external SDRAM / QSPI-Flash
• ChromART (DMA2D) is enabled
• LCD with 800x480 Resolution (LTDC - RGB565)

We are using embOS and created the project initially with TouchGFX 4.9.3.

The rendering time for a static frame with 4 buttons without images is about 15ms. With the moving images like above the rendering time is >70ms. The 30ms from the first post where with some optimization (only one wave,...)

I've made a more detailed analysis (see attached excel file). There are 5 "delays" with more than 7ms where the TouchGFX Library is doing something but I don't know what. Do you have any idea what happens there?

I would highly approciate, if you could invest some time for investigation. I can provide you more information if necessary, but I can't share the project within the forum.

Thank you very much

/Marco

Renderingtime static screen:

Renderingtime moving images:

[Martin KJELDSEN]

Is your QSPI set up properly? Reading from QSPI will also impact render time, regardless of ChromART.

[Marco.R]

Yes, QSPI is set up in memorymode and runs at 108MHz. (measured on hardware)

What I'm wondering is whether the 15ms for a static frame is also much too long, although there is no need to access the QSPI Flash.

What I noticed is that setupDataCopy is never called. Shouldn't this method be called if a data should be transfered from Flash?

[Martin KJELDSEN]

Yes, as i suspected you're not using the ChromART chip to transfer data to the framebuffer from QSPI. That's why i asked "Is ChromART enabled?" - But now i'm realizing i was too vague in my request.

Show me your touchgfx configuration (Maybe in boardconfiguration.cpp) and we'll be able to tell in no time probably.

/Martin

[Marco.R]

I'm enabling ChromART through enabling the following bits:

ACR->ARTEN

AHB1EN->DMA2DEN

Do I have to do something else?

Additionally I will send the configuration file directly to you.

Thanks

/Marco

[Martin KJELDSEN]

This is not enough - TouchGFX needs to know that it should use DMA2D to start data transfers. We have a dedicated class that knows how to use DMA2D to transfer "TouchGFX BlitOps" - Init of TouchGFX must include this.

Could you show me your touchgfx_generic_init() function? I'm not sure where it's located in your project.

/Martin

[Martin KJELDSEN]

STM32F7DMA dma;
STM32F7TouchController tc;
STM32F7Instrumentation mcuInstr;
 
static LCD24bpp display;
static uint16_t bitdepth = 24;
 
namespace touchgfx
{
void touchgfx_init()
{
 uint16_t dispWidth = 480;
 uint16_t dispHeight = 272; 
 
 HAL& hal = touchgfx_generic_init<STM32F7HAL>(dma, display, tc, dispWidth, dispHeight,(uint16_t*) 0, 
 0, 0);

The above code is an example of the init of TouchGFX using an STM32 DMA class. Then, to give you an idea, somewhere in that class we'll start a transfer using the parameters of the BlitOp given to use from one of the LCD classes:

HAL_DMA2D_BlendingStart_IT(&hdma2d,
 (unsigned int)blitOp.pSrc,
 (unsigned int)blitOp.pDst,
 (unsigned int)blitOp.pDst,
 blitOp.nSteps, blitOp.nLoops);

/Martin

[Marco.R]

Thats the code for the touchgfx initialization

static LCD16bpp display; //lint !e956 Justification: required by TouchGFX
static TouchGFX_P3_DMA dma; //lint !e956 Justification: required by TouchGFX
static TouchGFX_P3_TouchController tc; //lint !e956 Justification: required by TouchGFX
 
void SystemInitializationDriver::initTouchGFX() {
 HAL& hal = touchgfx_generic_init<TouchGFX_P3_HAL>(dma, display, tc, 800U, 480U, 0U, 0U, 0U);
 
 // setup for double buffering
 void* pMemBaseAddressSdram = reinterpret_cast<void*>(MEM_BASEADDRESS_SDRAM); //lint !e923
 hal.setFrameBufferStartAddress(pMemBaseAddressSdram);
 
 // The optimized strategy for single buffering requires the presence of a task delay function.
 // hal.registerTaskDelayFunction(&OSWrappers::taskDelay);
 
 // setup touch parameter
 hal.setTouchSampleRate(1);
 hal.setFingerSize(1U);
 
 // by default frame rate compensation is off.
 // Enable frame rate compensation to smooth out animations in case ther is periodic slow frame rates.
 hal.setFrameRateCompensation(false);
 
 // this platform can handle simultaneous DMA and TFT accesses to SDRAM, so disable lock to increase performance
 hal.lockDMAToFrontPorch(false);
 
 // MCU Instrumentation -> Todo: at the moment this feature is not used. the following lines is an example for initalization.
 // This can be deleted in productive version
 // Set MCU instrumentation and Load calculation
 //mcuInstr.init();
 //hal.setMCUInstrumentation(&mcuInstr);
 //hal.enableMCULoadCalculation(true);
}

I understand, that the DMA2D method should be called. My problem is that the method setupDataCopy within the DMA class will not be called by the touchgfx library. The other method setupDataFill (which is in the same class) will be called correctly.

Meanwhile I did a performancetest for QSPI. I get a troughput of about 50MBps. So this should not be the problem.

Is there anything else I should verify?

Thanks

/Marco

[Marco.R]

Hi @Martin KJELDSEN​ 

I found the reason why it takes so long. Because of the large images I use, I stored the images as L8_ARGB8888. When I change back to RGB565/ARGB8888 then the method setupDataCopy will called as expected. The rendering time falls from 73ms down to 42ms which is still high but much better. The cpu load falls from >90% down to 4%.

Is there a workaround to use L8_ARGB8888 and DMA2D together?

As written above the rendering time is still high. But I think It makes sense. I tried to calculate the theoretical renderingtime

Read from QSPI (108MHz):

Img1/Img2 (800x250px / 4Byte/px) -> 14.8ms each

Write to SDRAM (16bit data bus / 108MHz):

Background (Box / 800x480px) -> 3.6ms

Img1/Img2 -> 1.9ms each

In total it should take about 37ms. Is my calculation correct?

According to your experience is a rendering time of about 40ms expectable? Or should it go faster?

Do you have any advices how I can optimize the rendering time for large images?  Should I may caching the images into the SDRAM instead of reading out of Flash?

Thank you

Marco

[scottSD]

Marco.R,

These are good questions. I have been trying to figure out an implementation of L8_ARGB8888 myself. I asked a question about it, but have not had any responses.

I am still learning about it, but I did find that for the STM32f746g-Discovery kit (which I am currently using), I found a call that the function Martin mentioned (HAL_DMA2D_BlendingStart_IT() ) located in the file TouchGFX\target\STM32F7DMA.cpp. The function making this call is setupDataCopy() which appears to setup the dma copy based on which blitOp operation being used.

However, there doesn't appear to be an operation for indexed color (BlitOp.hpp):

enum BlitOperations
{
 BLIT_OP_COPY = 1 << 0, ///< Copy the source to the destination
 BLIT_OP_FILL = 1 << 1, ///< Fill the destination with color
 BLIT_OP_COPY_WITH_ALPHA = 1 << 2, ///< Copy the source to the destination using the given alpha
 BLIT_OP_FILL_WITH_ALPHA = 1 << 3, ///< Fill the destination with color using the given alpha
 BLIT_OP_COPY_WITH_TRANSPARENT_PIXELS = 1 << 4, ///< Deprecated, ignored. (Copy the source to the destination, but not the transparent pixels)
 BLIT_OP_COPY_ARGB8888 = 1 << 5, ///< Copy the source to the destination, performing per-pixel alpha blending
 BLIT_OP_COPY_ARGB8888_WITH_ALPHA = 1 << 6, ///< Copy the source to the destination, performing per-pixel alpha blending and blending the result with an image-wide alpha
 BLIT_OP_COPY_A4 = 1 << 7, ///< Copy 4-bit source text to destination, performing per-pixel alpha blending
 BLIT_OP_COPY_A8 = 1 << 8 ///< Copy 8-bit source text to destination, performing per-pixel alpha blending
};

BlitOp.hpp contains the BlitOp struct. That struct does contain a pointer to the CLUT (pClut) , so I am not sure why there is no operation for indexed color.

struct BlitOp
{
 uint32_t operation; ///< The operation to perform @see BlitOperations
 const uint16_t* pSrc; ///< Pointer to the source (pixels or indexes)
 const uint8_t* pClut; ///< Pointer to the source CLUT entires
 uint16_t* pDst; ///< Pointer to the destination
 uint16_t nSteps; ///< The number of pixels in a line
 uint16_t nLoops; ///< The number of lines
 uint16_t srcLoopStride; ///< The number of bytes to stride the source after every loop
 uint16_t dstLoopStride; ///< The number of bytes to stride the destination after every loop
 colortype color; ///< Color to fill
 uint8_t alpha; ///< The alpha to use
 uint8_t srcFormat; ///< The source format @see BitmapFormat
 uint8_t dstFormat; ///< The destination format @see BitmapFormat
};

And as far as I know, ChromeArt supports a CLUT.