TouchGFX Double Framebuffer Swapping Logic

[Jon Enz]

I'm a bit confused about the strategy used to determine when to swap buffers when TouchGFX is configured for double frame buffers. As illustrated here, the intent of the double frame buffer is to allow the old frame buffer to be presented if there is a long render time for the next frame. The issue is that I don't see how the TouchGFX implementation enables this. My understanding is that the function HAL::swapFrameBuffers is responsible for switching the pointer to the current frame buffer for the LTDC. Looking at calls to this function, the only one I can find is within the HAL_LTDC_LineEventCallback within TouchGFXGeneratedHAL.cpp.

extern "C"
{
    void HAL_LTDC_LineEventCallback(LTDC_HandleTypeDef *hltdc)
    {
        if (LTDC->LIPCR == lcd_int_active_line)
        {
            //entering active area
            HAL_LTDC_ProgramLineEvent(hltdc, lcd_int_porch_line);
            HAL::getInstance()->vSync();
            OSWrappers::signalVSync();
            // Swap frame buffers immediately instead of waiting for the task to be scheduled in.
            // Note: task will also swap when it wakes up, but that operation is guarded and will not have
            // any effect if already swapped.
            HAL::getInstance()->swapFrameBuffers();
            GPIO::set(GPIO::VSYNC_FREQ);
        }
        else
        {
            //exiting active area
            HAL_LTDC_ProgramLineEvent(hltdc, lcd_int_active_line);
            GPIO::clear(GPIO::VSYNC_FREQ);
            HAL::getInstance()->frontPorchEntered();
        }
    }
}

My understanding is that this callback function, specifically the condition that contains the call to swapFrameBuffers, will be executed every time the LTDC is ready to start drawing a new frame, what if the TouchGFX engine hasn't completed rendering the new frame? Why call that function from the ISR, and not from the TouchGFX engine when the frame is done rendering? What am I missing about how this works?

To give some more context, I'm implementing manual 180 deg rotation using the method stated here, so I am anticipating some long render times and lost frames. I am seeing artifacts and shifts on some of the frames currently that I would expect to be removed by a double frame buffer.

[Martin KJELDSEN]

There must be something else going on, because TouchGFX won't render to/mess with the framebuffer that is being transferred. Are you having memory bandwidth issues?

You can always try hal.LockDMAToFrontPorch(true); in your TouchGFXHAL.cpp initialize() function. This means that DMA operations won't happen until the LTDC has left the active area and reduces strain on memory.

Suffice it to say that TouchGFX is driven by syncronization signals from displays, display controllers or timers. No swapping should occur unless a transfer is complete, and that's how it is for the LTDC/DSI drivers in our application templates or what we can generate from the TouchGFX generator (LTDC only atm).

If you want you can always upload a video of the issue here. I'd quickly be able to tell you what it is i think.

/Martin

[Jon Enz]

I'm not sure what the cause of this issue is/was, but it seems to have been cleared up and is now working as expected. A colleague was working on some low level DMA/SPI code that may have been causing some bandwidth issues. If it starts happening again I will include a video in the thread.

I was more interested in why the call to swap inside the ISR for the vSync does not check that the task is done rendering the frame, or if it does, how? I'm not worried about the transfer to the hardware being complete. I'm worried that the application task didn't complete drawing the next frame, and the ISR switched it anyways. After seeing it work properly now, I think there is just something I'm missing about the logic.

[Jon Enz]

Scratch that.. The issue appeared again today.

A bit more about my configuration: I need to invert the framebuffer as stated in the original post, but I don't want to invalidate the entire screen and have the framework redraw every pixel. To avoid this, I have two extra frame buffers to hold the inverted images of what the framework draws. The framework draws to either FrameBuffer1 or FrameBuffer2, and in the TouchGFXHAL::endFrame() function the buffer is inverted and transferred to the corresponding invert buffer.

void TouchGFXHAL::endFrame()
{
    dma.flush();
 
    if (getInstance()->lockFrameBuffer() == (uint16_t*) &_sFrameBuffer1)
        invertBuffer((uint16_t*)&_sFrameBuffer1, (uint16_t*)&_sFrameBuffer1_Invert);
    else
        invertBuffer((uint16_t*)&_sFrameBuffer2, (uint16_t*)&_sFrameBuffer2_Invert);
 
    getInstance()->unlockFrameBuffer();
    HAL::endFrame();
}

Then I am able to trick the framework and LTDC into using the correct buffers using the TouchGFXHAL::getTFTFrameBuffer() and void TouchGFXHAL::setTFTFrameBuffer(uint16_t* address) functions.

/**
 * Gets the frame buffer address used by the TFT controller.
 *
 * @return The address of the frame buffer currently being displayed on the TFT.
 */
uint16_t* TouchGFXHAL::getTFTFrameBuffer() const
{
    // Calling parent implementation of getTFTFrameBuffer().
    //
    // To overwrite the generated implementation, omit call to parent function
    // and implemented needed functionality here.
 
    // This function is used by the application to determine the current
    // framebuffer. Since we pass the inverted buffer to the LTCD, we need to
    // lie to the application about what the current framebuffer is so it draws
    // in the correct place.
    if (TouchGFXGeneratedHAL::getTFTFrameBuffer() == (uint16_t*) &_sFrameBuffer1_Invert)
        return (uint16_t*) &_sFrameBuffer1;
    else
        return (uint16_t*) &_sFrameBuffer2;
}
 
/**
 * Sets the frame buffer address used by the TFT controller.
 *
 * @param [in] address New frame buffer address.
 */
void TouchGFXHAL::setTFTFrameBuffer(uint16_t* address)
{
    // Calling parent implementation of setTFTFrameBuffer(uint16_t* address).
    //
    // To overwrite the generated implementation, omit call to parent function
    // and implemented needed functionality here.
 
    // Tell the TFT controller to use the frame buffer that contains the
    // inverted image.
    if (address == (uint16_t*) &_sFrameBuffer1)
        address = (uint16_t*) &_sFrameBuffer1_Invert;
    else
        address = (uint16_t*) &_sFrameBuffer2_Invert;
 
    TouchGFXGeneratedHAL::setTFTFrameBuffer(address);
}

My current suspicion is that it's something with the LTDC. If I put a break point in the TouchGFXHAL::endFrame() function I can step through the generation of each frame. The artifacts I am seeing are not corruptions in the frame buffer because they appear and go away while the processor is halted. The artifact seems to be a shift up in the pixels along with dropping/shifting the RGB channels. The affected area seems to progress from the right side of the screen to the left.

See the video below. There is a counter in the Model::tick() function that is setting the "<value> MIN" field below the blue bar. This can be used to see when frames are generated (I hit continue from the TouchGFXHAL::endFrame() breakpoint). Around 29s when the counter gets to 94 the image is distorted but is corrected on the next refresh without the memory being touched (no new frame generated/inverted). When the count gets to 128 I take out the break point and let the program continue so you can see the distortion in real-time.

[Martin KJELDSEN]

​Maybe LTDC timing issues then? Can you double check your data sheet? Did you try hal.lockDMAToFrontPorch(true) ?  Strain on the memory bus will also just give you artifacts not in the buffer.

/Martin

[Martin KJELDSEN]

Can you tell me what's supposed to be there where the artifcats are? Something tells me they're images and you haven't programmet those images to flash. This does not really seem like a buffering-run-time-gitch-kind-of-issue. It's a pretty quiet application.

/Martin

[Jon Enz]

I'm sorry, the issue that I'm chasing in this post does not have to do with those images. They are supposed to be small icons generated from .png files, but I'm not sure why they're not working. I should have just removed them before taking the video. I haven't started troubleshooting that yet.

The artifacts I'm referring to for this issue are the flashing and shifts of the frame. Here is a screenshot of the video around 29s:

[Jon Enz]

I'll give that a try and see what happens. The comment for that function says Please see the chapter "Optimizing DMA During TFT Controller Access" for details on this setting. Can you point me to this resource? I'm not finding it by searching the TouchGFX documentation.

[Jon Enz]

I was able to (hopefully) resolve the issue today. After some messing with the LTDC and enabling the error interrupts I found that the FIFO buffer in the LTDC is experiencing buffer underruns. In my application (STMH753) the frame buffers are held in a 8MB external SRAM. All of my program RAM is currently configured in the 512KB SRAM. Both of these peripherals are on the 64-bit AXI bus. Looking at my linker script, my .bss and .data sections were not properly aligned to 64 bit. Taking out the . = ALIGN(4); or . = ALIGN(8); statements and replacing them with . = ALIGN(64); statements seems to have cleared up the issue for now.

I don't have the deepest understanding of why this helps. I know it's not efficient to access smaller addressed variables on the 64 bit bus, but this just adjusts the location of the entire data block, not necessarily the variables within the block. I would think the push/pop operations on the various stacks for the tasks would be causing the bandwidth issues, and they are happening somewhere randomly in the middle of the section of RAM.

If anyone has a deeper insight into this please let me know.