STM32F7 CPU_ENABLE_Cache SCB_EnableDCache() corrupts stack?

[James Murray]

Working on a bootloader on the STM32F745 and starting from example code there was the function call CPU_CACHE_Enable.

In the original examples, this is called right at the start of main(). I had moved this function call later as I (thought) I wanted to selectively jump to the application first.

I spent a lengthy debug session a few days back trying to understand why a variable in main() was becoming corrupted.

Over-simplified example.

extern int a;
 
main (void)
 
{
 
if (a == 1) {  <...this code runs....> }
 
CPU_CACHE_Enable();
 
if (a == 1) { <...this code DOES NOT run...> }
 
}

What I found was that the compiler was storing the address of the variable in R4 and assuming the R4 would be preserved across the function call, as per the ARM spec.

On entry to CPU_CACHE_Enable, R4 is pushed to the stack. However, the inline code SCB_EnableDCache is executed which invalidates and enables the cache. This stomps on the location where R4 got pushed. On exit, CPU_CACHE_Enable pops R4 back again, but it is garbage and the subsequent use of R4 as an address pointer returns the wrong data.

Maybe there is some, but I haven't seen any notes in the documentation to warn developers of this possibility. I have worked around it by enabling the cache in my startup code instead and adding a comment about the stack corruption.

Is this expected behaviour or am I doing something wrong? If this is normal, then could I suggest that warning comments be added to help others avoid my mistake.

James

[Tesla DeLorean]

Does sound odd.

Do you configure the MPU in write-thru mode?

Does the code use a DSB fencing instruction prior to the state change?

Does the memory address used for the stack change the behaviour, ie DTCMRAM?

GNU Compiler?

[James Murray]

The call to enable the cache is before the MPU is enabled.

Here's the code from core_cm7.h

__STATIC_INLINE void SCB_EnableDCache (void)
{
  #if (__DCACHE_PRESENT == 1U)
    uint32_t ccsidr;
    uint32_t sets;
    uint32_t ways;
 
    SCB->CSSELR = (0U << 1U) | 0U;          /* Level 1 data cache */
    __DSB();
 
    ccsidr = SCB->CCSIDR;
 
                                            /* invalidate D-Cache */
    sets = (uint32_t)(CCSIDR_SETS(ccsidr));
    do {
      ways = (uint32_t)(CCSIDR_WAYS(ccsidr));
      do {
        SCB->DCISW = (((sets << SCB_DCISW_SET_Pos) & SCB_DCISW_SET_Msk) |
                      ((ways << SCB_DCISW_WAY_Pos) & SCB_DCISW_WAY_Msk)  );
        #if defined ( __CC_ARM )
          __schedule_barrier();
        #endif
      } while (ways--);
    } while(sets--);
    __DSB();
 
    SCB->CCR |=  (uint32_t)SCB_CCR_DC_Msk;  /* enable D-Cache */
 
    __DSB();
    __ISB();
  #endif
}

One of the writes to SCB->DCISW causes a change to the memory location where R4 was pushed, corrupting it.

Before function call R4 = 0x40024000 (address of the start of backup SRAM), after function call, R4 = 0x0007418 (or similar.)

Initially I wondered if this was a gcc problem, but my present conclusion is that this is inherent in enabling the cache. There's no way that gcc could be expected to know that the stack will get trashed, that breaks all programming norms.

I have read the ARM core manual relating to the cache handling, but I don't fully understand the terminology, so am left unclear on whether this is expected behaviour or not.

I have not tried moving the stack.

James

[Tesla DeLorean]

For chuckles I'd probably put a __DSB(); on line 7 to ensure the stack pushing hits the memory, and the write buffers clear properly.

Is the cache already enabled?

I say GCC as it might push more registers rather than others, I'd half expect Keil to inline everything and keep everything in R0-R3 and LR

I know that random use of SCB_InvalidateDCache() has a lot of side-effects, often manifesting in stack corruption and hard faults. The By_Addr variant is strongly recommend in 99.99% of cases.

[Tesla DeLorean]

Keil in-lines

C:\Keil5xx\Arm\Packs\ARM\CMSIS\5.7.0\CMSIS\Core\Include\cachel1_armv7.h

/**
  \brief   Enable D-Cache
  \details Turns on D-Cache
  */
__STATIC_FORCEINLINE void SCB_EnableDCache (void)
{
  #if defined (__DCACHE_PRESENT) && (__DCACHE_PRESENT == 1U)
    uint32_t ccsidr;
    uint32_t sets;
    uint32_t ways;
 
    if (SCB->CCR & SCB_CCR_DC_Msk) return;  /* return if DCache is already enabled */
 
    SCB->CSSELR = 0U;                       /* select Level 1 data cache */
    __DSB();
 
    ccsidr = SCB->CCSIDR;
 
                                            /* invalidate D-Cache */
    sets = (uint32_t)(CCSIDR_SETS(ccsidr));
    do {
      ways = (uint32_t)(CCSIDR_WAYS(ccsidr));
      do {
        SCB->DCISW = (((sets << SCB_DCISW_SET_Pos) & SCB_DCISW_SET_Msk) |
                      ((ways << SCB_DCISW_WAY_Pos) & SCB_DCISW_WAY_Msk)  );
        #if defined ( __CC_ARM )
          __schedule_barrier();
        #endif
      } while (ways-- != 0U);
    } while(sets-- != 0U);
    __DSB();
 
    SCB->CCR |=  (uint32_t)SCB_CCR_DC_Msk;  /* enable D-Cache */
 
    __DSB();
    __ISB();
  #endif
}

/******************************************************************************
 * @file     cachel1_armv7.h
 * @brief    CMSIS Level 1 Cache API for Armv7-M and later
 * @version  V1.0.0
 * @date     03. March 2020
 ******************************************************************************/
/*
 * Copyright (c) 2020 Arm Limited. All rights reserved.
...

CPU_CACHE_Enable completely folds in

Getting "Maximum number of topic assignments have been exceeded." when posting after adding assembler in code block window (lots of #'s) @Amel NASRI​ 

In-lining as an image

[Tesla DeLorean]

>>Is the cache already enabled?

This, watch for this, especially in app called by boot loader.

Note new CMSIS checks for this, and forces function to in-line.

The check also has a secondary side-effect of fencing the pending write-buffers (ie implicit DSB)

[James Murray]

gcc inlines it too. That's not the problem as far as I can tell.

When I was single-stepping, the value of R4 did get stacked correctly. No other registers were stacked.

I'm using Eclipse and could see that the contents of R4 were now showing in memory on the stack.

While stepping through the function, that word of memory got overwritten. There's no way gcc

The cache shouldn't already have been enabled, but I can't be sure. I think the crux here is your comment "random use of SCB_InvalidateDCache() has a lot of side-effects, often manifesting in stack corruption" which looks to be exactly what I'm seeing.

Now that I've moved the code to within startup, it is enabling the cache before the stack is actually used, so hopefully I've worked around the problem.

I'd be interested in a comment from ST as this code is a part of their libraries and could surely trip someone else up too.

Ref: the cache already being enabled - yes that's likely a problem in my code. I was jumping back to the bootloader without a reset for a "warmstart".

James

[Tesla DeLorean]

>>The cache shouldn't already have been enabled, but I can't be sure. I think the crux here is your comment "random use of SCB_InvalidateDCache() has a lot of side-effects, often manifesting in stack corruption" which looks to be exactly what I'm seeing.

The hazard here is that the pending write-back or write-buffers get lost, so the data visible in memory is not reflective of what was written last (most recently). The pending data could be the entire cache-line, and thus all the pushed registers.

What the debugger might be seeing is the processor side cache, not what's in the memory behind it.