STM32G431 FOC R3_2_TIMx_UP and ADC1_2 Interrupt handler lockup with FreeRTOS

[Clive_S]

I have a strange problem with the ADC/TIM interrupts for a simple FOC motor application on STM32G431 on the B-B431B-ESC eval board. The build uses FreeRTOS.

There appears to be some strange behavior with R3_2_TIMx_UP_IRQHandler() and ADC1_2_IRQHandler() at startup under some conditions. An apparently benign code change (code that is not actually executed) causes a lockup in R3_2_TIMx_UP_IRQHandler() due to an aparent timing issue.

while (0x0u != pHandle->pParams_str->ADCDataReg1[pHandle->_Super.Sector]->JSQR)
{
    /* Nothing to do */
}

What appears to happen in the lockup case is

• R3_2_TIMx_UP_IRQHandler() called for the 1st time and triggers an ADC JSQR conversion (T1).
• R3_2_TIMx_UP_IRQHandler() called for the second time and triggers an ADC JSQR conversion (T2).
• ADC1_2_IRQHandler() is called for the first time for T1, but we now loose T2
• R3_2_TIMx_UP_IRQHandler() called for the third time and hangs because JSQR is not zero because the T2
  trigger was lost.

What should have happened is the first call to ADC1_2_IRQHandler() should have happened before the second call to R3_2_TIMx_UP_IRQHandler().

I can toggle between the correct and incorrect behavior by (for example) inserting a call to strtol() in a thread. However I have added a 5 second sleep to the start of all threads (including medium-frequency and safety) to ensure the thread code is present but the threads are effectively disabled initially. The lockup occurs quite early after the osKernelStart() call. FreeRTOS should take one pass through each thread and then they should all sleep leaving FreeRTOS in its idle loop. Adding the strtol moves things in memory a little but in this case the version that works
correctly has the strtol() call present. There is plenty flash and ram.

In the failing case I can re-trigger the ADC in R3_2_TIMx_UP_IRQHandler() but that just repeats the same cycle with the call to ADC1_2_IRQHandler being too late. However, if I then add one idle pass through R3_2_TIMx_UP_IRQHandler() (ie skip one tick) everything syncs up and the ADC1_2_IRQHandler() calls happen correctly before the next R3_2_TIMx_UP_IRQHandler() call and everything then works fine. That apears it could be a workaround, though I don't like it.

I'm struggling to work out what would cause these two interrupts to initially get out of sync. The behavior regarding adding or removing the one line of code has been 100% consistent through dozens of debug builds. I have seen other things flip between working and not working, this one just seems to be particularly stable. Adding that line of code will change memory locations but its existence may possibly also change startup timing and code alignment and there may be other minor knock-on effects but at the moment I don't have a solid root-cause.

Are there any specific startup constraints with the FreeRTOS build that I need to be aware of? There seems to be no timing constraint between R3_2_Init() and osKernelStart(), should there be?

Has anyone seen anything like this before, anything else I might have missed?

I also note that Inc/stm32g4xx_hal_conf.h defines TICK_INT_PRIORITY to be 0, with a comment that is lowest. 0 is usually highest in cortex. Is that intentional? Is there a requirement for TICK_INT_PRIORITY to be 0?

Thanks

Clive

[STuser2]

I always felt FOC can't be done with RTOS due to the addition of extra code and managing the tasks. Any advantage we get by using RTOS? 

[Clive_S]

Is that something you have tried and had issues with? My concern is there may be other undiscovered issues in the ST code. Though FOC works, I'm not sure how refined the bits around it are eg the whole start-transition phase is pretty poor.

Out of interest is you comment FOC specific or do you have a view on RTOS and 6-step?

Basically its a project requirement and startup aside FOC seems to be working. Obviously I'll need to be careful with interrupts and priorities.

[GMA]

Hello @Clive_S,

Which MCSDK version are you using?
If it is MCSDK 6.4x, an issue has been identified on our side. Could you try commenting out the while loop?

__weak void *R3_2_TIMx_UP_IRQHandler(PWMC_R3_2_Handle_t *pHandle)
{
...
    if (OPAMPParams != NULL)
    {
      /* We can not change OPAMP source if ADC acquisition is ongoing (Dual motor with internal opamp use case) */
//      while (0x0u != pHandle->pParams_str->ADCDataReg1[pHandle->_Super.Sector]->JSQR)
//      {
//        /* Nothing to do */
//      }

[Clive_S]

Hi @GMA ,

I have already tried just breaking out of the while loop, but it was not enough. By the time it gets there it has lost an event and doesn't recover. The workaroud we are testing is a proof of concept and a bit messy, but basically we break from the while and skip the subsequent setup.

if  (0x0u != pHandle->pParams_str->ADCDataReg1[pHandle->_Super.Sector]->JSQR) {
    retrigger = 1;
    goto retrigger;
}

 Next time we come through, we clear the retrigger flag and don't retrigger. After that it all goes back to normal.
Hmm.. I should also fix those variable names!

  if (retrigger) {
      retrigger = 0;
  } else {
      pHandle->pParams_str->ADCDataReg1[pHandle->_Super.Sector]->JSQR = pHandle->pParams_str->ADCConfig1[pHandle->_Super.Sector] | (uint32_t) pHandle->ADCTriggerEdge;
      pHandle->pParams_str->ADCDataReg2[pHandle->_Super.Sector]->JSQR = pHandle->pParams_str->ADCConfig2[pHandle->_Super.Sector] | (uint32_t) pHandle->ADCTriggerEdge;
     /* Enable ADC trigger source */
retrigger:
     /* LL_TIM_CC_EnableChannel(TIMx, LL_TIM_CHANNEL_CH4) */
     LL_TIM_SetTriggerOutput(TIMx, LL_TIM_TRGO_OC4REF);
  }

So far this has been stable and only appears to occur at startup.

Cheers

Clive

[Clive_S]

And yes MCSDK_v6.4.0

Cheers

Clive

[SAN92]

I'm seeing the exact same issue on a STM32G491RE without any RTOS!

[Clive_S]

Hi @SAN92 

Thanks, that is useful to know. Does it only happen at startup and is there enough info above to work around it?

Clive

[SAN92]

Hi,

I was not running the motor continuously for a long time, I actually wanted to verify the reliability of my rev-up sequence by starting and stopping the motor in a loop and there I saw the lock-up happening on roughly 1% of all attempts.

The workaround proposed by GMA that just removes the while loop entirely does not seem to work for me.

I tried to confirm your analysis by adding a debug variable that increments with every call of TIMx_UP_M1_IRQHandler() and decrements with every call of ADC1_2_IRQHandler().

By the time I get stuck at the while loop this counter already reads 65, not sure how that could happen. 65=64+1?

This is what my call stack looks like:

R3_2_TIMx_UP_IRQHandler
<Exception frame>
FOC_HighFrequencyTask
TSK_HighFrequencyTask
<Exception frame>
LL_TIM_CC_DisableChannel
R3_2_CurrentReadingPolarization

Errata 2.6.1 tells us there is a way for the conversion sequence to start without waiting for the trigger if JSQR is written. And your workaround code wraps around the only location where JSQR is written - coincidence?

I'm not very familiar with the MCSDK yet so I'm still kind of poking around in the dark..

BR

Update: the workaround suggested by Clive_S does not seem to work for me either..

[GMA]

Hello,

For the STM32G4 series with internal operational amplifiers (op amps) configuration, could you try using the attached file?