After downloading a new firmware blob via YMODEM transmssion, why does SBSFU report that installation is not validated instead of installing the new firmware?

tomptz · ‎2021-09-14

We implemented an OTA update procedure where we provide an STM32L476 with a new firmware via serial using the Ymodem protocol in a two-image SBSFU configuration.

We are executing long-running tests in which we use the same firmware image to test the update procedure over and over again.

The majority of update attempts succeed, but once in a while, we see that SBSFU manages to download the new firmware and reboots, but instead of installing the new firmware (swapping the images) , SBSFU reports that our installation was not validated and a rollback is attempted. A rollback should never be performed at this stage, because the backup firmware was overwritten with the new firmware and the image swap has not yet been executed.

This problem is difficult to reproduce but we have seen it on multiple of our hardware prototypes.

See SBSFU log below.

----

= [SBOOT] System Security Check successfully passed. Starting...

======================================================================

= =

= Secure Boot and Secure Firmware Update =

======================================================================

= [SBOOT] SECURE ENGINE INITIALIZATION SUCCESSFUL

= [SBOOT] STATE: CHECK STATUS ON RESET

INFO: A Reboot has been triggered by a Hardware reset!

INFO: Last execution detected error was: No error. Success.

= [SBOOT] STATE: CHECK NEW FIRMWARE TO DOWNLOAD

= [SBOOT] STATE: DOWNLOAD NEW USER FIRMWARE

File> Transfer> YMODEM> Send

= [SBOOT] STATE: REBOOT STATE MACHINE

========= End of Execution ==========

= [SBOOT] System Security Check successfully passed. Starting...

======================================================================

= =

= Secure Boot and Secure Firmware Update =

======================================================================

= [SBOOT] SECURE ENGINE INITIALIZATION SUCCESSFUL

= [SBOOT] STATE: CHECK STATUS ON RESET

INFO: A Reboot has been triggered by a Software reset!

INFO: Last execution detected error was: No error. Success.

= [SBOOT] STATE: CHECK NEW FIRMWARE TO DOWNLOAD

= [SBOOT] STATE: CHECK USER FW STATUS

Installation not validated: rollback procedure initiated (SLOT_ACTIVE_1 / SLOT_DWL_1)

========= End of Execution ==========N TO PREVIOUS USER FIRMWARE

= [SBOOT] System Security Check successfully passed. Starting...

======================================================================

= =

= Secure Boot and Secure Firmware Update =

======================================================================

= [SBOOT] SECURE ENGINE INITIALIZATION SUCCESSFUL

= [SBOOT] STATE: CHECK STATUS ON RESET

INFO: A Reboot has been triggered by a Software reset!

INFO: Last execution detected error was: Backed-up firmware not identified.

= [SBOOT] STATE: CHECK NEW FIRMWARE TO DOWNLOAD

= [SBOOT New Fw to be installed from slot SLOT_DWL_1

= [SBOOT] STATE: HANDLE CRITICAL FAILURERE

= [EXCPT] INSTALLATION CANCELLED: FORBIDDEN VERSION!

= [SBOOT] STATE: REBOOT STATE MACHINE

========= End of Execution ==========

= [SBOOT] System Security Check successfully passed. Starting...

======================================================================

= =

= Secure Boot and Secure Firmware Update =

======================================================================

= [SBOOT] SECURE ENGINE INITIALIZATION SUCCESSFUL

= [SBOOT] STATE: CHECK STATUS ON RESET

INFO: A Reboot has been triggered by a Software reset!

INFO: Last execution detected error was: Firmware version rejected by anti-rollback.

= [SBOOT] STATE: CHECK NEW FIRMWARE TO DOWNLOAD

= [SBOOT] STATE: CHECK USER FW STATUS

No valid FW found in the active slots nor new FW to be installed

Waiting for the local download to start...

= [SBOOT] STATE: DOWNLOAD NEW USER FIRMWARE

File> Transfer> YMODEM> Send

[Ready to receive]..............................................................= [SBOOT] STATE: HANDLE CRITICAL FAILURE.........[Abort: hal busy]

= [EXCPT] COM ERROR DURING DOWNLOAD !

= [SBOOT] STATE: REBOOT STATE MACHINE

========= End of Execution ==========

---

Any help would be appreciated.

Regards,

Tom

alister · ‎2021-09-15

>once in a while, we see that SBSFU manages to download

The X-CUBE-SBSFU is an example package that you've adapted to your requirements.

Your upload is implemented like this?

a client signals the app it wants to upload a new app
the app reboots passing that trigger somehow to its bootloader (i.e. SBSFU), or the app re-enters its bootloader calling an upload-app entry point
the bootloader uploads the new app using ymodem and, if the upload was successful, it writes its header to the swap sector to trigger its install

>but instead of installing <snip>, SBSFU reports that our installation was not validated and a rollback is attempted

As your bootloader is already executing, you should be able to debug this easily by putting breakpoints at strategic places.

>[SBOOT] STATE: HANDLE CRITICAL FAILURE.........[Abort: hal busy]

This is a clue where at least one bug lies.

If the console output is accurate, HAL drivers have busy flags.

A busy usually means the driver is being called by more than one thread or before an operation started by an earlier call has completed.

If you trace how the code gets to the line that sends that to the console, you should be able to find some suitable places to put breakpoints and restart the test, or you might be able to see just by reading where your bug is.

As X-CUBE-SBSFU is example code, you should read and accept what lines you you want and remove or replace what lines you don't want anyway.

Jocelyn RICARD · ‎2021-09-20

Hello Tom,

As you are testing update again and again, one possible cause of your issue could be the flash itself.

The endurance of the flash on STM32L4 is given at 10K cycles.

The SBSFU update procedure can use several write-erase cycles on some flash sectors.

Best regards

Jocelyn