Field updating of STM32 with data in external qspi flash

[imtiaz]

Posted on July 05, 2017 at 02:51

Hi All,

We are using an STM32F7 processor.

We have a typical application that will store LCD images and other asset type data in qspi flash. The QSPI flash will be memory mapped. We also have external memory mapped RAM.

In the code the images will be specified to go into the external qspi flash section , I guess using a scatter file definition of memory regions and &sharppragma in the code.

Could anyone give me some direction as to how we could update this software in the field - where we are not only updating the internal 2Mb flash , but also the external qspi flash. Currently I am using the ST system bootloader to update the internal flash.

I am assuming that a customised bootloader will be needed based on the address of the data , internal flash starting at 0x08000000 and external 0x90000000   it will have two different API's to write to either internal or external memory.

Does that mean that I will need to work with a hex file rather than a binary?

The setup of our hardware is very similar to ST's F769 eval and discovery boards. However these are programmed using dedicated programming interfaces like swd/jtag and stlink etc which have customised loaders.

In our case we will be doing OTA updates.

Highly appreciate pointer in the right direction.

Best Regards

Imtiaz 

#qspi-flash-memory-mapped-bootload-stm32f7

[Jeroen3]

Posted on July 05, 2017 at 07:52

Yes, you will have to run your own OTA upgrade software. I do not know if there is anything available.

What exactly is unclear?

If you have external memory, you can:

Download the full application image to it.

Verify the image.

Reboot in upgrade mode, copy from external to internal flash.

Reboot in download mode.

Download updated external flash data.

Reboot to normal application.

[imtiaz]

Posted on July 05, 2017 at 23:55

Hi All,

This is my current plan , in theory :

1. From the hex file created - create two binaries , one for internal flash and the other for external flash based on the address in the hex record
2. To upgrade the internal flash area use the system boot-loader as normal (uart / SPI tested , DFU)
3. On application start up - a special command will be sent to update the external flash if needed. Application will wait for 100 msec before continuing.
4. If required QSPI flash will be erased , written and verifed and then 
5. Application start up continues and memory maps the qspi flash
6. ... and Bobs your uncle

Would appreciate any holes in this theory before I start implementing.

The advantage of this method that I can see is that there is no custom bootloader , chips can be programmed in production without any programmer. Internal flash and external flash do not always have to be updated if there is no change in one of them.

Disadvantage - if the internal and external flash code get out of sync there will be a hardfault.

Regards

Imtiaz

[Jeroen3]

Posted on July 06, 2017 at 07:22

That is not really OTA (Over The Air). But it should work.

What you can do instead of the 100ms wait and the risk on a hardfault is using CRC on the external memory.

And use something like

char *version = '__DATE__' '__TIME__';

to create a string you can compare on boot to see if both memories match the same firmware build.

http://srecord.sourceforge.net/

  should be able to split your hex file to binaries, and compute and store STM32 hardware crc compatible checksum.

Are you able to program both memories using jtag/swd in one go already?

[Jack Peacock]

Posted on July 06, 2017 at 11:33

Where an over-the-air update gets complicated is in recovering from an incomplete download.  What happens if you lose communications halfway through an update?  You only have a partial firmware image, or a corrupted QSPI image.  What happens if it is started?  Don't count on a fault, it may just go into a loop.

Using the system ROM bootloader is easy but doesn't leave you any way to recover.  The usual solution is a custom bootloader that can reestablish communications and recover from a failed update.  Many of the STM32L parts have dual bank flash for just this reason: if an update fails the app can be recovered back to previous version.  On a restart the custom bootloader does a checksum to verify images and will detect a failure on the current bank, so it falls back to previous.

Same problem with the QSPI.  You need enough space for two copies, new and backup.  If this is an OTA update in the field it has to cover just about any contingency since there's no one to do it manually.  As for syncing QSPI to firmware, all you need is a tag file with the same checksum as the firmware.  If they don't match you know there's a fault.

A bulletproof update procedure is not a trivial problem, that's why you don't see a lot of example code.  Communications can be lost at any point.  If the update is large, you need to track progress and recover from a known good point, otherwise comm costs can be significant (think of OTA updates over a cell modem, in the middle of the Mojave Desert with unreliable coverage, where a 1MB image has to be restarted 5-10 times before it finishes without an error).

  Jack Peacock

[Jeroen3]

Posted on July 06, 2017 at 11:55

That is why I suggested to download the new application image for internal memory to the external memory first. 

Then only proceed with flashing internal memory if the downloaded image is verified.

[imtiaz]

Posted on July 06, 2017 at 21:59

Thanks guys for your feedback.

I have a fairly good idea how to do this now. My main gaps were how to work with a program that relies on the external flash for the program to run and how to separate the two binaries.

@

Peacock.Jack

In regards to failure from recovery - I have a wifi chip that downloads the firmware , stores it in its internal flash and verifies the MD5 hash of the file. Only then does it proceed to update the STM32F7 firmware . The only issue is that it does not have sufficientmemory to keep both internal/external image at once. So it has to be done in two parts. But this is quite good since most of the time the data in the external flash wont be changing.

Also- _ yes I will be having some kind of a CRC / Hash to match up the two images.

@

Lodder.Jeroen

I will look into SRecord , my plan was to write my own script , thanks.

@

Lodder.Jeroen

'

Are you able to program both memories using jtag/swd in one go already?' - not yet as we have to write a QSPI loader for our flash chip for the external programming tool and the debug IDE

Regards

Imtiaz

[Jeroen3]

Posted on July 07, 2017 at 07:35

abdullah.abdullah

‌ I might be able to help you with SRecord, I have used it before to split and checksum a full.hex. If you open a topic about it, mention me.

[imtiaz]

Posted on July 10, 2017 at 04:10

Hi Jeroen,

Can I take you up on the offer to help in using srecord .. just a few quick

pointers in how I can split the hex file into a couple of binaries.

I have downlaoded srecord-1.6.3-win32

Thanks

Imtiaz

[AVI-crak]

Posted on July 10, 2017 at 06:12

There is a big difference in the algorithms for the programmer, and in the algorithms for the device user.

For the programmer, quick debugging is required, in such cases, every update for 10 minutes is very difficult. It is necessary to perform separate compilation of the code - for the microprocessor and for external memory.

To separate the compilation, you must clearly define your desires. Collect all the data for writing in qspi in one place, into one huge static structure. And place this structure in the qspi section.A static structure may have data padding, or may not have it - from the point of view of the program code, it does not matter. However, there is a very big difference in the compilation speed of the code - 30 seconds or 2 hours (+ qspi 256MB). You just need to put the data to be filled into the processing of the re-processor by two values - process / ignore. I have a lot of small data, I use the connection files, otherwise it's very easy to get confused.

What we get:

Accessing data through a structure, bypassing magic addresses. GCC very optimally optimizes the start addresses of accesses to external memory. Even if you have a tenfold nesting of the fields of the structure - the starting address will be direct.Free editing of the main program, except for new graphic data.The headache bonus is an executable file for firmware 2 gigabytes in size. This is solved by the GCC configuration commands - exclude the qspi section from the firmware image. The appeal to memory is preserved, the time is shortened, the development will get acceleration.

The firmware for the user has a different structure.

A handheld bootloader in the initial addresses of the microprocessor, and a freely distributable firmware for the user. I strongly advise against taking the bootloader code from popular free resources - write your own. The worse your code is, the more difficult it will be for the Chinese to fake your device.Algorithms for data validation are a great many. But you have the advantage of not being killed by the bootloader. Which by the way can not be copied to qspi before updating, everything else is possible - but it can not !!!!By the way, the loss of power during the firmware update on the device is a user problem. This is exactly what is written in the instructions. Brick exchange or refund is not. If you ever give a slack - then you can say goodbye to your finances.