flash CRC integrity

[klemen]

Posted on January 15, 2014 at 21:14

Hello,

I know this has been discussed before, but reading through it all, I am still unable to grasp the solution of my problem:

What I am trying to do is check the integrity of the flash (code) on the STM32L1xx chip at startup. Since I need to calculate the CRC only at startup, I have decided not to use the built in CRC peripheral (mostly because of the 4 bytes step). So instead I used 1 byte step CRC with lookup table using the same polynomial (reversed actually, i.e. 0xEDB88320) to calculate the CRC of the .hex file.

Now, I am trying to insert the calculated CRC in the .hex file to upload it to stm32 and perform the check.

My questions are:

1.)  How can I insert the CRC to a specific address, so that I will be able to access the same address via the code/function in the program? Can I do it simply in a hex editor?

2.) The flash address (default) begins at 0x08000000 and ends  at 0x0801FFFF. My code is shorter. How can I calculate the CRC only for the code portion and not the entire flash at runtime? If I calculate the CRC for the entire flash it would be different right?

I was thinking about moving the flash start address for 4 bytes forward and write the CRC in the blank spot, so the address would be known.

Any suggestions please?

Thank you and best regards,

K

#i-accept-paypal

[Tesla DeLorean]

Posted on January 15, 2014 at 21:39

Pad the unused portion with a known fill character, CRC the whole space, less the four you are placing the data in. Use the HW, and use the same one in your packaging process. The FLASH is a multiple of 4 bytes, so no issue there. If you run the CRC test across the WHOLE FLASH, including the bytes you added, the nature of most computation (unless structured otherwise) is ZERO.

Note if you feed the content of CRC->DR back into itself, it will ZERO the register. The CRC is like a remainder to a long division, if you feed in the remainder, the division completes without a remainder.

Generally one would take the .HEX and process it, apply the CRC or checksum, tag it with sizes, etc, and then export the image you want to place in FLASH/ROM

How does someone move the FLASH memory forward? and have the processor silicon figure that out? Surely it's easier to write a simple C app to process the .HEX file.

When reading the .HEX, or .ELF/AXF one can figure out the size.

[chen]

Posted on January 16, 2014 at 10:11

Hi

''I was thinking about moving the flash start address for 4 bytes forward and write the CRC in the blank spot, so the address would be known.''

As Clive said - you cannot!

The start of your binary will be the vector table. The vector table must be on certain boundaries - you cannot arbitrarily move it along by 4 bytes!

''1.)  How can I insert the CRC to a specific address, so that I will be able to access the same address via the code/function in the program?''

You can create sections in the link map/linker file and place each section at a label.

As clive said - write a program that does the checksum, put this program into your build process. Take the calculated checksum and insert into you binary at the label/section (from  the linker)

You can make the compiler aware of this label by declaring it. Once the compiler knows about it - you can access it in your program - ie

'' I am trying to do is check the integrity of the flash (code) on the STM32L1xx chip at startup''

Your program can run the integrity check itself.

It is not a good idea to locate the checksum at a fixed address after you program (what happens if your program grows and your overflow past the CS address?)

You can either locate it somewhere at the beginning (inbetween the vector table and the start of program) or at the end (it always moves along with the end of your binary).

[klemen]

Posted on January 16, 2014 at 14:51

Dear Sirs,

thank you both for your answers. Regarding the movement of the address where the code begins - I am using Keil and under target options there is a window where the start address and the length can be modified (please see the image below). I have not tried this, and probably I am mistaken. Anyway, regarding the CRC calculation (following clive1 advice), I have created a simple program that reads the .hex file, pads the file with space characters up to flash size minus 4 bytes (0x00040000 - 4) and calculates the CRC. This CRC is then appended to the .hex file which yields the total size of the .hex file 0x00040000. The CRC of the entire file is then calculated and the output is indeed 0 (please see the image below). The CRC calculation I am doing on the .hex file has the same algorithm as the HW CRC on the STM32 (I have checked both on a example data in the debug mode and the output was the same). First, I have successfully uploaded the .hex file (without CRC check) to the controller and the code works fine. But uploading the program with the CRC check, the program loops in a while loop I have placed within the code after the CRC calculation: while(crc32 != 0), which means that the CRC does not sum (divide) to 0. My code on the STM for whole flash calculation:

// DECLARATION OF VARIABLES
const uint32_t FLASH_START_ADDRESS = 0x08000000; // Flash start address
const uint32_t FLASH_LENGTH = 0x00040000; // Flash size minus the 4 bytes for CRC
uint32_t crc32_flash = 0; // CRC flash calculation variable
.
.
.
// CALLER FUNCTION
// Calculate flash CRC for code integrity check
crc32_flash = crc32_flash_calculate((uint32_t*) FLASH_START_ADDRESS,FLASH_LENGTH);
// SOURCE
uint32_t crc32_flash_calculate(const uint32_t *flash_start_address, const uint32_t flash_length) {
// uint32_t pointer to flash address (FA) start 
uint32_t *pFA = (uint32_t*) flash_start_address;
// Flash length (FL) in bytes is divided by 4 (4 bytes step - uint32_t) 
uint32_t FL = (uint32_t) flash_length/4; 
uint32_t i, crc32; 
// Enable the CRC peripheral clock
RCC_AHBPeriphClockCmd(RCC_AHBPeriph_CRC,ENABLE);
// Reset the CRC data register (initial value 0xFFFFFFFF) 
CRC_ResetDR();
// Calculate CRC32 for the entire flash block
for(i=0;i<
FL
;i++) { 
CRC->DR = pFA[i];
} 
// Final CRC32
crc32 = CRC->DR; 
// Disable the CRC peripheral clock
RCC_AHBPeriphClockCmd(RCC_AHBPeriph_CRC,DISABLE);
// Return calculated CRC
return (crc32); 
}

And ending of the modified .hex file (the first space (20) is right after the program the compiler generates): etc... and ending: How does the compiler/linker know where the .hex program ends. Because it seems to me it does not upload the entire .hex file? You were talking about some tags? Could you please explain this? Really stuck, and appreciate any help and advice (again...). Thank you and best regards, K

[jpeacock2399]

Posted on January 16, 2014 at 15:48

You can store the flash checksum in the OTP area of the STM32.  As you installl field updates move to the next OTP position.  This gives you a fixed location for the checksum, outside the image, and a history of updates.

  Jack Peacock

[chen]

Posted on January 16, 2014 at 16:05

Hi

'' Regarding the movement of the address where the code begins - I am using Keil and under target options there is a window where the start address and the length can be modified (please see the image below).''

You cannot just insert data at the beginning of the binary.

AS I HAVE SAID 0x8000000 is the default boot address for STM32.

If you place data here - it will be mistaken as the boot address for 1)stack, 2)program counter.

It does not matter what tool you use to do it!

''Anyway, regarding the CRC calculation (following clive1 advice), I have created a simple program that reads the .hex file, pads the file with space characters up to flash size minus 4 bytes (0x00040000 - 4) and calculates the CRC. This CRC is then appended to the .hex file which yields the total size of the .hex file 0x00040000. The CRC of the entire file is then calculated and the output is indeed 0 (please see the image below)?''

Did you read this file as an Intel Hex?

Did you interpret the binary image correctly?

''First, I have successfully uploaded the .hex file (without CRC check) to the controller and the code works fine. But uploading the program with the CRC check, the program loops in a while loop I have placed within the code after the CRC calculation:''

Did you write a new Intel Hex file with your CRC changes and load that?

''How does the compiler/linker know where the .hex program ends.''

The linker knows where the end is because it appends all the compiled sections together. That is its job.

''Because it seems to me it does not upload the entire .hex file?''

Did you add the checksum into the hex file yourself or let the compiler/linker do it for you?

''You were talking about some tags? Could you please explain this?''

Not in great detail - read the GNU Linker manual for full details.

The compiler output will have different sections eg .text

The .text is he compiled code.

The linker takes the output from the compiler and joins together all the same sections from all the modules.

The linker then puts each section into a physical location (address inside the device address space for RAM, ROM etc).

If you define a label/tag in your code that the compiler does not know what to do with, the compiler passes this along.

When it reaches the linker - you can take that tag (it gets treated as a section) and tell the linker to put it into the address space somewhere in the device.

[Tesla DeLorean]

Posted on January 16, 2014 at 19:00

The .HEX is a packaging format, this is technically not what gets uploaded to the part. So you need to be sure you're doing the CRC of the binary data encapsulated by the file, not the hex data file itself.

The linker knows the size of the things it's creating, these are also expressed in the .ELF/.AXD file in descriptors about the various sections. From the .HEX file the end can be inferred from the last described location (maximal address). One would typically have the parsing app know the size of the finished ROM, it would fill the space with something like 0xFF, and then fill in the content by processing the .HEX file.

By tagging it, I mean you can put size and other data into the image in a post link step. For instance you could have a ''static const unsigned char sig[] ='' type construct and the tool you add thing like a size, or serial number to that before performing the check summing, etc. You could also place the size, or other information into unused vector table entries.

[klemen]

Posted on January 17, 2014 at 14:02

Thank you very much for your answers.

Indeed, it was not a proper intel .hex file. I appreciate all the help, but I will not be able to implement my own CRC check algorithm, because of the current lack of knowledge and to be honest at the moment I think it is not worth the time. I have been struggling for full three days now to write 4 bytes to the image and calculate the CRC of the flash! I gave up the idea and now I am trying to use the SRecord to modify the .hex file. But again, I am having problems. The SRecord has a STM32 crc checksum module, but I decided to first try the CRC32-CCITT calculation. I applied the same algorithm that the SRecord uses (at least it should be the same) on the ARM uC:

// Fast (LUT) CRC32 calculation update (one call, length bytes)
uint32_t crc_32_update(uint8_t *data, uint32_t length) {
// 4-Byte table 0xEDB88320 polynomial
static const uint32_t crc32_tbl[] = {0x00000000,0x77073096,0xEE0E612C,0x990951BA,0x076DC419,0x706AF48F,0xE963A535,0x9E6495A3,
0x0EDB8832,0x79DCB8A4,0xE0D5E91E,0x97D2D988,0x09B64C2B,0x7EB17CBD,0xE7B82D07,0x90BF1D91,
0x1DB71064,0x6AB020F2,0xF3B97148,0x84BE41DE,0x1ADAD47D,0x6DDDE4EB,0xF4D4B551,0x83D385C7,
0x136C9856,0x646BA8C0,0xFD62F97A,0x8A65C9EC,0x14015C4F,0x63066CD9,0xFA0F3D63,0x8D080DF5,
0x3B6E20C8,0x4C69105E,0xD56041E4,0xA2677172,0x3C03E4D1,0x4B04D447,0xD20D85FD,0xA50AB56B,
0x35B5A8FA,0x42B2986C,0xDBBBC9D6,0xACBCF940,0x32D86CE3,0x45DF5C75,0xDCD60DCF,0xABD13D59,
0x26D930AC,0x51DE003A,0xC8D75180,0xBFD06116,0x21B4F4B5,0x56B3C423,0xCFBA9599,0xB8BDA50F,
0x2802B89E,0x5F058808,0xC60CD9B2,0xB10BE924,0x2F6F7C87,0x58684C11,0xC1611DAB,0xB6662D3D,
0x76DC4190,0x01DB7106,0x98D220BC,0xEFD5102A,0x71B18589,0x06B6B51F,0x9FBFE4A5,0xE8B8D433,
0x7807C9A2,0x0F00F934,0x9609A88E,0xE10E9818,0x7F6A0DBB,0x086D3D2D,0x91646C97,0xE6635C01,
0x6B6B51F4,0x1C6C6162,0x856530D8,0xF262004E,0x6C0695ED,0x1B01A57B,0x8208F4C1,0xF50FC457,
0x65B0D9C6,0x12B7E950,0x8BBEB8EA,0xFCB9887C,0x62DD1DDF,0x15DA2D49,0x8CD37CF3,0xFBD44C65,
0x4DB26158,0x3AB551CE,0xA3BC0074,0xD4BB30E2,0x4ADFA541,0x3DD895D7,0xA4D1C46D,0xD3D6F4FB,
0x4369E96A,0x346ED9FC,0xAD678846,0xDA60B8D0,0x44042D73,0x33031DE5,0xAA0A4C5F,0xDD0D7CC9,
0x5005713C,0x270241AA,0xBE0B1010,0xC90C2086,0x5768B525,0x206F85B3,0xB966D409,0xCE61E49F,
0x5EDEF90E,0x29D9C998,0xB0D09822,0xC7D7A8B4,0x59B33D17,0x2EB40D81,0xB7BD5C3B,0xC0BA6CAD,
0xEDB88320,0x9ABFB3B6,0x03B6E20C,0x74B1D29A,0xEAD54739,0x9DD277AF,0x04DB2615,0x73DC1683,
0xE3630B12,0x94643B84,0x0D6D6A3E,0x7A6A5AA8,0xE40ECF0B,0x9309FF9D,0x0A00AE27,0x7D079EB1,
0xF00F9344,0x8708A3D2,0x1E01F268,0x6906C2FE,0xF762575D,0x806567CB,0x196C3671,0x6E6B06E7,
0xFED41B76,0x89D32BE0,0x10DA7A5A,0x67DD4ACC,0xF9B9DF6F,0x8EBEEFF9,0x17B7BE43,0x60B08ED5,
0xD6D6A3E8,0xA1D1937E,0x38D8C2C4,0x4FDFF252,0xD1BB67F1,0xA6BC5767,0x3FB506DD,0x48B2364B,
0xD80D2BDA,0xAF0A1B4C,0x36034AF6,0x41047A60,0xDF60EFC3,0xA867DF55,0x316E8EEF,0x4669BE79,
0xCB61B38C,0xBC66831A,0x256FD2A0,0x5268E236,0xCC0C7795,0xBB0B4703,0x220216B9,0x5505262F,
0xC5BA3BBE,0xB2BD0B28,0x2BB45A92,0x5CB36A04,0xC2D7FFA7,0xB5D0CF31,0x2CD99E8B,0x5BDEAE1D,
0x9B64C2B0,0xEC63F226,0x756AA39C,0x026D930A,0x9C0906A9,0xEB0E363F,0x72076785,0x05005713,
0x95BF4A82,0xE2B87A14,0x7BB12BAE,0x0CB61B38,0x92D28E9B,0xE5D5BE0D,0x7CDCEFB7,0x0BDBDF21,
0x86D3D2D4,0xF1D4E242,0x68DDB3F8,0x1FDA836E,0x81BE16CD,0xF6B9265B,0x6FB077E1,0x18B74777,
0x88085AE6,0xFF0F6A70,0x66063BCA,0x11010B5C,0x8F659EFF,0xF862AE69,0x616BFFD3,0x166CCF45,
0xA00AE278,0xD70DD2EE,0x4E048354,0x3903B3C2,0xA7672661,0xD06016F7,0x4969474D,0x3E6E77DB,
0xAED16A4A,0xD9D65ADC,0x40DF0B66,0x37D83BF0,0xA9BCAE53,0xDEBB9EC5,0x47B2CF7F,0x30B5FFE9,
0xBDBDF21C,0xCABAC28A,0x53B39330,0x24B4A3A6,0xBAD03605,0xCDD70693,0x54DE5729,0x23D967BF,
0xB3667A2E,0xC4614AB8,0x5D681B02,0x2A6F2B94,0xB40BBE37,0xC30C8EA1,0x5A05DF1B,0x2D02EF8D};
// Allocate variable for sequential crc calculation and initialize with 0xFFFFFFFF
uint32_t crc32 = 0xFFFFFFFF;
// Pointer to data for incrementing 
uint8_t *tmp = (uint8_t*) data;
// Process 8-bits at a time
while(length > 0) {
crc32 = (crc32 >> 8) ^ crc32_tbl[( crc32 ^ (*tmp) ) & 0xFF];
// Increment data pointer and decrement length
tmp++;
length--;
}
return (crc32 ^ 0xFFFFFFFF); 
}

Is it correct to call the function to calculate the flash crc as so (the device I am using has the address range of 0x08000000 - 0x0803FFFF):

const uint32_t FLASH_START_ADDRESS = 0x08000000; // Flash start address
const uint32_t FLASH_LENGTH = 0x00040000; // Flash size
crc32_flash = crc_32_update((uint8_t*) FLASH_START_ADDRESS,FLASH_LENGTH);

When the compiler/linker produces the .hex file I use the following command in SRecord to calculate the CRC and to append it as the last four bytes:

srec_cat test.hex -intel -crop 0x08000000 0x0803FFFC -fill 0xFF 0x08000000 0x0803FFFC -crc32-l-e 0x0803FFFC -CCITT -o test_out.hex -intel

The uC program currently has a while loop that loops indefinitely if the CRC is not equal to 0. And it loops indefinitely... I have noticed (on the flash progress bar) that when programming the chip the start address is indeed 0x08000000, but the end address is 0x0803FF00 (when it supposedly should be 0x0803FFFF). Please help as this is really getting to me. Best regards, K

[chen]

Posted on January 17, 2014 at 15:21

Hi

''Thank you very much for your answers.''

''I gave up the idea and now I am trying ''....

Hang on in there - we will ry our best to help you.

''I applied the same algorithm that the SRecord uses (at least it should be the same) on the ARM uC:''

Not sure what you mean by SRecord here. Google 'S-Record' and you should find it is a common term for Motorola S-Records - similar thing to Intel Hex.

The code looks like a CRC32 generator.

''Is it correct to call the function to calculate the flash crc as so (the device I am using has the address range of 0x08000000 - 0x0803FFFF):''

It will generate some kind of CRC. I do not know if it is the same algorithm in the ARM.

''When the compiler/linker produces the .hex file I use the following command in SRecord to calculate the CRC''

Again - The SRecord may not be raw binary that you think it is.

Lets take this in steps :

1. Get binary image

2. Algorithm to use for validation

3. Use algorithm on binary image to generate a 'check value'

4  Decide how/where 'check value' is going to reside in binary.

5 How do we get 'check value' into binary

Step 1:

What compiler/linker are you using?

Does it have an option to output raw binary (or are you going to have invesigate/write a S-Rec/IntelHex utility)

[Tesla DeLorean]

Posted on January 17, 2014 at 16:37

In the case where you invert the final CRC you break zero test option, the math doesn't work that way. You should in that case CRC the image on the processor, less the final 4-bytes, and then do a compare against those.

Print out those numbers if it is helpful for debugging/understanding what is going on.

Yeah, 3-days is a bit of a sad indictment, I could code a workable intel hex parsing app that could checksum/crc in an hour or so. I've written monitor apps that can do this in 8-bit assembler, and before the internet existed, it's a generational thing.

//****************************************************************************
uint32_t Crc32FastBlock(uint32_t Crc, uint32_t Size, uint8_t *Buffer)
{
 static const uint32_t CrcTable[] = { // For 0xEDB88320 Polynomial
 0x00000000,0x1DB71064,0x3B6E20C8,0x26D930AC,0x76DC4190,0x6B6B51F4,0x4DB26158,0x5005713C,
 0xEDB88320,0xF00F9344,0xD6D6A3E8,0xCB61B38C,0x9B64C2B0,0x86D3D2D4,0xA00AE278,0xBDBDF21C };
 while(Size--)
 {
 Crc = Crc ^ (uint32_t)*Buffer++;
 Crc = (Crc >> 4) ^ CrcTable[Crc & 0x0F]; // Right shifting, two rounds of 4-bits
 Crc = (Crc >> 4) ^ CrcTable[Crc & 0x0F];
 }
 return(Crc); // [e-mail]
}
//****************************************************************************
void FlashCrcTest(void)
{
 uint32_t CrcExpect, CrcCompute;
 uint8_t *RomBuffer = (uint8_t *)0x08000000;
#define ROMSIZE 0x40000
 CrcCompute = ~Crc32FastBlock(0xFFFFFFFF, (ROMSIZE - 4), RomBuffer);
 CrcExpect = *((uint32_t *)&RomBuffer[ROMSIZE - 4]);
 if (CrcCompute == CrcExpect)
 puts("Success");
 else
 printf("Fail %08X != %08X\n", CrcCompute, CrcExpect);
}
//****************************************************************************

Edit: Stupid forum ballsing up posts

[klemen]

Posted on January 17, 2014 at 18:42

Hello Mr. sung.chen_chung,

So step 1:

I am using Keil, which outputs a .hex but can be configured post build to output .bin also (using fromelf).

Step 2:

For validation I would preferably like to use the STM32's inbuilt CRC calculation. I have configured the CRC peripheral and also wrote a custom function which calculates the CRC in the same way. I've tested the outputs of both on the same data (entire ROM) and the output was the same.

Step 3, 4, 5 -> here is where my problem lies. It seems I do not know how to properly calculate the CRC for the .bin image and inject it to the end of the image.

@ Mr.clive1:

So if I calculate the CRC of the entire ROM minus 4 bytes and append the calculated 4 byte CRC at the end, and then use the same CRC algorithm on the entire ROM, the result will not be zero? Guess that was my first fundamental mistake...

The algorithm you provided should give the same output as the one I attached in the last post (minus the last 4 bytes), right?

Again, my main problem is I do not know how to properly fill the entire ROM, i.e. the .hex/.bin file (minus the last 4 bytes), calculate the CRC and append it to the end. It seems strange to me that there is so little information regarding this on the internet. The document of the STM32 states that the CRC is used for flash integrity check, but there are no examples or guidelines how to do this. As I have read, IAR has this integrated in the SW.

I would just need some standard procedure on how to verify the flash integrity using the CRC peripheral.

Thank you for your help and best regards,

K