cancel
Showing results for 
Search instead for 
Did you mean: 

C / C++ memory layout and optimization, problem with type punning, loading and saving.

HTD
Senior III

I have some data as static class members. I use `__attribute__((__packed__))` for some classes and structs. I use type punning to convert between types that I believe have the same memory layout, but different signatures (members of different translation units). It works. I assume the sequence and offsets of members is as in my class / struct definition. Knowing this I just save and load memory regions to and from files and it works.

Until I enable any optimization in the compiler. I noticed that when my configuration file was written by the debug version, it does not load in release version. The first member is loaded correctly, the subsequent ones are not.

My only guess here it can be related to a different memory layout using -O1 .. -O3. Can `__attribute__((__packed__))` be ignored when optimizations enabled? Is there a way to express in my code I want a specific memory layout for a structure?

When I have a class that has only static members, is it equivalent of a C struct in memory? When the class is marked with the attribute, does it affect its static members? Is it a way to ensure specific members are always put in specific relative memory locations?

Consider the code:

#include "stdint.h"
 
class __attribute__((__packed__)) X
{
public:
    static uint8_t a;
    static uint16_t b;
    static uint32_t c;
};
 
uint8_t X:a;
uint16_t X:b;
uint32_t X:c;
 
struct __attribute__((__packed__)) Y
{
    uint8_t a;
    uint16_t b;
    uint32_t c;
};
 
int main(void)
{
    Y s;
    s.a = 1;
    s.b = 2;
    s.c = 3;
    memcpy((void*)X.a, &s, sizeof(Y));
    bool ok = s.a == X.a && s.b == X.b && s.c == X.c;
}

When compiled with -O0, the `ok` is true. When compiled with -O1 - the `ok` is false.

What am I missing here?

12 REPLIES 12
Javier1
Principal

>>I noticed that when my configuration file was written by the debug version, it does not load in release version.

I fight with this every day.

there is a mess between configuration for debug build/release builds, it gets worse if you changed the name of the project at any point.

maybe this helps you

https://community.st.com/s/question/0D53W000011upRKSAY/how-can-i-get-packed-attribute-to-work-in-the-gcc-compiler-used-in-the-stm32cubeide

we dont need to firmware by ourselves, lets talk

Wow, how did you guess that I've changed my project's name? 😉 However, no file location issues so far, however, the __packed__ attribute seems to work weird. Thanks for the hint, I'll just code a test module and add it to my target project to be able to quickly test various configurations. I will test type punning and `memcpy()` - if the data are where I expect them to be.

BTW, changing the project's name works only with editing the .project (XML) file, doing that in STM32CubeIDE or worse, in TouchGFX initiates the project's self-destruct sequence 😉 So, just change project tag content, reload everything, then replace all the references and some configuration (build) settings manually. Works as charm. However, with the compiler optimization there's something odd going on and I will figure it out today.

HTD
Senior III

Unfortunately, that's not the problem. Packed attribute (and `#pragma pack(push, 1) / #pragma pack(pop)`) works regardless of optimization options. The struct data are packed, however, the ORDER of variables is changed!

Here are my static members in order of definition in class header:

0693W00000Y8wZVQAZ.pngNotice the weird address assignments! Without optimization the order of the static members is as defined in the header.

The obvious fix for it is to create a separate structure for the data and use that structure for dumping data to a file, but that's adding probably unnecessary complexity to a simple piece of code.

Bob S
Principal

GCC can/will change the order of variables in memory depending on optimization levels. I hit this issue with the AVR GCC when allocating variables in the EEPROM space. Data written when compiled with "-Og" were not readable with code compiled with code compiled with "-Os". The solution, as you mention above, was to declare a structure that filled the entire EEPROM space and put the variables in the structure. The order inside the structure is guaranteed by the C/C++ spec.

I haven't seen this issue with the STM32, but none of my STM32 code attempts to do what you are doing. I suspect you are seeing that same issue, with the three static members of X being placed in memory in a different order with different optimizations.

Pavel A.
Evangelist III

IMHO your code in lines 3-13 is er... to say it mildly ....

A POD type with all members public in C++ is called 'struct', no members need to be static.

But the usage of __attribute__((packed)) in gcc has non-intuitive subtleties. Modern gcc versions issue warnings when the __attribute__((packed)) is misplaced (and not honored). Do not ignore these warnings.

Maybe pragma pack is easier to use, as in the thread quoted by @Javier Muñoz​ 

Layout of plain old data types (structs/unions) does not depend on optimization level alone. Compiler folks do not dare to do that, otherwise angry users would come to them with fire and pitchforks.

Type punning can be broken by optimization (if you know what it is, you know why it is questionable).

It was the problem old as dinosaurs 😉 I had a piece of code working as a tool (thus static) and it had some stored data (configuration). In a hurry, I mixed it in a way they never should be mixed. The solution was just to use a packed struct with the data. I also made a little test to see what happens with the memory layout.

> Compiler folks do not dare to do that, otherwise angry users would come to them with fire and pitchforks.

Exactly! So packed (1) is packed (1), always. Thanks for reminding about compiler warnings when packed attribute is misplaced. Pragma pack doesn't seem to emit warnings. However, unlike __attribute__((packed)), it's valid in MSVC, so I can use it in code that is compiled in VS.

Also, memcpy() emits a warning. For this reason exactly.

KnarfB
Principal III

I don't get the semantics of the original code, did it ever compile? Lines 11..13 in particular? What's the purpose?

hth

KnarfB

Line 13 before fixing - not 😉 It's a bad example. The problem with it, beside it illustrates a bad idea - is (after fixed `uint16_t` to `uint32_t`) it compiles, runs and `ok` is `true`. It happens because the compiler has no reason to reorder the static members in memory. In my real world code reordering occurs. Static members are fine, but they are something completely different from regular struct members. When no optimization is used, they behave exactly as the class was a singleton instance. With optimization the difference becomes apparent.

KnarfB
Principal III

> Static members are fine, but they are something completely different from regular struct members

Yes, that's true. For a global variable (static or not), the final address is assigned by the linker, not the compiler.

What is uint8_t X:a; ? Syntax error? Did you mean X::a; ?

Don't see a use of defining three individual variables instead of one instance of X.

hth

KnarfB