memory allocation advice needed

[Palitsky.Gennady]

Posted on April 26, 2011 at 17:24

memory allocation advice needed

[Andrew Neil]

Posted on May 17, 2011 at 14:33

Always

  give links when posting the same question to multiple forums:

http://www.keil.com/forum/18837/

[Palitsky.Gennady]

Posted on May 17, 2011 at 14:33

However the fact you have to have the whole 16K in a monolithic lump scares me more, are you honestly telling me you can't pull this in at 128/256/512 bytes at a time and parse it?

Probably you are right. I can try to do it in smaller chunks. The reason for my decision was that data for a slave structure are located in different parts of a main profile, and the main profile is in .json format, so there are a lot of unrelated fields in it.

[Tesla DeLorean]

Posted on May 17, 2011 at 14:33

Well there should be plenty of papers and books on the interwebs dealing with dynamic memory allocation strategies for embedded. malloc/free, mix sized pools.

However the fact you have to have the whole 16K in a monolithic lump scares me more, are you honestly telling me you can't pull this in at 128/256/512 bytes at a time and parse it?

[Palitsky.Gennady]

Posted on May 17, 2011 at 14:33

Sorry I forgot to give link to cross-post on a Keil forum

http://www.keil.com/forum/18837

And there a lot of great suggestions there.

[Tesla DeLorean]

Posted on May 17, 2011 at 14:33

http://www.json.org/

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. While I'm not personally familiar with this format, you can surely load it a line at a time. What's the longest line length? If you are condensing one data stream to another, and it's going to get smaller, the buffers can sit on top of each other, or at the very least significantly overlap. If this can be done in it's own ''application'', run that first, and then run your primary application. Neither steals the others memory. Using a generic malloc/free would also work fine in your scenario as it wouldn't cause any heap fragmentation. Also why would it be a problem for the stack to descend, below regular bounds, into the heap of an un-started application? Write a test harness, and reimplement and test your new parser on a PC, don't try to initially test/debug it on the STM32.

[Palitsky.Gennady]

Posted on May 17, 2011 at 14:33

you can surely load it a line at a time. What's the longest line length?

I can do it. But it means that any time the order of data in original format has changed, i will need to rewrite the parsing function. I can initialize the original profile structure in a flash, use pointers to calculate addresses and load lines one a a time, but it seems less effective to me, and since data in a profile have different types, I am afraid of any address calculation problems related to data alignment. If this can be done in it's own ''application'', run that first, and then run your primary application. Neither steals the others memory. This idea seems really attractive. I already have a bootloader and application running on this chip. i just have to figure out how to fit another 'initializing' application here Using a generic malloc/free would also work fine in your scenario as it wouldn't cause any heap fragmentation. I thought about this. But don't I need to allocate enough heap to accommodate these structures ? And isn't it then going to be wasted for application ? 

[Tesla DeLorean]

Posted on May 17, 2011 at 14:33

I thought about this. But don't I need to allocate enough heap to accommodate these structures ? And isn't it then going to be wasted for application ?

If your application has a lot of global or statically allocated space (uninitialized), then yes, you will clearly have a problem. You might have to revisit your application, or do some more floor planning of your actual memory usage rather than let the compiler/linker figure it out behind the curtain. Most embedded systems try to avoid dynamic allocation because of the fragmentation issue that come with using a lot of malloc/free/realloc's churning up the heap, or the need to allocate space under interrupt, and the potential for that to fail. That said, it doesn't preclude you from using a large heap, and malloc calls to define large memory regions once at startup, ie never free'd. These would behave like large statically allocated buffers, but with the added benefit that you could configure them more flexibly in the field, not on-the-fly, but rather with boot parameters, or whatever. If the parsing/condensing code malloc's and then free's all it's resources, all these will be available on the heap for the main application to use and the heap will not fragment.

[Palitsky.Gennady]

Posted on May 17, 2011 at 14:33

Clive, it seems that i am missing something.

if i am only going to use malloc to place my large buffer on a heap at a startup (actually i may need it  another time to upload to the pc), and not going to use dynamic memory allocation for anything else in application, what would be the advantage of placing this buffer on a heap against just statically allocating it ? The amount of ram used seems the same to me. And thanks a lot for clearing things up for me.

[John F.]

Posted on May 17, 2011 at 14:33

''a system profile structure ... saved in a serial dataflash''

So the data are constant? ''At power-up I need to extract from this profile another structure - slave profile'' Why at power up? Why not store the pre-extracted slave profile also in the serial data flash? ''send it to the slave microcontroller over CAN'' That's a lot of data to send over CAN. The data will have to be ''chopped up'' to fit into CAN messages. Are you sure this is the best approach?