Are there any tools available for debugging memory leaks?

[Starman]

I'm using an STM32F767ZI, and am programming it in C without any additional libraries (bare metal).

I believe there is a memory leak somewhere as a memory allocation that typically would work seems to fail after the program has been running for some time.

I'm familiar with tools such as dmalloc and valgrind, however an embedded environment is (obviously) very different and adds complications.

Are there any reliable tools, similar to the two named above, which could help me to track down memory leaks? Since the system is intended to run indefinitely, it would be preferable if the tool offered functions which can be called from within GDB in order to analyse the currently allocated memory.

If not, what is the best way to go about making something like this myself which can give useful info about where exactly the memory was allocated?

From searching around, I've come across a variety of (unhelpful for my use-case) answers, including suggesting that dynamically allocated memory should be entirely avoided? My system makes HTTP requests, parses them as well as the JSON body, then extracts a value at a specific key. It would be nice to avoid dynamically allocated memory if possible, so is there any way to go about it for something like this? Clearly the issue is that the HTTP response size is unknown, as well as the JSON body.

For reference, here is the link to the HTTP parser I've written: https://github.com/Starman3787/http

And this is a link to the code on the embedded system, if it helps (the HTTP parser used here is essentially the same, although slightly modified to work on an embedded system and reading the response over UART): https://github.com/Starman3787/btc-split-flap/tree/dev/software_v2

[Paul1 · Accepted solution]

Personally I've managed to avoid Alloc in most of my designs specifically because it is difficult to ensure equipment will keep working after a long unsupervised time.

Some suggestions/options:

• Issue: If you are allocing varying sizes, then you will need a way to clean up fragmented memory, or over time the tiny fragments that occur between alloc'd blocks from repeated alloc/release will multiply, wasting memory and appearing as a memory leak (growing unavailable amount of memory), *even when you don't have a memory leak*. If there is a state where all alloc'd memory is released then that can be the periodic defrag, if not then you will need to include defragging in your design, if don't want to defrag then change your style.
• Instead of allocing varying sizes, consider alloc'ing fixed size sectors (like a diskdrive). All sectors same size, and your usage of the sectors usage shouldn't require contiguous sectors. Doing this converts memory from words to fewer more manageable blocks, and the blocks be used as non-contiguous removed the fragmenting issue. i.e. have a bunch of blocks for the headers of your messages, and a separate batch of blocks that can hold non-contiguous chunks of the message bodied (i.e. Disk directory and files). You might be able to find some freeware ramdisk code. set the sector size to match your messages or other alloc'd elements. (Done this a long time ago and it worked fine, no leaks).
• Option: when working on the buffer copy all sectors to one contiguous large buffer, optionally release sectors, do your work on buffer, then store buffer back to the alloc'd sectors (or send it and release buffer).
• Simple but requires lots of memory: Always allocate messages of same size (largest message size supported in your design). If possible alter your design so the largest required message size is limited/smaller.
• Basically work with blocks instead of bytes/words. You will have to choose based on your project's requirements and your implemented style. Don't be afraid to change your style, take it as a challenge to level up.

Paul

[Tesla DeLorean]

Walk the allocation list, monitoring for​ fragmentation.

A​dd a light wrapping of malloc/free to find who is responsible for orphans or double releases, etc.

[Starman]

Thanks for the helpful response.

I didn't consider that it might be related to memory fragmentation, however this seems likely to be one of the contributing factors as from my tests, the program stopped when calling "realloc()" in order to increase the size of the allocated memory.

Defragging sounds like it would add lots of complexity, and cause more potential for bugs. I think half my problem is not free'ing up memory which needs to be free'd up (due to too much complexity already), leading to a reduced amount of memory available and therefore defragmentation preventing any further allocations. Defragging won't make it any easier to prevent memory from being lost by not being free'd (unless there is some tool which can tell me what has and hasn't been free'd, which there doesn't seem to be).

Alloc'ing in fixed size sectors is probably quite reasonable for me to do, especially since the data received over UART seems to be sent in fixed chunks of a size of up to 1460 IIRC, so it shouldn't be too hard to base the allocation sizes around that.

However, from thinking this through further, I'm wondering this:

For a simple application like this where I only actually need a single value at a certain key from the JSON, does it even make sense to store the entire parsed response, or is it just overkill? Do you think it would be a better idea to: receive the chunks and put them into blocks of a fixed size of 1460 => parse the headers and check the response status etc. => parse the JSON until the required key is found, store and return the value of that and discard everything else (by free'ing up the chunks)? Perhaps I just added a lot of unneeded complexity before? Would be good to get your thoughts on this.

Thanks again.

[Starman]

What do you mean by "walk the allocation list"? How can I do this?

[Paul1]

1. I'm from the days of 64K (RAM+ROM=64K), so I would tend to store the minimum required even when I have MB
2. Instead of alloc, have a fixed array (array of messages/structs) and grab elements from that. A member in the struct can tell if free/used, another member could be enumerated type of element. Using an array gives full control, no Alloc issues if not using Alloc.

[Starman]

64K wow, it's amazing that programs used to run with so few resources. It makes sense not to store data that will never be used, so I'll probably opt for storing as little as I can.

How big should this array be? I'll need to check how many chunks of data usually seem to be sent over the UART for each response (I hope the number is similar each time) but if not, how is a situation where there are more chunks than are available handled?

[Tesla DeLorean]

Allocators typically use a linked-list construct ahead of the location they return to you. https://en.wikipedia.org/wiki/Linked_list

You can also hide data ahead of the address passed to the application code.

[Starman]

oh i see

how can i access this linked list? i'm using arm-none-eabi-gcc to compile this, where can i find the location of this list?

[Paul1]

You have to design the system to handle the needs, or slow things down when getting near full. I expect you'll need to analyze the messages you support to determine how many to reserve for each. If you run out of space then you may be able to handle the same way you would handle a corrupt message? (i.e. discard and wait for a retry? Send a NAK/Fail?)