Does Cube MX build the libc.a printf/canf/heap/etc library?

[PHolt.1]

Yesterday's post here

https://community.st.com/s/question/0D53W00002F4MHrSAN/is-there-a-bug-in-the-newlib-heap-on-the-32f417-cube-ide-library?t=1682872893044

refers.

This article

https://nadler.com/embedded/newlibAndFreeRTOS.html

suggests that the freertosconfig.h file can contain configUSE_NEWLIB_REENTRANT =1 and this causes MX to build libc.a with the proper mutex calls.

That suggests MX has the sources. But I have Cube IDE installed (been using it for a couple of years, but have never used MX) and there are no sources there for this stuff.

I find this amazing because I have never seen ST supply any sources for these parts of stdlib. And the many supplied versions of libc.a (which can be found in c:\ST - there are 66 of these) come only in a compiled form, and not "weak" so the mutex calls, which are dummy functions, cannot be fixed by making use of the "weak" override mechanism. I had to weaken the whole libc.a and then I could do it.

[Pavel A.]

The newlib comes pre-built and installed with the toolchain. No source.

The newlib.h is available in arm-none-eabi/include/newlib.h (full version) and include/newlib-nano/newlib.h (nano version).

There you can find _RETARGETABLE_LOCKING and other parameters of these pre-built libraries.

(of course they cannot be redefined)

[PHolt.1]

Exactly - I cannot see how configUSE_NEWLIB_REENTRANT = 0 or 1 can ever be effective when it comes to providing the newlib printf and heap functions (these alone, AFAICT, need mutexing to be thread-safe) with functioning mutex features.

So I don't get this idea at all.

The rest of Cube IDE newlib does not need mutexing. The scanf family seems to be ok, for example.

It looks like some very silly person at ST

• got hold of some stdlib sources
• added in dummy mutex calls (some sources out there actually have these calls in place)
• compiled libc.a in a non weak mode (63 versions for different CPUs etc)
• didn't document it

In the meantime the internet fills up with reports of crashes (never solved) and with long threads on eevblog by myself documenting how to solve this by making it weak (with objcopy) and then replacing the printf family with a decent usable 3rd party printf.

So much time wasted.

[Pavel A.]

Well now you see why other (paid) toolchain/compiler/runtime lib vendors still survive in their business.

[Bob S]

configUSE_NEWLIB_REENTRANT does not affect newlib (obviously, as it is already compiled). In only affect how FreeRTOS and tasks/threads interact with newlib functions. When defined and != 0, it causes each FreeRTOS task to contain a copy of the newlib "reent" structure. That means each task now gets its own "errno" variable as well as file pointers (ugh) and whatever else is crammed in there.

[PHolt.1]

OK thanks. I get that.

But AIUI that works only if you configure FreeRTOS to use newlib's malloc and free, instead of its own heap (e.g. heap_4.c).

But why would you do that. You give FR a memory block and it does what it wants in there. It works fine. It runs its own private heap in there (same as LWIP and TLS do actually - each one runs its own private heap in the RAM blocks you give them. so you have three sets of malloc and three sets of free : - ) ).

You also get the benefit of knowing exactly where FR is keeping its process stacks. I put mine in the 64k CCM, and wrote a function which writes out this 64k into a file, and got somebody on freelancer.com to write me a win32 GUI prog which displays this 64k block graphically, so I can see FR's stack situation.

https://peter-ftp.co.uk/screenshots/202305010617034717.jpg

But heap (the malloc and free calls) is not the only problem with libc.a. The more immediate one is that the newlib printf family, when outputting floats and longs, uses the heap too, so the printf family is not thread safe, because a) printf is not thread-safe (because it uses the heap) and b) the heap itself is not thread safe because the mutex protection around malloc and free uses dummy mutex function calls. So the whole thing is screwed up twice over.

Why does newlib printf use the heap? This is explained here

https://nadler.com/embedded/newlibAndFreeRTOS.html

It is the Steele and White "perfect float" implementation from 1990. The source which ST used in newlib (not newlib-nano) is actually from 1990! I proved it by comparing Cube disassembly output against that 1990 source. But this isn't needed in most embedded applications. Far easier to replace the printf family and then just fix the mutex calls around malloc and free. That is what I did.

I solved all this, as posted previously, mostly on eevblog (search for "weakened"). But it took a while.

Also newlib's heap has a nasty bug which is going to give anybody who wants to get FR to use that heap, a whole load of fun

https://www.eevblog.com/forum/programming/help-needed-with-some-heap-test-code/msg4841192/#msg4841192

But maybe not every libc.a has this bug in its heap. ST supply 63 different versions of libc.a : - )

[Dave Nadler]

@PHolt.1​ wrote:

This article

https://nadler.com/embedded/newlibAndFreeRTOS.html suggests that the freertosconfig.h file can contain configUSE_NEWLIB_REENTRANT =1 and this causes MX to build libc.a with the proper mutex calls

It does no such thing; please read the article again more carefully.

It only discusses configuration of FreeRTOS.

If you're having difficulties with the unfortunate ST-provided libraries, use the public ARM-provided libraries instead, or build your own with the options you need.

Really now...

[PHolt.1]

" use the public ARM-provided libraries instead"

I basically did that. I replaced the printf library, and fixed the mutex calls.

But you could just pick up more bugs. Look at that bug in newlib malloc, above.

[Piranha]

> But AIUI that works only if you configure FreeRTOS to use newlib's malloc and free

As Bob already said, the configUSE_NEWLIB_REENTRANT enables a separate struct _reent for each thread. It's necessary for the ancient errno macro and other global state stupidities in the standard C runtime library, which should have been deprecated decades ago. Anyway it has nothing to do with the heap memory allocation.

> It runs its own private heap in there (same as LWIP...

For lwIP it's just one of the three options. It can use a custom heap, a private heap or a memory pools. The pools are preferred because of a better performance and no fragmentation. My configuration:

#define MEM_SIZE                        0
#define MEM_USE_POOLS                   1
#define MEM_USE_POOLS_TRY_BIGGER_POOL   1
#define MEMP_USE_CUSTOM_POOLS           1

> I had to weaken the whole libc.a and then I could do it.

Dave's article describes a way to do it without such a PITA.

> I basically did that.

The ARM GNU Toolchain, not ST's miracle...

> It does no such thing; please read the article again more carefully.

As always... But congratulations on finding the Dave's article, to which I gave you a link a year ago:

https://community.st.com/s/question/0D53W00001fwzz7SAA/32f417-how-to-prevent-lowleveloutput-in-ethernetifc-overwriting-the-last-dma-buffer

[PHolt.1]

I use social media to find help, and to help others where I can.

I thank those who contributed by explaining something usefully.