newlib/malloc locking mechanism to be thread-safe

[mattias norlander]

Several users have over the years rightfully complained about newlib/malloc not being thread-safe. This problem does not present itself until the code is executed on target and is quite difficult to debug.

Below we will show a solution for this, but first let's elaborate on the problem. To describe the problems, lets consider two common Use Cases (UCs):

UC1: Bare-metal or RTOS application with malloc in interrupts

Even though quite bad coding practice, nothing forbids the user from performing dynamic memory allocation from inside an interrupt service routine (ISR).

Imagine that the application is executing FunctionA in (non-ISR-context) application code.

FunctionA makes a call to malloc to allocate some memory and while this happens, the CPU receives an interrupt request and thereby branches to associated ISR. Inside the ISR, malloc is again called to allocate some memory.

This will lead to memory corruption since the first call to malloc performed in FunctionA, is quite likely to be partly or totally overwritten by the second call to malloc from the ISR. FunctionA or some other function sharing the intended global data will likely crash at a later point during application execution.

UC2: RTOS application with malloc in several threads

In this use case an application is running FreeRTOS (or another RTOS). There are two or more threads which both contains calls to malloc or any other function which implicitly uses malloc.

ThreadA makes a call to malloc. The scheduler at this point switches context to ThreadB which is allowed to execute and make a second call to malloc.

As described in UC1 the same problem will now arise. The first malloc call will be corrupted by the second and as a result the application is likely to crash or behave in an unexpected way at a later point in time, making the root-cause problem all the more difficult to identify.

Summary of problems:

newlib/malloc is not thread-safe. Using newlib/malloc in ISRs or in RTOS threads is not safe by default. The user may be further confused by that malloc is also called by some other newlib functions such as strtok.

To support UC1 in the bare-metal case and to support UC2 in the RTOS case, we provide an implementation of locks for newlib/malloc.

CAUTION / DISCLAIMER:

It is very important to state that this implementation of locks for newlib/malloc will temporarily disable interrupts and thereby effect the real-time behavior of your application!

Please carefully consider the implications of disabling interrupts in your application before adopting the lock mechanisms.

The new approach is safe-by-default in terms of re-entrance from ISR and using RTOS threads, at the cost of real-time penalty.

How to use the new implementation of locks for newlib/malloc:

1. Download the zip-file linked with this post.

2. Extract and drop/replace the sysmem.c in a source folder of your project.

3. sysmem.c has a dependency: #include “cmsis_compiler.h�?. This file is typically generated as part of a STM32CubeMX generated projects. It is however not part of the generated code in an Empty project. The CMSIS files can be copied from the MCU corresponding STM32Cube firmware package. Typically located in the User folder on your OS:

           STM32Cube\Repository\STM32Cube_FW_X_Y\Drivers\CMSIS\Core\Include

4. By default the symbol __NEWLIB_SAFE_FROM_INTERRUPT__ is defined in sysmem.c which will disable interrupts while executing malloc. Consider if this is desirable in your application. If not desirable, comment out the symbol.

           If __NEWLIB_SAFE_FROM_INTERRUPT__ is defined, all interrupts are disabled, even in an RTOS use case, when malloc is called.

           If __NEWLIB_SAFE_FROM_INTERRUPT__ is undefined, application will hang if malloc is called from an interrupt and RTOS locks will be used if they are defined (see 6).

5. main.h

           a) If no main.h exists, extract and drop it in some Inc folder of your project.

           b) If main.h exists in your project then simply copy the necessary defines from the downloaded main.h into your main.h

6. If your application uses an RTOS you will need to define RTOS_SUSPEND_ALL_TASKS and RTOS_RESUME_ALL_TASKS in main.h to gain task safe malloc

           For FreeRTOS define them as follows:

                       #define RTOS_SUSPEND_ALL_TASKS vTaskSuspendAll

                       #define RTOS_RESUME_ALL_TASKS xTaskResumeAll

Please download the new sysmem.c attached to this post.

This new sysmem.c is, at this point, only distributed here on the forum. This sysmem.c version has not yet been made part of the code generated by STM32CubeIDE, STM32CubeMX nor is it bundled in the CubeFW pacakages.

A shout-out to Dave Nadler for his vigorous work on helping the community to understand and overcome the problem with newlib/malloc not being thread-safe.

Feel free to submit questions and feedback in this forum thread!

[wolfgang wayalnd]

I just use it for debugging, on a stm32f769i_discovery board. I haven't

tried it on any of my other stm32 boards of which I have many.

[Piranha]

Of course ST couldn't do it without breaking and messing up something...

https://community.st.com/t5/stm32cubemx-mcu/cubemx-generated-threadsafe-stm32-lock-h-potential-bug/td-p/77220