When is going malloc() / dynamic allocation to be fixed?

[DavidAlfa]

Edit: It seems there's a bug that will randomly delete sysmem.c and syscalls.c.

If it happens, cubeMX won't regenerate them.

Also no warnings at compile time, so you won't suspect anything was deleted.

So the solution is to make a new empty project and copy these files back.

___________________________________________________________________________________ 

I don't know if it's a bug, wrong initialization, or lacking code.

I remember once in 2019 where it worked, I could run malloc(X) and the pointer would be null ​if wanting too much.

But since then, none of the Cube IDE versions have worked correctly.

I can run a 100KB malloc on a ​16KB stm32, yet it will return a valid pointer.

Of course a hard fault will happen when you try to access the memory.

I had to modify​ every project to use static allocation, which is sometimes a problem because I could save a lot of RAM when certain parts of the code is not being used.​

The minimum heap allocation​ just reserves the heap at compile time, doesn't fix anything.

Also, free() doesn't seem to work correctly.​

I can malloc 64KB on a 128KB device and use it.

But if I malloc 1KB and 16KB, then free both, I can't malloc 64KB (it does, but outside the memory limits).

I don't understand why, as these pointers where freed up,​ so memory fragmentation shouldn't be happening?​

I'm spending​ more time in looking over these bugs than on my own ones...

Don't tell me​ that I have to increase heap. Because I could put 16TB of heap, and still get hard faults when it goes too far.

Why doesn't malloc check the remaining space?

[Tesla DeLorean]

Hard to say but the _sbrk() implementation has been a train-wreck for a long while.

Thing is with these tool chains, you get what you paid for, and them some..

[DavidAlfa]

Tell that to Microchip or TI.

MplabX was a hell of an IDE when it first came out back in 2009 or so, but became a solid IDE.

And TI support is amazing.

If you are selling millions and getting a big chunk of the market​ share, how the hell can you let the software getting so bad? That is your main target.

Stm32 are powerful and cheap MCUs, if the software is also good, developers​ will feel comfortable and will develop more with them, then just sit and fill your pockets!

With 100+MHz​, 2MB flash, 512KB RAM, you should be very comfortable.

This beast can run anything!

But my actual feel when starting a project is "Will I get it to run? Or some weird bug will appear and trash all my work?".​

[Pavel A.]

ST sells chips. They don't want to compete with software partners in their ecosystem (Keil, IAR and others).

[DavidAlfa]

Very reasonable to buy​ Atollic and make a new half-crippled ide, announcing new MCUs but not fixing such important issues.

Keil and IAR have terrible interfaces, also Keil is a pita to set up. When you have to add a project and discover you can't just import a folder, but you must add every file manually!

[DavidAlfa]

So, I returned to IDE v1.4.0.

Not only that malloc works perfectly, but also the speed increased dramatically! I can ensure it's not placebo effect.

I cleanly installed 1.6.1, CubeMX wizard was 5x slower.

The MCU list being slowly drawed line by line like in an old DOS computer, took 2-3 seconds.

Deleted everything, cleanly installed V1.4.0, MCU listing drawed on 1/2 second.

Still slow as hell, considering I'm running it from a computer with SSD, 16GB RAM, i7 3770K at 4.4GHz.

I assigned more ram in stm32cubeide.ini, but didn't make any difference at all:

-Xms1024m

-Xmx8192m

Not everything is ST's fault, Java is usually slower and resource-eater, also eclipse has it's own issues. 

But clearly CubeIDE is getting worse on every new release.

Instead enhacing the IDE, you guys seem to be getting it worse intentionally. Almost 1 year with broken malloc, and just ignoring it.

Not to mention the dropping of Error_handler(__FILE__, __LINE__) 3 years ago.

Thankfully the users came for help, I've been using this macro for a while:

https://community.st.com/s/question/0D50X00009XkffVSAR/stm32cubemx-v421-errorhandler-definition-issues-in-mainh

It seems we must keep a collection of dirty hacks to fix everything that gets broken on every new release.

I don't see any sense on that.

This works now:

// MCU: STM32F411CEU6 (128KB RAM)
// Test allocating 8-256KB, increasing by 8KB on every itineration
  char* data;
  for (uint16_t t=8;t<257;t+=8){
	  data = malloc(t*1024);
	  if(!data){             // In 1.4.0, it correctly returns 0 when trying to allocate 128KB
		  asm("nop");    // But on later versions it will return valid pointer and cause a Hard fault
	  }
	  else{
		  free(data);
		  data = NULL;
	  }
  }

[DavidAlfa]

In v1.4.0, _sbrk is declared in sysmem.c. 

But could't find it in v1.6.1, neither is sysmem.c created. Where is it?

[PMath.4]

"Where is it?"

syscalls.c

[DavidAlfa]

Anyways, checking the disassembly, the difference is huge!

Clearly, the latter does absolutely nothing about checking limits.

Cube IDE v1.4.0

_sbrk:
08000a20:   push    {r7, lr}
08000a22:   sub     sp, #24
08000a24:   add     r7, sp, #0
08000a26:   str     r0, [r7, #4]
59          const uint32_t stack_limit = (uint32_t)&_estack - (uint32_t)&_Min_Stack_Size;
08000a28:   ldr     r2, [pc, #80]   ; (0x8000a7c <_sbrk+92>)
08000a2a:   ldr     r3, [pc, #84]   ; (0x8000a80 <_sbrk+96>)
08000a2c:   subs    r3, r2, r3
08000a2e:   str     r3, [r7, #20]
60          const uint8_t *max_heap = (uint8_t *)stack_limit;
08000a30:   ldr     r3, [r7, #20]
08000a32:   str     r3, [r7, #16]
64          if (NULL == __sbrk_heap_end)
08000a34:   ldr     r3, [pc, #76]   ; (0x8000a84 <_sbrk+100>)
08000a36:   ldr     r3, [r3, #0]
08000a38:   cmp     r3, #0
08000a3a:   bne.n   0x8000a42 <_sbrk+34>
66            __sbrk_heap_end = &_end;
08000a3c:   ldr     r3, [pc, #68]   ; (0x8000a84 <_sbrk+100>)
08000a3e:   ldr     r2, [pc, #72]   ; (0x8000a88 <_sbrk+104>)
08000a40:   str     r2, [r3, #0]
70          if (__sbrk_heap_end + incr > max_heap)
08000a42:   ldr     r3, [pc, #64]   ; (0x8000a84 <_sbrk+100>)
08000a44:   ldr     r2, [r3, #0]
08000a46:   ldr     r3, [r7, #4]
08000a48:   add     r3, r2
08000a4a:   ldr     r2, [r7, #16]
08000a4c:   cmp     r2, r3
08000a4e:   bcs.n   0x8000a60 <_sbrk+64>
72            errno = ENOMEM;
08000a50:   bl      0x8008b0c <__errno>
08000a54:   mov     r3, r0
08000a56:   movs    r2, #12
08000a58:   str     r2, [r3, #0]
73            return (void *)-1;
08000a5a:   mov.w   r3, #4294967295
08000a5e:   b.n     0x8000a74 <_sbrk+84>
76          prev_heap_end = __sbrk_heap_end;
08000a60:   ldr     r3, [pc, #32]   ; (0x8000a84 <_sbrk+100>)
08000a62:   ldr     r3, [r3, #0]
08000a64:   str     r3, [r7, #12]
77          __sbrk_heap_end += incr;
08000a66:   ldr     r3, [pc, #28]   ; (0x8000a84 <_sbrk+100>)
08000a68:   ldr     r2, [r3, #0]
08000a6a:   ldr     r3, [r7, #4]
08000a6c:   add     r3, r2
08000a6e:   ldr     r2, [pc, #20]   ; (0x8000a84 <_sbrk+100>)
08000a70:   str     r3, [r2, #0]
79          return (void *)prev_heap_end;
08000a72:   ldr     r3, [r7, #12]
80        }
08000a74:   mov     r0, r3
08000a76:   adds    r7, #24
08000a78:   mov     sp, r7
08000a7a:   pop     {r7, pc}
08000a7c:   movs    r0, r0
08000a7e:   movs    r0, #2
08000a80:   lsls    r0, r0, #16
08000a82:   movs    r0, r0
08000a84:   lsls    r4, r3, #5
08000a86:   movs    r0, #0
08000a88:   lsrs    r0, r6, #32
08000a8a:   movs    r0, #0

Cube IDE v1.6.1  

_sbrk:
0801e64c:   ldr     r2, [pc, #16]   ; (0x801e660 <_sbrk+20>)
0801e64e:   ldr     r1, [pc, #20]   ; (0x801e664 <_sbrk+24>)
0801e650:   ldr     r3, [r2, #0]
0801e652:   cmp     r3, #0
0801e654:   it      eq
0801e656:   moveq   r3, r1
0801e658:   add     r0, r3
0801e65a:   str     r0, [r2, #0]
0801e65c:   mov     r0, r3
0801e65e:   bx      lr
0801e660:   ldrb    r4, [r3, #4]
0801e662:   movs    r0, #0
0801e664:   ldrb    r0, [r4, #4]
0801e666:   movs    r0, #0

[DavidAlfa]

It seems there's a bug somewhere.

When creating a new project, sysmem.c and syscalls.c are generated correctly.

However, if these files are deleted, cubeMX doesn't regenerate them.

At some point, sysmem.c and syscalls.c dissapeared. I'm still trying to find the culprit.

A moment ago it happened after reopening cubeMX and enabling FastFS, but I couldn't reproduce the problem again. The gremlins...

With them, malloc works again in v1.6.1.. 

But the thing is that without these files, it compiles ok, without any warning.