HAL Library seems to not follow classic programming guidelines

[Werner Dähn]

Posted on June 09, 2018 at 15:42

I understand that programming enterprise software on a 8 core CPU with 4GHz and a simple STM32 MCU is different, nevertheless I would have added a few things to the HAL layer from the beginning:

1. All HAL Init calls check if there is a collision with other resources. I configure PA1 as GPIO, then I use UART1 which needs PA1, hence the UART config should return an error. Granted, CubeMX is supposed to help you with that in a graphical manner but it does not in case you assign pins dynamically. And even if, what's wrong with validating that in a debug compile firmware a second time? Wouldn't that help beginners significantly?
2. Programming a MCU is very hardware related. You need to open that gate to provide power to... by setting a register, switch the line to alternate pin by another register etc. From a HAL library I would expect to abstract exactly that. I want to enable UART1 on pin PA6, therefore the__HAL_RCC_GPIOA_CLK_ENABLE is called implicitly by the HAL_GPIO_Init function, the remap is called and the corresponding AF clock enabled and.... Things like that. Why should I remember every dependency if the HAL can do that for me?Another example: How many lines of code do you need to get an quadruple encoder (up/down counter) to work? Logically one call saying which pins to use and maybe choose the timer number. The counter specific X2 falling/X2 rising/X4 setting is needed also. Everything else can be derived from that information. Today you need 20-30 lines of code and if just one is wrong, you will not find out easily.
3. The implementation is often too basic. Most obvious in the UART part of HAL. You can do char polling, add a interrupt callback or DMA. Fine. But useless as in most cases you would use a ring buffer. No rocket science but since when is that requested and yet still not available out of the box? Another missing receiver method is when timing plays a role like with serial packets. Every 20ms I get a packet that starts with 0x01 and ends with 0xEF and lasts for up to 5ms.

Wouldn't such things help greatly to get started? And to debug? And make the user experience better? And lower frustrations? And cause proper error messages rather than simply not working?

Or am I missing something.

Note: this post was migrated and contained many threaded conversations, some content may be missing.

[turboscrew]

Posted on June 14, 2018 at 19:46

The thing is: if the HAL modules are all self contained, they have to duplicate code. If they are built on top of each other (like now), then, to use one module, you need to drag in half the library due to the dependencies. And in consumer products, bigger MCUs  are not that cheap.

Also, STM32 is not the only architecture used...

And: someone needs to write the libraries, like HAL. The famous Someone Else is getting too busy these days... ;)

[turboscrew]

Posted on June 14, 2018 at 19:51

Funny. I never had to do anything with Multibus. But I did become familiar with Unibus and VME.

[Jack Peacock_2]

Posted on June 14, 2018 at 20:08

Even 40+ years ago there was a trend to C like system languages on larger machines.  A good example was BLISS on PDP-10/20, close to the C compiler used in embedded now.  Intel had PL/M, although it never caught on outside a few customers who could afford the big iron to run it.

BLISS had all the niceties we take for granted, especially the structures with bit fields mapped to peripheral registers.  Register knowledge was always the programmer's responsibility, not the compiler.  The compiler did work well, but in the end core memory was just too expensive; DEC delivered frameworks in assembler (not so fond memories of maintaining TRAFFIC-20 screen manager services on a TOPS-20 system in the 70s).  It wasn't until decades later that DEC fully committed to an HLL for the operating system, when it had to move a mix of BLISS and assembler from 32-bit VAX to 64-bit Alpha.  Same applies now, the economics are good for some ARM Cortex parts, especially A series with large DDR memory, but frameworks on M series are still held back by flash cost.

The underlying problems with C and frameworks are the same now as 40 years ago...on chip resources.  The extra 64K the HAL requires translates into significant cost increment in the BOM for mass produced products.  One of those economic facts of life left out of programming classes is the brutal drive to slash costs on the assembly line.  Sure there's the qualitative argument about time to market, maintainability, and fewer bugs using frameworks like HAL, but the reality is the 75 cents per unit for the extra memory is a deal breaker when production runs are in the 100K range.  That's why I'm sitting in front of a screen today fixing PIC16 assembly code from 20 years ago, while hoping I can eventually push through a redesign to use an M0+ and GCC.

ST needs the HAL to compete with other vendors delivering the same kind of framework, primarily to bring in novice embedded programmers.  If prior programming experience is CSS, Javascript and JSON I imagine that first exposure to embedded would be a nightmare of complexity.  The HAL does mitigate the shock, to some extent, but it brings to mind the old argument of the perils of teaching programming 101 using BASIC instead of a structured language.  Great results, at first, but a couple of years later all the dead end practices come back to haunt.

(And yes, my CS 101 class used Dartmouth BASIC on a UNIVAC, but a year later I discovered ALGOL.  That's the ancestor of all structured languages, including C, for those who never heard of it.)

  Jack Peacock

[Alan Chambers]

Posted on June 15, 2018 at 00:18

It seems to me that SPL (and maybe HAL) attempts to create general purpose APIs for specific peripherals. These do not represent any particular use case of the hardware among the sea of possibilities, but provide slightly more programmer friendly access to the registers than CMSIS. This is why, when you want to set up a UART (say), you have to trawl through all the dependencies and setup pins in GPIO, clocks in RCC, the UART itself, and so on. It's all a bit of a pain. I think this level of abstraction is a mistake. It's too low.

I've said this before, but suggest a better level of abstraction is to create single purpose APIs each representing a particular use case of the hardware. These 'drivers' (for want of a name) would encapsulate all the dependencies between RCC, UART (or whatever), GPIO, NVIC and all the rest. I imagine a library of many small drivers which each perform some specific function and are agnostic about the particular peripherals they use to achieve this. My canonical example is a PWM output. You would in theory just need to tell it what pin to use, and it would work out everything else.

Vaguely inspired by mbed and Arduino, I did once go to the effort of capturing all the hardware dependencies and other metadata necessary to do just this using lookup tables. I then implemented a PWM output which worked exactly as just described, and a debounced EXTI input, a simple SPI driver, and a number of other drivers. The library worked quite well, and I even demonstrated it to ST, but there were issues: the run time code includes a lot of lookup tables which it only needs during intialisation; detecting and resolving hardware conflicts was problematic; and some other things I can't remember. Every pin had a lookup table to tie alternate function indices to functions on specific peripherals...

This is where Cube shines. It knows all the hardware dependencies at code generation time - no need for run time support - and it makes sure you have all the correct flags and whatnot set up, and that you avoid conflicts. But there are issues. The code Cube generates is garbage, but the main one is that the level of abstraction in HAL appears to have the same problem as SPL - it's too low. You basically use Cube to separately configure general purpose peripherals rather than select single purpose use cases involving potentially several peripherals... It does help with pins, interrupts and DMA streams, but it's still really all about the trees and not about the forest. Can you even choose a frequency for a timer yet?

So I had a go at re-implementing Cube. Sort of. The idea was to have a GUI in which you select from a list of available single purpose drivers (like a PWM output). This is at a higher level of abstraction than individual peripherals. Each driver is configured in the GUI. For the PWM driver, you choose from a list of available timers (conflicts are highlighted). Having chosen the timer, you choose from a list of the available pins (conflicts...). And that's pretty much it. At run time you can set the frequency and duty, and turn it on and off.

You build up the lowest level of your application by selecting and configuring these drivers. When you generate the code, it creates instances of your selected drivers with all the initialisation data they need to enable RCC clocks and all the rest. Presto! Now you have an application-facing API you can use directly. All the hardware shenanigans is complete, and you can worry about your application logic. Don't be distracted by the PWM example: these 'drivers' could be anything: simple inputs to complex middleware. There is no reason why drivers cannot be efficient lightweight implementations - the important feature is the abstraction level at which they are selected and configured.

I think I got enough of this Cube-a-like together for a plausible demonstration. The idea does seem to have some merit. But it needs some love. I thought it would be better if each driver (including its configuration GUI, base code and code generator) was implemented as a plugin for the main GUI. This makes the whole thing more modular, and allows it to be opened up to third party and open source development. I imagined whole libraries of really useful simple plugin drivers: don't like ST's PWM output driver? Use Fred's. Or mine. Or write your own (and maybe share it). All the drivers might be implemented in terms of something low level like CMSIS. They are preferably readable so they can be used as models or templates for new drivers.

By far the hardest part of this demonstrator was encoding all the metadata for whole families of processors. I used the subset of STM32F4s covered in RM0090 to create a SQLite database to support the GUI, and that was tedious enough. [I guess I didn't really need all the registers, fields and whatnot for this particular application.] Once you have a reasonably comprehensive and accurate database, the basic GUI, and a clearly defined API for plugins, this all becomes pretty straightforward. I reckon to really make this fly, ST are much better placed to create and share that database. The existing SVDs don't really cut it, and mining Cube for data was only partially successful.

Anyway, sorry that was so long. Been trying to find a 'better way' for years.

Al

[eBirdman]

Posted on June 14, 2018 at 22:35

Indeed making ideal hardware abstraction library is a high art. All requirements are mutually exclusive. If chip maker does not make all hardware modules multiplexed and reused, then all their functionality they offer to us today will not fit on the chip as large as a PCB with hundreds of pins...... if they do multiplex, then drivers become complex and those drivers become tightly dependent on each other... what can a tool developer do? Either create a large bunch of files (all dependent on each other because of the reused resources on the chip) or create huge smaller number of files with bunch of macros and 'if' definitions, etc.... in any case we, the developers, will complain....  so indeed it is hard and complex art...

But saying that, still, not all the MCU' makes and MCU supporting software tools are made equal Some chip makes did not even start the HAL-like tools support for their chips - they are on the loosing path for sure.....

[eBirdman]

Posted on June 14, 2018 at 23:14

Jack, you are also a high tech pterodactyl    I burst in laughter from your ' I discovered ALGOL.  ...... for those who never heard of it ' ... after my graduation from college we all thought this is a Super-Drooper language which will stay forever   Our first cross compiler was  from Algol and another from PL/M (PDP? DEC?)... Algol did not survive for embedded coding but PL/M cross compiler running on the PDP clone was used on many projects from 70s through early 80s ... but I was mainly involved in low level design from schematic though assembler language drivers......

I never heard of BLISS though.... don't know why but it feels so good to know that there are other old timers still here, who are doing development as decades ago ....creativity  is what makes meaning in our lives... I truly want all of you/us to engage in this electronic game for many decades forward not just for money making but for the fun of it..............................

[eBirdman]

Posted on June 15, 2018 at 02:05

Alan, this is EXACTLY how software development tools will be made in the future by the chip manufacturers (for they know their chip better)  to hide/encapsulate their hardware underwear under the HAL API which will be oriented towards the use case of each module - just as you have said. Add graphic tools similar to CubeMX or MCUXpresso or whatever, which will configure APIs for a particular functionality (for ex. complimentary PWM, singular PWM, differential ADC, singular ADC, then choose port, etc...).

Downside of your approach is that there will be large number of  particular APIs for many use cases but this is a blessing for a developer - he just has to find exactly what he needs without hacking any low level configurations but only his application related configuration (for ex. speed of UART, parity...).

You may start on your own - on this path you cannot fail - your HAL CONFIGURATOR will be the best in the industry (but don't forget graphics or partner with somebody who does graphics only) . I am not joking - may be this is your calling ?

[henry.dick]

Posted on June 15, 2018 at 03:51

'

a better level of abstraction is to create 

single purpose APIs'

such highly stylized APIs work well for simple tasks performed by simple peripherals. aka arduino like.

for power peripherals, you have to write lots of such APIs which defeats the purpose of having them in the first place; or to force the user to manually edit such APIs.

Not fun, either way.

' The idea was to have a GUI in which you select from a list of available single purpose drivers (like a PWM output).'

a better approach is taken up by Cypress' PSOC team. Its implementation is to treat each module like a hardware diagram, wired up by the user, to perform a desired task. Each 'wiring' action is then translated into automagically generated code.

Hands down, the most beautiful and seamless implementation I have ever seen, even better than PE.

That team is likely the most valuable asset of Cypress. I so hope a large chip vendor buy up Cypress so to broaden that approach onto more chips.

[turboscrew]

Posted on June 15, 2018 at 07:10

There is STM32duino... At least for some MCUs...

http://wiki.stm32duino.com/index.php?title=Main_Page

 

There's also the problem that one looses touch with the lower levels. The libraries are not that handy if you need to work with MCU that has, say, 16kB of flash an 1 kB of RAM.

[AvaTar]

Posted on June 15, 2018 at 08:41

That team is likely the most valuable asset of Cypress. I so hope a large chip vendor buy up Cypress so to broaden that approach onto more chips.

Last times, it had been the other way around - Cypress was buying up other vendors.

But I think they don't 'spread as thin' as ST, and don't churn out new Cortex M silicons as fast.

IMHO the idiom 'death by a thousand cuts' is somehow appropriate for CubeMX.