Unit Testing Support with VS Code?

[Suess]

Is there and/or will there be an approved off-target unit testing framework for STM32 (in my case, STM32H5)? I've been trying to stand up a test project generated by STM32CubeMX, imported into VS Code using Unity unit-testing framework, but running into pitfalls. In the past, I've used PlatformIO's with ease, was hoping the STM32 VSCode ext pack would be similar out the gate.

Over all, I'm aiming to test against the business logic and not needing a full-blown simulator. My rule of thumb is its good practice to do some kind of off-site (off-board) unit testing.

Even if there is a solid blog post with how to setup STM32 + CMake + Unity (or equivalent) and vanilla C using the CubeMX generated project, that would be most helpful. Most of the blog posts I find are either outdated, doing something custom, or a different architecture.  Yes, I'll fully admit, my strong points are MSVC and C#'s tooling for test frameworks, and not so much with CMake/CTest.

Thank you!

[Suess]

As an honorable mention, I've found the top 3 contenders for unit testing frameworks using C on off-target tests. However, it would be helpful to see a good clean example of how to implement with VS Code from a generated MX project

• Unity unit-testing framework - C - Specialized for embedded (supported by Platform.IO)
• CppUTest - C/C++ - Specialized for embedded and C++
• GoogleTest - C/C++ mostly favored for C++

[Suess]

Has anyone else been successful in creating Unit Tests using VSCode + CMake with STM32's extension set?

I was able to create a sample CMake + Unity example, and i ported over the basics to my STM32 C project (which builds). Its generating the Windows .EXE, however, the CTest is throwing a generic error, as if it can't find it.

Oddly enough, when I execute the "\build\Unit-Tests\Tests\Tests\suite_1_app.exe" it does execute and tells me that the tests pass

C:/xxxx/MyProject/Tests/SampleTest.c:19:test_fubar:PASS

-----------------------
1 Tests 0 Failures 0 Ignored
OK

VS Code's Build Output:

Below is a sample of the successful build, but fails to execute.

...
[main] Building folder: C:/xxxx/src/MyProject/build/Unit-Tests/Tests 
[build] Starting build
[driver] NOTE: You are building with preset Unit-Tests!, but there are some overrides being applied from your VS Code settings.
[proc] Executing command: cube-cmake --build C:/xxxx/src/MyProject/build/Unit-Tests/Tests --
[build] [1/6] Building C object CMakeFiles/source_code.dir/Core/Src/app/StarbusBsp.c.obj
[build] [2/6] Building C object Tests/CMakeFiles/suite_1_app.dir/SampleTest.c.obj
[build] [3/6] Building C object Unity/CMakeFiles/UnityLib.dir/unity.c.obj
[build] [4/6] Linking C static library libsource_code.a
[build] [5/6] Linking C static library Unity\libUnityLib.a
[build] [6/6] Linking C executable Tests\suite_1_app.exe
[driver] Build completed: 00:00:01.038
[build] Build finished with exit code 0
[proc] The command: ctest --show-only=json-v1 -T test --output-on-failure --output-on-failure --no-tests=error failed with error: Error: spawn ctest ENOENT
[ctest] There was an error running ctest to determine available test executables

[Suess · Accepted solution]

Got it!!  99% there. I'll update my GitHub samples repository for easier replication in the next few days:

• https://github.com/DamianSuess/Learn.STM32

Here's rundown on how to add unit tests using VSCode + STM32 + Unity Unit Testing Framework on Windows OS; as our products use vanilla C. Remember, this tests business logic only! Not your board-specific code as STMicro does not provide an emulator like Microchip or Espressif at this time.

Overview

The following is using a "Common" folder for Unity as we have projects for both the Bootloader and Application (OpCode) in the same repository.

The secret sauce is in downloading a proper platform-specific compiler with CMake! You can't rely on CMake that is bundled with STM32 extension pack, as VS Code won't discover the tests. Frankly, I don't like duplicating tooling on my machines.. plays hell with your PATH definitions.

• Off-Site Unit Testing (platform specific as it runs on your OS)
• Platform specific compiler (to make EXE not ELF)

Known Issues:

• "Unity" shows up in the "Testing" list even though there are no unit tests there.
• Sometimes the compiler builds platform specific code into Unit Test app and builds fail (even though it's not referenced ANYWHERE!)
• Build error messages are VERY POOR! It just says, failed, with little trace as to what/where. (I've been spoiled by other unit testing system.)

Tools Needed:

• VS Code + STM32 Extension
• Unity Unit Testing Framework
• MinGW-w64 (via MSys2)
  • pacman -S --needed base-devel mingw-w64-ucrt-x86_64-toolchain
  • pacman -S --needed mingw-w64-ucrt-x86_64-ninja
  • pacman -S --needed mingw-w64-ucrt-x86_64-cmake
• Set Environment Variable to MinGW's "xxx\bin\" folder

Folder Structure:

source/
source/Bootloader/
source/App/
--[ App.code-workspace
--[ CMakeLists.txt
--[ CMakePresets.json
source/App/cmake/
--[ gcc-x86_64-windows.cmake
source/App/Tests/
--[ CMakeLists.txt
--[ SampleTests.c
--[ SampleTests.h
source/Common/Unity/
--[ CMakeLists.txt
--[ unity_internals.h
--[ unity.c
--[ unity.h

App/App.code-workspace:

{
  "folders": [
    {
      "path": "."
    },
    {
      "path": "../Common/Unity"
    }
  ],
...

App/CMakePresets.json:

{
  "version": 3,
  "configurePresets": [
    {
      "name": "Unit-Tests",
      "generator": "Ninja",
      "binaryDir": "${sourceDir}/build/UnitTests",
      "toolchainFile": "${sourceDir}/cmake/gcc-x86_64-windows.cmake",
      "cacheVariables": {
        "CMAKE_BUILD_TYPE": "Debug",
        "BUILD_TESTS": "ON",
        "CMAKE_C_COMPILER": "gcc",
        "CMAKE_CXX_COMPILER": "g++"
      }
    }
...

App/CMakeLists.txt:

cmake_minimum_required(VERSION 3.22)
set(CMAKE_C_STANDARD 11)
set(CMAKE_C_STANDARD_REQUIRED ON)
set(CMAKE_C_EXTENSIONS ON)
set(CMAKE_PROJECT_NAME MyProject-App)
set(CMAKE_EXPORT_COMPILE_COMMANDS TRUE)
project(${CMAKE_PROJECT_NAME})
enable_language(C ASM)

if(NOT BUILD_TESTS)
  # Normal on-Target build configurations
elseif(BUILD_TESTS)

  message("---------------------------------->")
  message("-[ Building with CTest version ${CMAKE_VERSION}")
  message("---------------------------------->")

  add_library(source_code STATIC
    "Core/Src/app/SomeFile.c"
    # Another biz-logic file to test
  )

  # Add include paths for the source code library
  target_include_directories(source_code PUBLIC
    "Core/Inc"
  )

  include(CTest)

  # Add Libraries: add_subdirectory(SourceDir BinaryDir)
  add_subdirectory("../Common/Unity" "${CMAKE_CURRENT_BINARY_DIR}/Unity")

  # Add Tests
  add_subdirectory("Tests")
else()
  message(FATAL_ERROR "Invalid build configuration. Please set BUILD_TESTS to ON or OFF.")
endif()

App/cmake/gcc-x86_64-windows.cmake:

set(CMAKE_C_COMPILER_ID GNU)
set(TOOLCHAIN_PREFIX "x86_64-w64-mingw32-")
set(CMAKE_C_COMPILER                ${TOOLCHAIN_PREFIX}gcc.exe)
set(CMAKE_ASM_COMPILER              ${CMAKE_C_COMPILER})
set(CMAKE_CXX_COMPILER              ${TOOLCHAIN_PREFIX}g++.exe)
set(CMAKE_LINKER                    ${TOOLCHAIN_PREFIX}g++.exe)
set(CMAKE_OBJCOPY                   ${TOOLCHAIN_PREFIX}objcopy.exe)
set(CMAKE_SIZE                      ${TOOLCHAIN_PREFIX}size.exe)
set(CMAKE_EXECUTABLE_SUFFIX_ASM     ".exe")
set(CMAKE_EXECUTABLE_SUFFIX_C       ".exe")
set(CMAKE_EXECUTABLE_SUFFIX_CXX     ".exe")

Common/Unity/CMakeLists.txt:

# CMakeLists for Unity library
# Create a static library from the Unity source files
add_library(UnityLib STATIC unity.c)
target_include_directories(UnityLib PUBLIC ${CMAKE_CURRENT_LIST_DIR})

App/Tests/CMakeLists.txt:

# Register "Test Suite A" .c files with CMake as test executables
add_executable(suite_1_app
  "SampleTest.c"
)

# souce_code  - Defined in root or business logic's folder "CMakeLists.txt"
# UnityLib    - Defined in "Common/Unity/CMakeLists.txt"
target_link_libraries(suite_1_app
  source_code
  UnityLib
)

# Name of Test Group: test_suite1_name
# Files to test:      suite_1_app
add_test(TestSuite1 suite_1_app)

App/Tests/SampleTest.c:

#include "SampleTest.h"
#include "../../Common/Unity/unity.h"

void test_fubar(void);
int add(int a, int b);

int main(void)
{
  UNITY_BEGIN();
  RUN_TEST(test_fubar);
  return UNITY_END();
}

void setUp(void)
{
}

void tearDown(void)
{
}

void test_fubar(void)
{
  int result = add(2, 3);
  TEST_ASSERT_EQUAL(5, result);
}

int add(int a, int b)
{
  return a + b;
}

App/Tests/SampleTest.h:

#ifndef _SAMPLETEST_H_
#define _SAMPLETEST_H_

/** @brief Include necessary headers here. */
void Fubar();

#endif /* _SAMPLETEST_H_ */

[Suess]

If this methodology is wrong, please, I'm all ears for doing it a more STM32 way.

As any platform goes, I would rather see a formally supported framework from a manufacturer. I'm all for improvement suggestions.