OP-TEE panic in 5.0 ecosystem on CubeMx custom board

[LinuxAddict]

As the title says, I'm working with ecosystem 5 on a custom board with bootchain DTB files generated by CubeMX and modified according to lab material for the STM32MP1 workshop presentation "Lab-MP15_FromPartNumber_To_BasicDeviceTree_with_CubeMx.pdf". My high level goal is to build a bootchain capable of programming an eMMC via USB DFU.

When I load the TF-A and then FIP that I've built via USB-DFU I get an OP-TEE panic:

NOTICE: CPU: STM32MP153CAC Rev.Z
NOTICE: Model: STMicroelectronics custom STM32CubeMX board - openstlinux-6.1-yocto-mickledore-mp1-v23.06.21
INFO: PMIC version = 0x21
INFO: Reset reason (0x15):
INFO: Power-on Reset (rst_por)
INFO: FCONF: Reading TB_FW firmware configuration file from: 0x2ffe2000
INFO: FCONF: Reading firmware configuration information for: stm32mp_io
INFO: Using USB
INFO: Instance 2
INFO: Boot used partition fsbl1
NOTICE: BL2: v2.8-stm32mp1-r1.0(debug):()
NOTICE: BL2: Built : 15:37:50, Jul 29 2024
INFO: BL2: Doing platform setup
INFO: RAM: DDR3-DDR3L 16bits 533000kHz
INFO: Memory size = 0x20000000 (512 MB)
INFO: DFU USB START...
INFO: phase ID :3, Manifestation 3 at c712ddfa
INFO: Send detach request
INFO: Receive DFU Detach
INFO: DFU USB STOP...
INFO: BL2: Loading image id 1
INFO: Loading image id=1 at address 0x2ffff000
INFO: Image id=1 loaded: 0x2ffff000 - 0x2ffff1ea
INFO: FCONF: Reading FW_CONFIG firmware configuration file from: 0x2ffff000
INFO: FCONF: Reading firmware configuration information for: dyn_cfg
INFO: FCONF: Reading firmware configuration information for: stm32mp1_firewall
INFO: BL2: Loading image id 4
INFO: Loading image id=4 at address 0xde000000
INFO: Image id=4 loaded: 0xde000000 - 0xde00001c
INFO: OPTEE ep=0xde000000
INFO: OPTEE header info:
INFO: magic=0x4554504f
INFO: version=0x2
INFO: arch=0x0
INFO: flags=0x0
INFO: nb_images=0x1
INFO: BL2: Loading image id 8
INFO: Loading image id=8 at address 0xde000000
INFO: Image id=8 loaded: 0xde000000 - 0xde031bb8
INFO: BL2: Skip loading image id 9
INFO: BL2: Loading image id 2
INFO: Loading image id=2 at address 0xc0500000
INFO: Image id=2 loaded: 0xc0500000 - 0xc05138a8
INFO: BL2: Skip loading image id 16
INFO: BL2: Loading image id 5
INFO: Loading image id=5 at address 0xc0100000
INFO: Image id=5 loaded: 0xc0100000 - 0xc01e866c
NOTICE: BL2: Booting BL32
INFO: Entry point address = 0xde000000
INFO: SPSR = 0x1d3
I/TC: Early console on UART#4
I/TC:
I/TC: Embedded DTB found
I/TC: OP-TEE version: Unknown_3.19 (gcc version 12.2.0 (GCC)) #3 Mon Aug 5 16:44:35 UTC 2024 arm
I/TC: WARNING: This OP-TEE configuration might be insecure!
I/TC: WARNING: Please check https://optee.readthedocs.io/en/latest/architecture/porting_guidelines.html
I/TC: Primary CPU initializing
E/TC:0 0 Panic

I'm building OP-TEE with the following command line:
$ CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=3 CFG_STM32MP_PROFILE=system_services CFG_DRAM_SIZE=0x20000000 make -f $PWD/../Makefile.sdk CFG_EMBED_DTB_SOURCE_FILE=stm32mp153c-som-dfu-mx all

It seems that I should have debugging and/or log messages enabled but the panic message is quite lacking detail. I'm looking for suggestions as to how to try and debug this panic and/or how to fix it.

[debugging]

Hi, have you checked the last pages of the document for the well un-known UART bug ?

[LinuxAddict]

Howdy. Thanks for the response. I do think I'm facing a different problem. The referenced slides indicate a timeout problem with the source u-boot shipped in ecosystem V5. My panic is happening in OP-TEE before it has had a chance to load or execute u-boot.

I may need to provide some additional background:

I'm trying to re-create parts of a BSP from an existing (third party) design under Ecosystem V5. We have a booting and running system complete with eMMC images. Unfortunately the boot binaries contained within BSP that was provided does not support DFU over USB. This is needed for our production process to program our eMMC.

Simply modifying the BSP we already have has proven difficult since it appears to be based on a much older version of the ecosystem and, well, we are unable to access the inputs to this process. To this end I've started rebuilding inputs to ecosystem V5 from Cube MX following the presentation materials linked from this published FAQ:

https://community.st.com/t5/stm32-mpus/faq-stm32mp1-how-to-create-a-device-tree-adapted-to-your-design/ta-p/49260

Since my original post I'm now assuming that OP-TEE panic I'm seeing is either the result of an invalid build flags and/or that OP-TEE DTS generated by CubeMX requires additional modifications which I have yet to discover.

Here is my command line for building the FIP containing OP-TEE:

$ CFG_TEE_CORE_DEBUG=y CFG_TEE_CORE_LOG_LEVEL=3 CFG_STM32MP_PROFILE=system_services CFG_DRAM_SIZE=0x20000000 make -f $PWD/../Makefile.sdk CFG_EMBED_DTB_SOURCE_FILE=stm32mp153c-som-dfu-mx all

My reading of the Makefiles suggests that the platform and MPU variants are extracted from the compatible node in the DTS root. This seems to be confirmed because when I build OP-TEE explicitly adding build options to the command line (PLATFORM=stm32mp1 and CFG_STM32MP15=y) I still get an OP-TEE panic at the same place as without those flags.

For reference, here is our full OP-TEE DTS:

// SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-3-Clause)
/*
 * Copyright (C) 2024, STMicroelectronics - All Rights Reserved
 * Author: STM32CubeMX code generation for STMicroelectronics.
 */

/* For more information on Device Tree configuration, please refer to
 * https://wiki.st.com/stm32mpu/wiki/Category:Device_tree_configuration
 */

/dts-v1/;
#include <dt-bindings/pinctrl/stm32-pinfunc.h>
#include <dt-bindings/clock/stm32mp1-clksrc.h>

#include "stm32mp153.dtsi"
#include "stm32mp15xc.dtsi"
#include "stm32mp15xxac-pinctrl.dtsi"

/* USER CODE BEGIN includes */
/* USER CODE END includes */

/ {
	model = "STMicroelectronics custom STM32CubeMX board - openstlinux-6.1-yocto-mickledore-mp1-v23.06.21";
	compatible = "st,stm32mp153c-som-dfu-mx", "st,stm32mp153";

	memory@c0000000 {
		device_type = "memory";
		reg = <0xc0000000 0x20000000>;

		/* USER CODE BEGIN memory */
		/* USER CODE END memory */
	};

	reserved-memory {
		#address-cells = <1>;
		#size-cells = <1>;
		ranges;

		/* USER CODE BEGIN reserved-memory */
		/* USER CODE END reserved-memory */
	};

	/* USER CODE BEGIN root */

    aliases {
        serial0 = &uart4;
    };
    chosen {
        stdout-path = "serial0:115200n8";
    };

	vin: vin {
		compatible = "regulator-fixed";
		regulator-name = "vin";
		regulator-min-microvolt = <5000000>;
		regulator-max-microvolt = <5000000>;
		regulator-always-on;
	};
    vddcore: vddcore {
        compatible = "regulator-fixed";
        regulator-name = "vddcore";
        regulator-min-microvolt = <1350000>;
        regulator-max-microvolt = <1350000>;
        regulator-always-on;
    };
    vdd: vdd {
        compatible = "regulator-fixed";
        regulator-name = "vdd";
        regulator-min-microvolt = <3300000>;
        regulator-max-microvolt = <3300000>;
        regulator-always-on;
    };


	/* USER CODE END root */

	clocks {
		/* USER CODE BEGIN clocks */
		/* USER CODE END clocks */

		clk_hsi: clk-hsi {
			clock-frequency = <64000000>;

			/* USER CODE BEGIN clk_hsi */
			/* USER CODE END clk_hsi */
		};
		clk_lse: clk-lse {
			clock-frequency = <32768>;

			/* USER CODE BEGIN clk_lse */
			/* USER CODE END clk_lse */
		};
		clk_hse: clk-hse {
			clock-frequency = <24000000>;
			st,bypass;

			/* USER CODE BEGIN clk_hse */
			/* USER CODE END clk_hse */
		};
	};

}; /*root*/

/*Warning: the configuration of the secured GPIOs should be added in (addons) User Section*/
&pinctrl {
	rtc_pins_mx: rtc_mx-0 {
		pins {
			pinmux = <STM32_PINMUX('I', 8, ANALOG)>; /* RTC_LSCO */
		};
	};

	/* USER CODE BEGIN pinctrl */
    uart4_pins_mx: uart4_mx-0 {
        pins1 {
            pinmux = <STM32_PINMUX('B', 2, AF8)>; /* UART4_RX */
        bias-disable;
        };
        pins2 {
            pinmux = <STM32_PINMUX('G', 11, AF6)>; /* UART4_TX */
            bias-disable;
            drive-push-pull;
            slew-rate = <0>;
        };
    };
	/* USER CODE END pinctrl */
};

&pinctrl_z {
	/* USER CODE BEGIN pinctrl_z */
	/* USER CODE END pinctrl_z */
};

&rcc{
	status = "okay";

	/* USER CODE BEGIN rcc */
	/* USER CODE END rcc */

	st,clksrc=<
		CLK_CKPER_DISABLED
		CLK_ETH_PLL4P
		CLK_SDMMC12_PLL4P
		CLK_STGEN_HSE
		CLK_SPI2S1_PLL4P
		CLK_SPI2S23_PLL4P
		CLK_SPI45_PCLK2
		CLK_I2C46_PCLK5
		CLK_SDMMC3_PLL4P
		CLK_USBO_USBPHY
		CLK_ADC_PLL4R
		CLK_I2C12_PCLK1
		CLK_UART1_PCLK5
		CLK_UART24_PCLK1
		CLK_UART35_PCLK1
		CLK_FDCAN_HSE
		CLK_MPU_PLL1P
		CLK_AXI_PLL2P
		CLK_MCU_PLL3P
		CLK_RTC_LSI
		CLK_MCO1_DISABLED
		CLK_MCO2_DISABLED
	>;
	st,clkdiv = <
		DIV(DIV_MPU, 1)
		DIV(DIV_AXI, 0)
		DIV(DIV_MCU, 0)
		DIV(DIV_APB1, 1)
		DIV(DIV_APB2, 1)
		DIV(DIV_APB3, 1)
		DIV(DIV_APB4, 1)
		DIV(DIV_APB5, 2)
		DIV(DIV_RTC, 0)
		DIV(DIV_MCO1, 0)
		DIV(DIV_MCO2, 0)
	>;
	st,pll_vco {
		pll1_vco_1300Mhz: pll1-vco-1300Mhz {
			src=< CLK_PLL12_HSE >;
			divmn = < 2 80 >;
			frac = < 0x800 >;
		};
		pll2_vco_1066Mhz: pll2-vco-1066Mhz {
			src=< CLK_PLL12_HSE >;
			divmn = < 2 65 >;
			frac = < 0x1400 >;
		};
		pll3_vco_800Mhz: pll3-vco-800Mhz {
			src=< CLK_PLL3_HSE >;
			divmn = < 2 99 >;
		};
		pll4_vco_750Mhz: pll4-vco-750Mhz {
			src=< CLK_PLL4_HSE >;
			divmn = < 1 61 >;
			frac = < 0x1000 >;
		};
		/* USER CODE BEGIN rcc_st-pll_vco */
		/* USER CODE END rcc_st-pll_vco */
	};

	pll1:st,pll@0 {
		compatible = "st,stm32mp1-pll";
		reg = <0>;

		st,pll = < &pll1_cfg1 >;

		pll1_cfg1: pll1_cfg1 {
			st,pll_vco = < &pll1_vco_1300Mhz >;
			st,pll_div_pqr = < 0 1 1 >;
		};
		/* USER CODE BEGIN pll1 */
		/* USER CODE END pll1 */
	};

	pll2:st,pll@1 {
		compatible = "st,stm32mp1-pll";
		reg = <1>;

		st,pll = < &pll2_cfg1 >;

		pll2_cfg1: pll2_cfg1 {
			st,pll_vco = < &pll2_vco_1066Mhz >;
			st,pll_div_pqr = < 1 1 0 >;
		};
		/* USER CODE BEGIN pll2 */
		/* USER CODE END pll2 */
	};

	pll3:st,pll@2 {
		compatible = "st,stm32mp1-pll";
		reg = <2>;

		st,pll = < &pll3_cfg1 >;

		pll3_cfg1: pll3_cfg1 {
			st,pll_vco = < &pll3_vco_800Mhz >;
			st,pll_div_pqr = < 3 3 3 >;
		};
		/* USER CODE BEGIN pll3 */
		/* USER CODE END pll3 */
	};

	pll4:st,pll@3 {
		compatible = "st,stm32mp1-pll";
		reg = <3>;

		st,pll = < &pll4_cfg1 >;

		pll4_cfg1: pll4_cfg1 {
			st,pll_vco = < &pll4_vco_750Mhz >;
			st,pll_div_pqr = < 5 5 5 >;
		};
		/* USER CODE BEGIN pll4 */
		/* USER CODE END pll4 */
	};

};

&rtc{
	pinctrl-names = "default";
	pinctrl-0 = <&rtc_pins_mx>;
	status = "okay";

	/* USER CODE BEGIN rtc */
	/* USER CODE END rtc */
};

/* USER CODE BEGIN addons */
&bsec {
    board_id: board_id@ec {
        reg = <0xec 0x4>;
        st,non-secure-otp;
    };
};

&uart4 {
    pinctrl-names = "default";
    pinctrl-0 = <&uart4_pins_mx>;
    status = "okay";

};

&cpu0 {
    cpu-supply = <&vddcore>;
};

&cpu1 {
    cpu-supply = <&vddcore>;
};

&pwr_regulators {
    vdd-supply = <&vdd>;
};

/* USER CODE END addons */

Thanks,

_Linux Addict_ 

[debugging]

I have been through the same (horrors) migrating a custom board from ECO 1 to 5 and took 1.5 months to boot ...  .Though booting, still have not been able to get all I/O working. Do you plan to run OPTEE in DDR only or SRAM swap with DDR ?

[LinuxAddict]

Agreed, this process seems poorly documented: A link from an FAQ on the WIKI to a google drive folder containing dozens of presentations and buried in a "random" zip folder is one document that kinda-sorta describes the process for bringing up a board. It tells you what steps need to be done for an ST board but not why nor what is actually required for a non-ST board. I expected much better.

Rant aside, the short answer is "whichever path is most expedient". I'm going through this pain only so that I can get DFU over USB working. Once we get DFU working I'll use it to program the images we already have (which are working without OP-TEE). It seems heavy handed to require OP-TEE in the ecosystem--there are certainly applications where it is not required and in those cases its inclusion adds only unnecessary complexity.

As for the OP-TEE panic I'm seeing I was able to track down that OP-TEE is crashing as it is probing the DT and drivers referenced. So at least I've narrowed down the problem space and have reasonable confidence in my build flags and environment. I'm assuming that OP-TEE build correctly with the ST's DTS files so eventually I should be able to figure out what the problem bits are in my DTS and get execution handed off to u-boot to program my Flash.

[LinuxAddict · Accepted solution]

I've finally figured the cause for *this* OP-TEE panic.

The fundamental issue is that my device-tree specified a vddcore regulator of 1.35V which was copied from the presentation slides detailing DT modifications for OP-TEE. My device is an STM32MP153C with operating point of 650 MHz at 1.2V. The operating point for the target device in the presentation is 800 MHz at 1.35V.

OP-TEE couldn't find a matching regulator and panicked without useful output.

The lack of useful output is because I kept trying to enable CFG_TEE_CORE_DEBUG on the command and didn't realize that Makefile.sdk was silently turning it back off. Makefile.sdk modified, useful debug output appears, issue able to be diagnosed and resolved.

[debugging]

Great info. Those are things that could go into a "common pitfalls" checklist.