Bringing up Ethernet on STM32MP153 with KSZ8081RNB in RMII mode and 25 MHz clock on ETH_CLK (no PHY Crystal)

velaros · ‎2021-08-04

Hello!

I have an issue on a custom board with STM32MP153CAB and KSZ8081RNB.

TF-A, FIP and kernel were built in Developer package. Linux distribution was built as startard configuration st-image-core in Distribution package.

Clocks and device trees are configured like specified in https://wiki.st.com/stm32mpu/wiki/Ethernet_device_tree_configuration#RMII_with_25MHz_on_ETH_CLK_-28no_PHY_Crystal-29-2C_REF_CLK_from_PHY_-28Reference_clock_-28standard_RMII_clock_name-29_is_provided_by_a_RCC_SoC_internal_clock-29

Here are fragments of device tree for U-Boot and kernel (generated by STM32CubeIDE and manually edited):

eth1_pins_mx: eth1_mx-0 {
		pins1 {
			pinmux = <STM32_PINMUX('A', 1, AF11)>, /* ETH1_REF_CLK */
					 <STM32_PINMUX('C', 1, AF11)>, /* ETH1_MDC */
					 <STM32_PINMUX('G', 8, AF2)>, /* ETH1_CLK */
					 <STM32_PINMUX('G', 13, AF11)>, /* ETH1_TXD0 */
					 <STM32_PINMUX('G', 14, AF11)>; /* ETH1_TXD1 */
			bias-disable;
			drive-push-pull;
			slew-rate = <1>;
		};
		pins2 {
			pinmux = <STM32_PINMUX('A', 2, AF11)>; /* ETH1_MDIO */
			bias-disable;
			drive-push-pull;
			slew-rate = <0>;
		};
		pins3 {
			pinmux = <STM32_PINMUX('A', 7, AF11)>, /* ETH1_CRS_DV */
					 <STM32_PINMUX('C', 4, AF11)>, /* ETH1_RXD0 */
					 <STM32_PINMUX('C', 5, AF11)>; /* ETH1_RXD1 */
			bias-disable;
		};
		pins4 {
			pinmux = <STM32_PINMUX('B', 11, AF11)>; /* ETH1_TX_EN */
		};
	};
 
	eth1_sleep_pins_mx: eth1_sleep_mx-0 {
		pins {
			pinmux = <STM32_PINMUX('A', 1, ANALOG)>, /* ETH1_REF_CLK */
					 <STM32_PINMUX('A', 2, ANALOG)>, /* ETH1_MDIO */
					 <STM32_PINMUX('A', 7, ANALOG)>, /* ETH1_CRS_DV */
					 <STM32_PINMUX('B', 11, ANALOG)>, /* ETH1_TX_EN */
					 <STM32_PINMUX('C', 1, ANALOG)>, /* ETH1_MDC */
					 <STM32_PINMUX('C', 4, ANALOG)>, /* ETH1_RXD0 */
					 <STM32_PINMUX('C', 5, ANALOG)>, /* ETH1_RXD1 */
					 <STM32_PINMUX('G', 8, ANALOG)>, /* ETH1_CLK */
					 <STM32_PINMUX('G', 13, ANALOG)>, /* ETH1_TXD0 */
					 <STM32_PINMUX('G', 14, ANALOG)>; /* ETH1_TXD1 */
		};
	};
 
&ethernet0{
	pinctrl-names = "default", "sleep";
	pinctrl-0 = <&eth1_pins_mx>;
	pinctrl-1 = <&eth1_sleep_pins_mx>;
	status = "okay";
 
	/* USER CODE BEGIN ethernet0 */
 
	phy-mode = "rmii";
	max-speed = <100>;
	phy-handle = <&phy0>;
	local-mac-address = [XX XX XX XX XX XX];
	mdio0 {
			#address-cells = <1>;
			#size-cells = <0>;
			compatible = "snps,dwmac-mdio";
			phy0: ethernet-phy@1 {
					reg = <1>;
					clocks = <&rcc ETHCK_K>;
					clock-names = "rmii-ref";
       		};
	};
 
	/* USER CODE END ethernet0 */
};

At first, I tested it in U-Boot.

After power on and reset it seems like MPU does not see PHY (and leds on RJ-45 are off). "mdio list" command shows only this:

ethernet@5800a000:

After I set ipaddr and make ping, leds on RJ-45 become on and the answer to ping is received. The same thing after dhcp or bootp command.

"mdio list" now shows:

ethernet@5800a000:
1 - Micrel KSZ8081 <--> ethernet@5800a000

When kernel loads, the following log appears:

...
[    2.828774] stm32-dwmac 5800a000.ethernet: IRQ eth_lpi not found
[    2.833564] stm32-dwmac 5800a000.ethernet: PTP uses main clock
[    2.839294] stm32-dwmac 5800a000.ethernet: no reset control found
[    2.845980] stm32-dwmac 5800a000.ethernet: User ID: 0x40, Synopsys ID: 0x42
[    2.852185] stm32-dwmac 5800a000.ethernet:   DWMAC4/5
[    2.857216] stm32-dwmac 5800a000.ethernet: DMA HW capability register supported
[    2.864426] stm32-dwmac 5800a000.ethernet: RX Checksum Offload Engine supported
[    2.871772] stm32-dwmac 5800a000.ethernet: TX Checksum insertion supported
[    2.878618] stm32-dwmac 5800a000.ethernet: Wake-Up On Lan supported
[    2.884974] stm32-dwmac 5800a000.ethernet: TSO supported
[    2.890173] stm32-dwmac 5800a000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[    2.897994] stm32-dwmac 5800a000.ethernet: Enabled Flow TC (entries=2)
[    2.904475] stm32-dwmac 5800a000.ethernet: TSO feature enabled
[    2.910324] stm32-dwmac 5800a000.ethernet: Using 32 bits DMA width
[    2.917336] libphy: stmmac: probed
...
[   14.178529] stm32-dwmac 5800a000.ethernet eth0: PHY [stmmac-0:01] driver [Micrel KSZ8081 or KSZ8091] (irq=POLL)
[   14.264265] stm32-dwmac 5800a000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported
[   14.333679] stm32-dwmac 5800a000.ethernet eth0: registered PTP clock
[   14.369464] stm32-dwmac 5800a000.ethernet eth0: configuring for phy/rmii link mode
...

And I can not make ethernet work in kernel though it must start work automatically with DHCP.

ifconfig says that link is up, but ethtool shows that link is not detected. Autonegotiation does not start on "ethtool -r eth0".

I launched the same linux image (with my builds of TF-A, FIP and device trees) on STM32MP157C-DK2 board. Ethernet starts work immediately after kernel booting or cable plugging in RJ-45. In U-Boot it works the same way as on my board.

What can be the isuue?

PatrickF · ‎2021-09-13

Hi,

I'm puzzled with your statements, what is your target ?

ETH_CLK providing 25MHz to XI. REF_CLK signal from PHY to STM32MP15x ETH pin.
ETH_CLK providing 50MHz to XI. No REF_CLK wire (i.e. internal STM32MP15x connection to ETH IP)

I'm not sure the Micrel driver support the 1st option (i.e. it might support only crystal for 25MHz), so might explain why there is no 25MHz.

I'm not SW expert, but as far as I have understand the Micrel driver might assume 50MHz as soon as external clock is defined in the DT.

Could you please confirm which ecosystem are you using (ecosystem v2.1 / kernel 5.4 or ecosystem v3.0 / kernel 5.10) ?

It is important to know as I already mentioned, the device Tree for ETH_CLK setting has evolved between this 2 ecosystems versions.

Regards.

In order to give better visibility on the answered topics, please click on 'Accept as Solution' on the reply which solved your issue or answered your question.

velaros · ‎2021-09-13

I am using the v3.0 ecosystem with kernel 5.10 (I guess so because I downloaded archives from this page https://wiki.st.com/stm32mpu/wiki/STM32MP1_Developer_Package).

I want to test my KSZ8081RNB in both modes you mentioned:

1) 25MHz mode: ETH_CLK output of STM32 feeds 25MHz to XI of KSZ8081 and KSZ8081's REF_CLK output feeds 50 MHz to ETH_REF_CLK input of STM32. Micrel's datasheet says that this mode is supported with both crysal and external clock. As I have seen in kernel drivers sources, this mode is really supported by SW for this PHY.

After checking signal on PCB and with /sys/kernel/debug/clk/clk_summary I discovered that when I setup STM32's ETH_CLK output to generate 25MHz clock, STM32 doesn't generate it for some reason (as I shown in message above). Of course, PHY doesn't not work properly when it doesn't receive clock.

I guess there may be some problem in STM32 clock settings.

2) 50MHz mode: ETH_CLK output of STM32 feeds 50MHz to XI of KSZ8081 and no REF_CLK wire used (internal connection provides ETH_REF_CLK).

In this mode all works properly.

So my question is what am I doing wrong when I setup STM32 in 25MHz mode.

PatrickF · ‎2021-09-13

Did you double check your DT ? Apart clock settings (which seems ok), st,ext-phyclk; should be present ONLY in 50MHz case.

See wiki:

(Both are very same DT regarding &ethernet0 node, except the st,ext-phyclk;)

I repeat myself, st,eth-ref-clk-sel (or st,eth_ref_clk_sel for uBoot) is for ecosystem v2.1 using kernel 5.4.

For ecosystem v3 and kernel 5.10, it is now st,ext-phyclk (for both kernel and uBoot).

Regards.

In order to give better visibility on the answered topics, please click on 'Accept as Solution' on the reply which solved your issue or answered your question.

velaros · ‎2021-09-13

Yes, I checked my devicetrees carefully. I used st,ext-phyclk in 50 MHz and it worked fine. Where I can read that it is used since 5.10? Wiki says only about st,eth-ref-clk-sel.

Moreover, I tried using st,ext-phyclk in 25 MHz (please don't blame me for this) and got interesting results. ETH_CLK became enabled in clk_summary, also I can see 25 MHz to PHY and 50 MHz from PHY on PCB with oscilloscope. Autonegotiation done successfully in kernel but I cannot ping any address ("network unreachable") even when I configure network manually (while at 50 MHz it automatically connects with DHCP). Maybe with this hack I somehow enabled ETH_CLK but MAC uses internal ref clock instead of external and not works properly.

PatrickF · ‎2021-09-13

Hi,

could you please provide me the wiki link talking about st,eth-ref-clk-sel ?

There is more than one wiki: there is the latest ecosystem wiki and the archives wiki.

https://wiki.st.com/stm32mpu for latest ecosystem (V3.x, kernel 5.10). See STM32MP15 ecosystem release note
https://wiki.st.com/stm32mpu-ecosystem-v2/ archive for ecosystem v2.1 (kernel 5.4). STM32MP15 ecosystem release note

I don't blame you for trials, it is normal and I probably been doing same.

I think when you use st,ext-phyclk , you select clocking of the MAC from ETH_CLK instead of REF_CLK (see SYSCFG ETH_REF_CLK_SEL bit), so the MAC is wrongly clocked with 25MHz instead of 50MHz.

As far as I understand, autonegotiation is done inside the PHY and SW use MDIO bus to get result, so this will work, but you cannot send/receive any data.

not normal that 25MHz clock is not started with 25MHz config from wiki.

please check the clock is really defined in the ethernet0 DT node (and not in a sub node as I see on your first post)

       clock-names = "stmmaceth",
                     "eth-ck",   <-- check this
                     "mac-clk-tx",
                     "mac-clk-rx",
                     "ethstp";
       clocks = <&rcc ETHMAC>,
                <&rcc ETHCK_K>,          <-- check this      
                <&rcc ETHTX>,
                <&rcc ETHRX>,
                <&rcc ETHSTP>;

Regards.

In order to give better visibility on the answered topics, please click on 'Accept as Solution' on the reply which solved your issue or answered your question.

velaros · ‎2021-09-14

st,eth-ref-clk-sel is mentioned in the same page you and me references above: https://wiki.st.com/stm32mpu/wiki/Ethernet_device_tree_configuration

st,ext-phyclk is not mentioned in this page. Nevertheless, when I search "phyclk" in wiki it shows that this page contains it.

Maybe I can see the old page because I haven't some permission. As you can see, my version is from October 2020 while the last edition was made 2 month ago.

I have clock definitions as in your example, they are defined by default in stm32mp151.dtsi in kernel dts directory.

In the first message I define "rmii-ref" clock for Micrel driver to know the frequency of ETH_CLK and setup proper PHY mode (https://www.kernel.org/doc/Documentation/devicetree/bindings/net/micrel.txt).

Maybe I missed some important configuration from the newer version of wiki page.

PatrickF · ‎2021-09-14

Hi,

I have to apologize, your are right regarding st,eth-ref-clk-sel (for Linux DT) and st,eth_ref_clk_sel (for uBoot DT).

The wiki page I see internally is a draft for Ecosystem V3.0 (it is for next minor release planned in some weeks). My fault not having seen that before.

Unfortunately, with my SW knowledge, I have not much more to help you for the 25MHz case (the one you intend to use) and explain why the clock is not started.

I will check if somebody else could help you on this thread.

Regards.

In order to give better visibility on the answered topics, please click on 'Accept as Solution' on the reply which solved your issue or answered your question.

OlivierK · ‎2021-09-14

Hi velaros (Community Member)

Looking at the Ethernet driver in <linux-5.10.10>/drivers/net/ethernet/stmicro/stmmac/dwmac-stm32.c

Indeed the st,ext-phyclk seems also needed for the RMII 25MHz no crystal configuration with the ostl DV 3.0 version. Which is not what the wiki documentation says, apologies for that. Each DV version and each configuration has a specific DT configuration to respect (DV2.x, DV3.0, DV3.1)

Regarding the fact ping is still not available in 25MHz configuration, check the SYSCFG ETH_REF_CLK_SEL bit as explained by Patrick above.

I think when you use st,ext-phyclk , you select clocking of the MAC from ETH_CLK instead of REF_CLK (see SYSCFG ETH_REF_CLK_SEL bit), so the MAC is wrongly clocked with 25MHz instead of 50MHz.

Regards,

Olivier

velaros · ‎2021-09-15

I checked this and it seems that SYSCFG ETH_REF_CLK_SEL bit is set, so MAC is really clocked wrongly.

I digged in the linux driver (https://github.com/STMicroelectronics/linux/blob/v5.10-stm32mp/drivers/net/ethernet/stmicro/stmmac/dwmac-stm32.c) and discovered that in stm32mp1_set_mode() function it always sets this bit with enabling clock.

So using 25 MHz mode without crystal is impossible at all:

...
case PHY_INTERFACE_MODE_RMII:
	val = SYSCFG_PMCR_ETH_SEL_RMII;
	if ((clk_rate == ETH_CK_F_25M || clk_rate == ETH_CK_F_50M) &&
		(dwmac->eth_ref_clk_sel_reg || dwmac->ext_phyclk)) {
		dwmac->enable_eth_ck = true;
		val |= SYSCFG_PMCR_ETH_REF_CLK_SEL;
	}
	pr_debug("SYSCFG init : PHY_INTERFACE_MODE_RMII\n");
	break;
...

I changed this to the following:

...
case PHY_INTERFACE_MODE_RMII:
	val = SYSCFG_PMCR_ETH_SEL_RMII;
	if (clk_rate == ETH_CK_F_25M || clk_rate == ETH_CK_F_50M) {
		dwmac->enable_eth_ck = true;
		if (clk_rate == ETH_CK_F_50M && 
			(dwmac->eth_ref_clk_sel_reg || dwmac->ext_phyclk)) {
			val |= SYSCFG_PMCR_ETH_REF_CLK_SEL;
		}
	}
	pr_debug("SYSCFG init : PHY_INTERFACE_MODE_RMII\n");
	break;
...

And now it finally works fine in 25 MHz mode, and I can see clock on PCB.

I don't really get the meaning of "st,ext-phyclk" so my patch may be wrong. But the current driver definitely seems buggy. I didn't checked U-Boot but there also may be similar bugs.

As I understand, in the master of the ST Linux repository this driver is reworked, so it may be not the issue in future releases.

OlivierK · ‎2021-09-15

Hi velaros (Community Member)

Indeed you've just found a bug in the dwmac-stm32.c driver DV3.0 version. And you are right in the next versions (DV3.1 +) the "st,ext-phyclk" will be removed for the 25MHz no crystal configuration. Your patch is correct, I will report that as bug for correction on the DV3.0 version.

Thank you for this thread of messages and for your contribution in solving this issue.

Regards,

Olivier