LWIP + D-Cache problem (STM32H747I-DISCO)

hzoli · ‎2024-02-10

Greetings,

I have a problem with LWIP when the D-Cache is enabled.
My test program:

int main(void)
{
  /* USER CODE BEGIN 1 */
  /* USER CODE END 1 */

/* USER CODE BEGIN Boot_Mode_Sequence_0 */
  int32_t timeout;

/* USER CODE END Boot_Mode_Sequence_0 */

/* Enable the CPU Cache */
  /* Enable I-Cache---------------------------------------------------------*/
  SCB_EnableICache();

  /* Enable D-Cache---------------------------------------------------------*/
  SCB_EnableDCache();

/* USER CODE BEGIN Boot_Mode_Sequence_1 */

  /* Wait until CPU2 boots and enters in stop mode or timeout*/
  timeout = 0xFFFF;

  while((__HAL_RCC_GET_FLAG(RCC_FLAG_D2CKRDY) != RESET) && (timeout-- > 0));
  if ( timeout < 0 )
  {
  Error_Handler();
  }

/* USER CODE END Boot_Mode_Sequence_1 */
  /* MCU Configuration--------------------------------------------------------*/
  /* Reset of all peripherals, Initializes the Flash interface and the Systick. */
  HAL_Init();

  /* USER CODE BEGIN Init */
  /* USER CODE END Init */

  /* Configure the system clock */
  SystemClock_Config();

/* Configure the peripherals common clocks */
  PeriphCommonClock_Config();

/* USER CODE BEGIN Boot_Mode_Sequence_2 */
/* When system initialization is finished, Cortex-M7 will release Cortex-M4 by means of HSEM notification */

/*HW semaphore Clock enable*/
__HAL_RCC_HSEM_CLK_ENABLE();

/*Take HSEM */
HAL_HSEM_FastTake(HSEM_ID_0);

/*Release HSEM in order to notify the CPU2(CM4)*/
HAL_HSEM_Release(HSEM_ID_0,0);

/* wait until CPU2 wakes up from stop mode */
timeout = 0xFFFF;

while((__HAL_RCC_GET_FLAG(RCC_FLAG_D2CKRDY) == RESET) && (timeout-- > 0));

if ( timeout < 0 )
{
Error_Handler();
}

/* USER CODE END Boot_Mode_Sequence_2 */
  /* USER CODE BEGIN SysInit */
   
  /* USER CODE END SysInit */
  /* Initialize all configured peripherals */
  MX_GPIO_Init();
  MX_USART1_UART_Init();
  MX_LWIP_Init();
  MX_RAMECC_Init();
  /* USER CODE BEGIN 2 */

  printf("H747 - M7 - " __DATE__ " " __TIME__ "\r\n");

  struct udp_pcb *pcb;
  err_t err;
  struct pbuf *p;

  pcb = udp_new();

  if (!pcb)
  {
     printf("udp_new() failed\r\n");
     Error_Handler();
  }

  ip_addr_t dest_ip;
  IP4_ADDR(&dest_ip, 10, 97, 0, 1);
  const u16_t dest_port = 1997;

  if ((err = udp_connect(pcb, &dest_ip, dest_port)) != ERR_OK)
  {
     printf("udp_connect() failed\r\n");
     Error_Handler();
  }

  /* USER CODE END 2 */
  /* Infinite loop */

  /* USER CODE BEGIN WHILE */
  while (1)
  {
    /* USER CODE END WHILE */
    /* USER CODE BEGIN 3 */

   MX_LWIP_Process();

   p = pbuf_alloc(PBUF_RAW_TX, 3, PBUF_RAM);
   if (!p)
   {
       printf("pbuf_alloc() failed\r\n");
       Error_Handler();
   }

   pbuf_take(p, (char*)"123", 3);

   if (udp_send(pcb, p) != ERR_OK)
   {
       printf("udp_send() failed\r\n");
       //Error_Handler();
   }

   pbuf_free(p);
  }

  /* USER CODE END 3 */
}

it's working fine when comment out the SCB_EnableDCache().

Log when D-Cache disabled:

pbuf_alloced_custom(length=0)
ethernet_input: dest:0hx:0hx:0hx:0hx:0hx:0hx, src:0hx:0hx:0hx:0hx:0hx:0hx, type:ff
pbuf_remove_header: old 0x24004048 new 0x24004056 (14)
ip_input: iphdr->dest 0xffffff0a netif->ip_addr 0x200610a (0xffff0a, 0x610a, 0xff000000)
ip4_input: packet not for us.
pbuf_free(0x24004028)
pbuf_free: deallocating 0x24004028
pbuf_alloced_custom(length=0)
ethernet_input: dest:0hx:0hx:0hx:0hx:0hx:0hx, src:0hx:0hx:0hx:0hx:0hx:0hx, type:ff
pbuf_remove_header: old 0x24003a28 new 0x24003a36 (14)
etharp_update_arp_entry: 10.97.0.1 - e0:d5:5e:31:03:c3
etharp_find_entry: found matching entry 0
etharp_update_arp_entry: updating stable entry 0
etharp_input: incoming ARP request
etharp_input: ARP request was not for us.
pbuf_free(0x24003a08)
pbuf_free: deallocating 0x24003a08
pbuf_alloced_custom(length=0)
ethernet_input: dest:0hx:0hx:0hx:0hx:0hx:0hx, src:0hx:0hx:0hx:0hx:0hx:0hx, type:ff
pbuf_remove_header: old 0x24003408 new 0x24003416 (14)
ip_input: iphdr->dest 0xffffff0a netif->ip_addr 0x200610a (0xffff0a, 0x610a, 0xff000000)
ip4_input: packet not for us.
pbuf_free(0x240033e8)
pbuf_free: deallocating 0x240033e8
sct calling h=ip_reass_tmr t=975 arg=0x801af6c
tcpip: ip_reass_tmr()
sys_timeout: 0x240055ac abs_time=22308 handler=ip_reass_tmr arg=0x801af6c
sct calling h=etharp_tmr t=974 arg=0x801af78
tcpip: etharp_tmr()
etharp_timer
sys_timeout: 0x24005598 abs_time=22321 handler=etharp_tmr arg=0x801af78
pbuf_alloc(length=3)
pbuf_alloc(length=3) == 0x30044008
udp_send
pbuf_add_header: failed as 0x30044010 < 0x30044018 (not enough space for new header size)
pbuf_alloc(length=8)
pbuf_alloc(length=8) == 0x30044024
pbuf_chain: 0x30044024 references 0x30044008
udp_send: added header pbuf 0x30044024 before given pbuf 0x30044008
udp_send: sending datagram of length 11
udp_send: UDP packet length 11
udp_send: UDP checksum 0x0000
udp_send: ip_output_if (,,,,0x11,)
pbuf_add_header: old 0x30044058 new 0x30044044 (20)
ip4_output_if: st0
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 31 | (v, hl, tos, len)
+-------------------------------+
| 16 |000| 0 | (id, flags, offset)
+-------------------------------+
| 255 | 17 | 0x0000 | (ttl, proto, chksum)
+-------------------------------+
| 10 | 97 | 0 | 2 | (src)
+-------------------------------+
| 10 | 97 | 0 | 1 | (dest)
+-------------------------------+
ip4_output_if: call netif->output()
pbuf_add_header: old 0x30044044 new 0x30044036 (14)
ethernet_output: sending packet 0x30044024
Prepare bytes: E0h D5h 5Eh 31h
pbuf_free(0x30044024)
pbuf_free: deallocating 0x30044024
pbuf_free: 0x30044008 has ref 1, ending here.
pbuf_free(0x30044008)
pbuf_free: deallocating 0x30044008

Log when D-Cache enabled:

sct calling h=ip_reass_tmr t=358 arg=0x801afec
tcpip: ip_reass_tmr()
sys_timeout: 0x240055ac abs_time=26418 handler=ip_reass_tmr arg=0x801afec
sct calling h=etharp_tmr t=328 arg=0x801aff8
tcpip: etharp_tmr()
etharp_timer
etharp_timer: expired pending entry 0.
etharp_free_entry: freeing entry 0, packet queue 0x30044024.
pbuf_free(0x30044024)
pbuf_free: deallocating 0x30044024
pbuf_free: deallocating 0x30044008
sys_timeout: 0x24005598 abs_time=26460 handler=etharp_tmr arg=0x801aff8
pbuf_alloc(length=3)
pbuf_alloc(length=3) == 0x30044008
udp_send
pbuf_add_header: failed as 0x30044010 < 0x30044018 (not enough space for new header size)
pbuf_alloc(length=8)
pbuf_alloc(length=8) == 0x30044024
pbuf_chain: 0x30044024 references 0x30044008
udp_send: added header pbuf 0x30044024 before given pbuf 0x30044008
udp_send: sending datagram of length 11
udp_send: UDP packet length 11
udp_send: UDP checksum 0x0000
udp_send: ip_output_if (,,,,0x11,)
pbuf_add_header: old 0x30044058 new 0x30044044 (20)
ip4_output_if: st0
IP header:
+-------------------------------+
| 4 | 5 | 0x00 | 31 | (v, hl, tos, len)
+-------------------------------+
| 20 |000| 0 | (id, flags, offset)
+-------------------------------+
| 255 | 17 | 0x0000 | (ttl, proto, chksum)
+-------------------------------+
| 10 | 97 | 0 | 2 | (src)
+-------------------------------+
| 10 | 97 | 0 | 1 | (dest)
+-------------------------------+
ip4_output_if: call netif->output()
etharp_find_entry: found empty entry 0
etharp_find_entry: selecting empty entry 0
etharp_request: sending ARP request.
pbuf_alloc(length=28)
pbuf_alloc(length=28) == 0x30044068
etharp_raw: sending raw ARP packet.
pbuf_add_header: old 0x30044088 new 0x3004407a (14)
ethernet_output: sending packet 0x30044068
Prepare bytes: FFh FFh FFh FFh
ETH_ERROR_BUSY
pbuf_free(0x30044068)
pbuf_free: deallocating 0x30044068
etharp_query: queued packet 0x30044024 on ARP entry 0
pbuf_free(0x30044024)
pbuf_free: 0x30044024 has ref 1, ending here.
pbuf_free(0x30044008)
pbuf_free: 0x30044008 has ref 1, ending here.

Has anyone encountered this problem?

KDJEM.1 · ‎2024-02-12

Hello @hzoli and welcome to the Community 🙂,

I recommend you to take a look at How to create project for STM32H7 with Ethernet an... - STMicroelectronics Community and at STM32H747_Disco_M7_ETH example may help you. The Dcache is enabled in these examples.

Also, I advise you to check the MPU configuration.

Pease note that, if the cache is enabled, it is requires to follow this order: configure MPU firstly, then enable cache.

  /* MPU Configuration--------------------------------------------------------*/
  MPU_Config();

  /* Enable I-Cache---------------------------------------------------------*/
  SCB_EnableICache();

  /* Enable D-Cache---------------------------------------------------------*/
  SCB_EnableDCache();

Thank you.

Kaouthar

To give better visibility on the answered topics, please click on Accept as Solution on the reply which solved your issue or answered your question.

View solution in original post

KDJEM.1 · ‎2024-02-12

Hello @hzoli and welcome to the Community 🙂,

I recommend you to take a look at How to create project for STM32H7 with Ethernet an... - STMicroelectronics Community and at STM32H747_Disco_M7_ETH example may help you. The Dcache is enabled in these examples.

Also, I advise you to check the MPU configuration.

Pease note that, if the cache is enabled, it is requires to follow this order: configure MPU firstly, then enable cache.

  /* MPU Configuration--------------------------------------------------------*/
  MPU_Config();

  /* Enable I-Cache---------------------------------------------------------*/
  SCB_EnableICache();

  /* Enable D-Cache---------------------------------------------------------*/
  SCB_EnableDCache();

Thank you.

Kaouthar

To give better visibility on the answered topics, please click on Accept as Solution on the reply which solved your issue or answered your question.

SofLit · ‎2024-02-12

Hello,

You need to disable the cache for Rx and Tx ETH descriptors and LWIP heap using MPU. You have a data coherency issue:

So as said by @KDJEM.1 , refer to the link https://community.st.com/t5/stm32-mcus/how-to-create-project-for-stm32h7-with-ethernet-and-lwip-stack/ta-p/49308

Also please refer to the AN4839 "Level 1 cache on STM32F7 Series and STM32H7 Series" for cache usage and mistakes to avoid.

To give better visibility on the answered topics, please click on "Accept as Solution" on the reply which solved your issue or answered your question.

hzoli · ‎2024-02-12

Thank you all! After I configured the MPU, the LWIP heap address and modified the linker script, now working perfectly.

@SofLit Thank you for the tip and the documentation link about "Level 1 cache".