Problem with slow response from LWIP to incoming data on STM32F7

[Jeremi Wójcicki]

Posted on September 15, 2017 at 19:00

Hi guys,

I am developing software for Stm32F7 with ethernet intefrace (nucleo 144 board). I have prepared and compiled LWIP stack with netconn api enabled. I generated configuration files with cubemx using default settings. I start the stack in a separate thread and use a standard TCP/IP code to mirror incoming data (the RTOS echo example):

void EthernetTask(void const * argument)

{

/* USER CODE BEGIN EthernetTask */

LWIP_UNUSED_ARG(argument);

/* initialize LightWeight IP Stack */

MX_LWIP_Init();

/* Create variables needed for servicing a connection */

struct netconn *conn, *newconn;

err_t err, accept_err;

struct netbuf *buf;

void *data;

u16_t len;

/* Create a new connection identifier. */

conn = netconn_new(NETCONN_TCP);

if (conn != NULL)

{

/* Bind connection to well known port number 7. */

err = netconn_bind(conn, NULL, 7);

if (err == ERR_OK)

{

/* Tell connection to go into listening mode. */

netconn_listen(conn);

/* Infinite loop */

while (1)

{

/* Grab new connection. */

accept_err = netconn_accept(conn, &newconn);

/* Process the new connection. */

if (accept_err == ERR_OK)

{

while (netconn_recv(newconn, &buf) == ERR_OK)

{

do

{

netbuf_data(buf, &data, &len);

netconn_write(newconn, data, len, NETCONN_COPY);

}

while (netbuf_next(buf) >= 0);

netbuf_delete(buf);

}

/* Close connection and discard connection identifier. */

netconn_close(newconn);

netconn_delete(newconn);

}

}

}

else

{

netconn_delete(newconn);

}

}

/* USER CODE END EthernetTask */

}

The stack works fine and I can receive data and send it back to the user. The problem is that the reaction time of the stack is very long, in the sense that the echo takes like even 1 second to bounce back to the user. It does not seem that my uC is busy doing anything else, it more seems like the stack waits quite some time before reacting to data. It is even reflected in ping, every second time the answer seems to be coming too late:

Reply from 192.168.3.10: bytes=32 time<1ms TTL=255

Request timed out.Reply from 192.168.3.10: bytes=32 time<1ms TTL=255Request timed out.Reply from 192.168.3.10: bytes=32 time<1ms TTL=255Request timed out.Reply from 192.168.3.10: bytes=32 time<1ms TTL=255Request timed out.Reply from 192.168.3.10: bytes=32 time<1ms TTL=255Request timed out.Reply from 192.168.3.10: bytes=32 time<1ms TTL=255Request timed out.

If I send big chunks of data, that fill up the receive buffer (my PBUF_POOL_SIZE is set to 592) the response come immediately (also when I keep sending the data ping arrives every time). To me its seems like the LWIP stack waits for some time to fill up the buffer and if nothing arrives for a certain time (which seems to be quite long to me) then he processes the buffer contents. Is there some configuration option for the LWIP / ETH where this behavior (timeout?!) can be controlled? It would need my device to react on a few ms basis, rather than ~1s one.

Thanks,

Jeremi

[Jeremi Wójcicki · Accepted solution]

Posted on January 03, 2018 at 15:29

After quite some time of experimentation I discovered that the issue is in the ethernetif.c driver generated by cubeMX (thanks to the 

http://lwip.100.n7.nabble.com/STM32-RBUS-Receive-Buffer-Unavailable-bit-set-after-debugger-break-td11154.html

  thread, where they reported similar problem however due to a different reason, but the same file). Here I post my solution (even though I did not go too deeply in understanding what the problem exactly was it seem to be related to the location of the DMA descriptiors in the memory - SRAM1 or SRAM2 banks) so if sb would bump into a similar problem in the future, perhaps it will save him some time.

Comparing ethernetif.c from cubeMX and from the firmware library I noticed that this part was missing:

/*

@Note: The DMARxDscrTab and DMATxDscrTab must be declared in a non cacheable memory region In this example they are declared in the first 256 Byte of SRAM1 memory, so this memory region is configured by MPU as a device memory (please refer to MPU_Config() in main.c).

In this example the ETH buffers are located in the SRAM2 memory,

since the data cache is enabled, so cache maintenance operations are mandatory. */#if defined ( __CC_ARM )ETH_DMADescTypeDef DMARxDscrTab[ETH_RXBUFNB] __attribute__((at(0x20010000)));/* Ethernet Rx MA Descriptor */

ETH_DMADescTypeDef DMATxDscrTab[ETH_TXBUFNB] __attribute__((at(0x20010080)));/* Ethernet Tx DMA Descriptor */

uint8_t Rx_Buff[ETH_RXBUFNB][ETH_RX_BUF_SIZE] __attribute__((at(0x2004C000))); /* Ethernet Receive Buffer */

uint8_t Tx_Buff[ETH_TXBUFNB][ETH_TX_BUF_SIZE] __attribute__((at(0x2004D7D0))); /* Ethernet Transmit Buffer */

#elif defined ( __ICCARM__ ) /*!< IAR Compiler */

#pragma data_alignment=4

#pragma location=0x20010000

__no_init ETH_DMADescTypeDef DMARxDscrTab[ETH_RXBUFNB];/* Ethernet Rx MA Descriptor */

#pragma location=0x20010080

__no_init ETH_DMADescTypeDef DMATxDscrTab[ETH_TXBUFNB];/* Ethernet Tx DMA Descriptor */

#pragma location=0x2004C000

__no_init uint8_t Rx_Buff[ETH_RXBUFNB][ETH_RX_BUF_SIZE]; /* Ethernet Receive Buffer */

#pragma location=0x2004D7D0

__no_init uint8_t Tx_Buff[ETH_TXBUFNB][ETH_TX_BUF_SIZE]; /* Ethernet Transmit Buffer */

#elif defined ( __GNUC__ ) /*!< GNU Compiler */

ETH_DMADescTypeDef DMARxDscrTab[ETH_RXBUFNB] __attribute__((section('.RxDecripSection')));/* Ethernet Rx MA Descriptor */

ETH_DMADescTypeDef DMATxDscrTab[ETH_TXBUFNB] __attribute__((section('.TxDescripSection')));/* Ethernet Tx DMA Descriptor */

uint8_t Rx_Buff[ETH_RXBUFNB][ETH_RX_BUF_SIZE] __attribute__((section('.RxarraySection'))); /* Ethernet Receive Buffer */

uint8_t Tx_Buff[ETH_TXBUFNB][ETH_TX_BUF_SIZE] __attribute__((section('.TxarraySection'))); /* Ethernet Transmit Buffer */

#endif

Adding it to my ethernetif.c fixed the problem of lost packets!

[Terje K Wahl]

Posted on February 06, 2018 at 11:04

Thank you for posting your solution. I had the exact same problem, and this fixed it.

[Simon schmid1]

Posted on June 06, 2018 at 10:12

Thank you, had the same problem ! Nucleo-F746ZG and CubeMX.

Lots of TCP Out-of-Order and Dup-ACK.

Now wireshark notices no Errors anymore. 

[Imen.D]

Posted on June 20, 2018 at 17:54

Hello

Wojcicki.Jeremi

,

Unfortunately, we didn't reproduce the issue after check by our team with LwIP example and relocated ETH buffers and descriptors in DTCM-RAM.

It seems your code misses the relocation of ETH descriptor from DTCM-RAM to SRAM1/SRAM2. Because if this relocation is not done, the linker will locate them at the default memory location which is DTCM-RAM.

So, in your case the default config is ETH buffers and descriptors which are located in DTCM-RAM, and you relocated them in SRAM1/SRAM2.

In fact,CubeMx doesn't relocate these buffers to SRAM1/SRAM2, and when generated a code with CubeMx, the ETH buffers and descriptors are located in DTCM-RAM.

With Regards,

Imen.

[sanjeev majumdar]

hi

i am also facing the same issue of ping timeout even after moving the ethernet buf & descriptors to SRAM1/SRAM2. the example works f9 in stm32f769 eval board but i observe randon ping loss in custom board having same processor.

[sw]

Jeremi, thanks a lot for this post! We experienced the same problem here with CubeMX generated code on a custom board that uses STM32F777VI. I would have never imagined that it's the ethernet driver causing these awfully slow response times. With your solution it works like a charm.

[allard]

This is still broken in 2020!!

This fix still works, if it doesn't make sure the right area's are added to MPU_Config:

 /* Configure the MPU as Normal Non Cacheable for Ethernet Buffers in the SRAM2 */
  MPU_InitStruct.Enable = MPU_REGION_ENABLE;
  MPU_InitStruct.BaseAddress = 0x2004C000;
  MPU_InitStruct.Size = MPU_REGION_SIZE_16KB;
  MPU_InitStruct.AccessPermission = MPU_REGION_FULL_ACCESS;
  MPU_InitStruct.IsBufferable = MPU_ACCESS_NOT_BUFFERABLE;
  MPU_InitStruct.IsCacheable = MPU_ACCESS_NOT_CACHEABLE;
  MPU_InitStruct.IsShareable = MPU_ACCESS_SHAREABLE;
  MPU_InitStruct.Number = MPU_REGION_NUMBER0;
  MPU_InitStruct.TypeExtField = MPU_TEX_LEVEL1;
  MPU_InitStruct.SubRegionDisable = 0x00;
  MPU_InitStruct.DisableExec = MPU_INSTRUCTION_ACCESS_ENABLE;
 
  HAL_MPU_ConfigRegion(&MPU_InitStruct);
  
  /* Configure the MPU as Device for Ethernet Descriptors in the SRAM2 */
  MPU_InitStruct.Enable = MPU_REGION_ENABLE;
  MPU_InitStruct.BaseAddress = 0x2004C000;
  MPU_InitStruct.Size = MPU_REGION_SIZE_256B;
  MPU_InitStruct.AccessPermission = MPU_REGION_FULL_ACCESS;
  MPU_InitStruct.IsBufferable = MPU_ACCESS_BUFFERABLE;
  MPU_InitStruct.IsCacheable = MPU_ACCESS_NOT_CACHEABLE;
  MPU_InitStruct.IsShareable = MPU_ACCESS_SHAREABLE;
  MPU_InitStruct.Number = MPU_REGION_NUMBER1;
  MPU_InitStruct.TypeExtField = MPU_TEX_LEVEL0;
  MPU_InitStruct.SubRegionDisable = 0x00;
  MPU_InitStruct.DisableExec = MPU_INSTRUCTION_ACCESS_ENABLE;
 
  HAL_MPU_ConfigRegion(&MPU_InitStruct);

In combination with(Keil):

ETH_DMADescTypeDef  DMARxDscrTab[ETH_RXBUFNB] __attribute__((at(0x2004C000)));/* Ethernet Rx DMA Descriptors */
 
ETH_DMADescTypeDef  DMATxDscrTab[ETH_TXBUFNB] __attribute__((at(0x2004C080)));/* Ethernet Tx DMA Descriptors */
 
uint8_t Rx_Buff[ETH_RXBUFNB][ETH_RX_BUF_SIZE] __attribute__((at(0x2004C100))); /* Ethernet Receive Buffers */
 
uint8_t Tx_Buff[ETH_TXBUFNB][ETH_TX_BUF_SIZE] __attribute__((at(0x2004D8D0))); /* Ethernet Transmit Buffers */

[RKara.3]

Can you please show full MPU_Config functions? It does not work for me. What are other settings , like i.e. flash interface, art accelerator, instuction prefetch and CPU cache?