How to set "Don't fragment" flag LWIP short TCP paquets

[JMarq.1]

Hello,

I am sending very short TCP paquets (600 bytes maximum) in a PC <=> MCU transfer scenario. I wish the software running on the PC to answer using an empty (len=0) TCP flag ACK packet as soon as it gets every TCP packet.

Using an STM32F207ZG and LWIP stack on STM32CubeIDE, TCP packets are sent correctly but PC software answers an ACK after 50ms approximately. This is too much.

Using an MCU from another brand with a TCP/IP stack different than LWIP, I can send paquets setting "Don't fragment" flag and the receiving software on the PC answers every packet with an ACK in less than 10us.

How can I send packets on LWIP forcing to be set "Don't fragment" flag?

[Pavel A.]

Confused?  Maybe you meant nodelay? In LwIP this is called tcp_nagle_disable() 

[JMarq.1]

Hi @Pavel A. 

I forgot to mention that I am using RAW (no-RTOS). I already tried tcp_nagle_disable(); as pointed in (https://support.xilinx.com/s/question/0D52E00006hpkatSAA/lwip-with-fragmented-packets?language=en_US) and that didn't work to me, packets on Wireshark where shown with "Don't fragment" IP option set to 0. I also tried pcb->flags |= TF_NODELAY; prior to tcp_write() and it didn't work to me neither. I tried also playing with IP options "IP_FRAG" and "IP_REASSEMBLY", neither they worked.

In LWIP source code ip4_frag.c I read:

 

 

/**
 * The IP reassembly code currently has the following limitations:
 * - IP header options are not supported
 * - fragments must not overlap (e.g. due to different routes),
 * currently, overlapping or duplicate fragments are thrown away
 * if IP_REASS_CHECK_OVERLAP=1 (the default)!
 *
 * @todo: work with IP header options
 */

 

 

Does this mean that IP header options are not available? I hardly believe that.

I'm absolutely confused and lost. Sorry, for confusing you also.

Thanks

[LCE]

Maybe the time has come to get into the lwIP source and modify it.

Been there, done that. Not nice concerning future updates of lwIP, but possible.

So try to find out where and how the IP header is built and modify it.

[waclawek.jan]

This is a generic lwip question, rather than STM32-specific, so maybe better discussed in the lwip-users mailing list.

Search indicates, that the don't fragment flag used to be set by default around 2004 (which could probably be relatively simple to restore, as mentioned, in src/core/ipv4/ip.c ip_output_if(), where now is IPH_OFFSET_SET(iphdr, 0);) and here is a thread discussing to set it selectively (but that probably requires further insight to pull it off).

JW

[JMarq.1]

Thank you @waclawek.jan for pointing me to that. I changed it in ip4.c to IPH_OFFSET_SET(iphdr, PP_NTOHS(IP_DF)); IP frames now are being sent always with "Don't fragment" flag set. Nonetheless, PC software continues ACKing after 50ms for STM32 MCU instead of 10us on the other MCU. I'll look to other direction about what's going on.

Closing the post as it was about how to set this flag.

[LCE]

Concerning the 50 ms ACK delay, have you checked that the PHY and the STM's ETH MAC both are working in full-duplex mode?

There once was a bug in the HAL driver for the PHY LAN8742, so the PHY negotiated for full duplex, but the HAL read the wrong register and set the STM's MAC to half duplex, and that lead to a lot of strange "behavior"...

[JMarq.1]

Good point. I checked that and it's running at 100M/Full.

To my surprise, the old hardware with old MCU (the one that is communicating much faster) is 100M-Half !! (:O) I forced 100M-Half on the new hardware with STM32 and there is no difference, still 50ms there. So, in that case, half or full is the same behaviour.

[Pavel A.]

Do you see fragmentation in Wireshark?  With a direct connection between STM32 and the PC, it is unlikely that 600 bytes will be fragmented. A bug such as mentioned by @LCE  looks more likely

[JMarq.1]

No, packets are not being fragmented. I just was trying to find what the problem for this difference in ACKing interval might be because old MCU was sending all packets with "Don't fragment" flag set and new STM32 MCU wasn't. But, I was wrong, this is not explaining the issue.

I'm still trying to find differences to explain that.... looking into differences between wireshark captures from both MCU etc....

Thanks

[Pavel A.]

Then maybe you need a better network sniffer than wireshark. Something professional ( = expensive) that sees more low level (very short packets, bad crc or preamble...)  Poke in the PHY with a scope and look for anomalies.

[LCE]

Please post it here, when you find out what's going on.

Good hunting!

[Piranha]

It seems that the topic author has confused the IP fragmentation with TCP segments. As the TCP is a stream (not message) based protocol, it doesn't have a requirement for segment size to be related to the size of user provided data chunks. And there is no strictly defined behavior and timeouts to sending ACKs either. Even if you will find a pattern, which triggers a specific behavior on, for example, Windows PC, there is no guarantee that this holds true for MacOS, Linux or other systems. And there is no even guarantee that the same behavior holds true for the same system after some OS or driver update!

 

There once was a bug in the HAL driver for the PHY LAN8742

@LCE , why are you saying "once was"? For non-reworked drivers, which are still the only available ones for F1 and F2 series, it's still there, because ST doesn't care.

@Pavel A. , PHY and the physical layer in general has nothing to do with packet sizes and delays for a higher layers like TCP.

[Pavel A.]

> PHY and the physical layer in general has nothing to do with packet sizes

Hmm.... Things happen. Packet sizes translate to transmission times; the longer the time the more is the window for a glitch... or something like that.

[Piranha]

With such an approach the number and models of switches and even the wire length has to be "tested"... You mean the ST's broken bloatware freezing and corrupting and dropping frames? Yes, that is possible. Though typically such a behavior is more of an erratic type, but the reported behavior seems to be consistent. Most likely the "issue" is not a real issue, just the lack of understanding of TCP functionality.

@JMarq.1 , in English the "paquets" is spelled as "packets". ;)

[Pavel A.]

Sure, everything should be tested, and there are serious tools for this (from Chariot to automated logic analyzers, of course you know). No, I mean the external PHYs, their clocks, power supply schemes, shielding and so on.

[Piranha]

Such hardware issues result in dropped packets, which can be detected easily by a software test. Those test tools are necessary for those, who implement Ethernet hardware on ASIC or FPGA. No, I don't have or even know those tools, because I don't need those. Of course, I have a decent oscilloscope, but I haven't even used it as an aid in the development of Ethernet related code. While hardware issues are definitely possible, searching for some unlikely ones just because of a behavior of a clearly broken bloatware is just a waste of time.

Which means that absence of bugs can be reached only by this:

programs should be composed correctly, not just debugged into correctness

https://amturing.acm.org/award_winners/dijkstra_1053701.cfm