SPWF04S HTTP-POST is sent in chunked transfer but not ended as per spec with ''0'', Why????

[Jonathan Harding]

Posted on May 02, 2018 at 12:36

I am posting data up to a server using HTTP post, the module sends the data in chunked mode. It is a relatively short packet, so is sent in one transfer. However the module does not end the transfer by sending a last chunk character, I.e. '0' therefore leaving the server expecting more data.

The module then closes the connection with FIN-ACK, yet leaving the server expecting more data, so obviously the server responds with an error on every post. Rather annoying!

Please fix or suggest how the chunked transfer can terminated correctly!

FYI:

3.6.1 Chunked Transfer Coding     The chunked encoding modifies the body of a message in order to    transfer it as a series of chunks, each with its own size indicator,    followed by an OPTIONAL trailer containing entity-header fields. This    allows dynamically produced content to be transferred along with the    information necessary for the recipient to verify that it has    received the full message.         Chunked-Body   = *chunk                         last-chunk                         trailer                         CRLF         chunk          = chunk-size [ chunk-extension ] CRLF                         chunk-data CRLF        chunk-size     = 1*HEX        last-chunk     = 1*('0') [ chunk-extension ] CRLF         chunk-extension= *( ';' chunk-ext-name [ '=' chunk-ext-val ] )        chunk-ext-name = token        chunk-ext-val  = token | quoted-string        chunk-data     = chunk-size(OCTET)        trailer        = *(entity-header CRLF)     The chunk-size field is a string of hex digits indicating the size of    the chunk. The chunked encoding is ended by any chunk whose size is    zero, followed by the trailer, which is terminated by an empty line.     The trailer allows the sender to include additional HTTP header    fields at the end of the message. The Trailer header field can be    used to indicate which header fields are included in a trailer (see    section 14.40).

[Elio Cometti]

Posted on May 02, 2018 at 18:37

Hello,

this is surprising to me...

However, I just tried to POST a form from a file containing 20 characters (direction=up&counter=2).

The command I used to send the form to my POST test server is:

AT+S.HTTPPOST=192.168.1.112,/post,5000,,,,,f.form

while the data captured by wireshark is:

00000000  50 4f 53 54 20 2f 70 6f  73 74 20 48 54 54 50 2f POST /po st HTTP/

00000010  31 2e 31 0d 0a 43 6f 6e  74 65 6e 74 2d 54 79 70 1.1..Con tent-Typ00000020  65 3a 20 61 70 70 6c 69  63 61 74 69 6f 6e 2f 78 e: appli cation/x00000030  2d 77 77 77 2d 66 6f 72  6d 2d 75 72 6c 65 6e 63 -www-for m-urlenc00000040  6f 64 65 64 0d 0a 48 6f  73 74 3a 20 31 39 32 2e oded..Ho st: 192.00000050  31 36 38 2e 31 2e 31 31  32 3a 35 30 30 30 0d 0a 168.1.11 2:5000..00000060  54 72 61 6e 73 66 65 72  2d 45 6e 63 6f 64 69 6e Transfer -Encodin00000070  67 3a 20 63 68 75 6e 6b  65 64 0d 0a 55 73 65 72 g: chunk ed..User00000080  2d 41 67 65 6e 74 3a 20  48 54 54 50 2d 43 6c 69 -Agent:  HTTP-Cli00000090  65 6e 74 0d 0a 0d                                ent...00000096  0a                                               .00000097  31 34 0d 0a                                      14..0000009B  64 69 72 65 63 74 69 6f  6e 3d 75 70 26 63 6f 75 directio n=up&cou000000AB  6e 74 3d 32                                      nt=2000000AF  0d 0a                                            ..000000B1  30 0d 0a 0d 0a                                   0....

then the server reply follows and the TCP connection is eventually closed by the SPWF04.

[Jonathan Harding]

Posted on May 02, 2018 at 18:57

Hi Elio

Thanks for looking into this for me.

I have just repeated your findings and the terminator was present and CORRECT.

It seems certain content types are not sending the terminator. Up to press I have tried:

.html - NO Terminator

.fhtml - NO Terminator

.json - Terminator CORRECT

.form - Terminator CORRECT

I'm not sure about the others, but I think I can get around it by changing my content type now.

Thanks

Jonathan

[Jonathan Harding]

Posted on May 03, 2018 at 15:39

Hi Elio

I can repeat it with several strings but below is a direct copy of the serial strings sent to the module.

Note, I have changed the delimiter to $ so as to support json format.

\' is just ' , the below is formatted to be put straight into a string with sprintf.

This fails:

AT+S.FSD=2:\\Upload.html\r

AT+S.FSC=2:\\Upload.html$108\r{\'uid\':\'230025001351363035343137\',\'model\':5000,\'token\':\'16293677d61b4b06face4c6db2de20efbb6d6188b8cd556613\'}\r\rAT+S.HTTPPOST=xxxxxxxxxx.westeurope.cloudapp.azure.com$/device/get_data_device/$3002$$$$$2:\\Upload.html\r

This works:

AT+S.FSD=2:\\Upload.json\r

AT+S.FSC=2:\\Upload.json$108\r{\'uid\':\'230025001351363035343137\',\'model\':5000,\'token\':\'16293677d61b4b06face4c6db2de20efbb6d6188b8cd556613\'}\r\rAT+S.HTTPPOST=xxxxxxxxxx.westeurope.cloudapp.azure.com$/device/get_data_device/$3002$$$$$2:\\Upload.json\r

I also use wire shark and the terminator is clearly present when using json but NOT with html.

baud 115200, and no delay between sending the strings.

I don't believe anything else has changed from default.

Jonathan

[Elio Cometti]

Posted on May 03, 2018 at 14:28

Hello Jonathan,

maybe I'm not catching the point... I have saved the string abobe in file with several extensions (html, fhtml, jpg, ...): the Content-Type header is changed accordingly and the trailing 30 0d 0a 0d 0a is always present.

Am I doing the same as you?

Regards,

Elio