POST large data with a single request

[Luca DC]

Posted on February 01, 2017 at 16:06

Hi,

I'm using a SPWF01SA1 module to send data from my application to a central server using POST requests.

I need to transfer 64 kByte of binary data (application/octet-stream) using a single POST request as I want the destination server to receive a single transfer command which must be validated and confirmed back for correct reception.

I've seen that the command AT+S.HTTPPOST has limitations on maximum length so I cannot use it.

Then I've upgraded the firmware to version 3.5 to try command AT+S.HTTPREQ, build my own POST header and be able to send data after the command.

In this way I've managed to send up to something less than 12 kByte. During transfer, the RTS signal is raised every 1024 byte so I inserted a pause to let the buffer flush. No matter how long I wait between the HTTPREQ command and the data (should I wait?) or between RTS signals (all of which stay deasserted for a fixed time of less than 3.5 ms and I've raised my pause time up to 1 s): when the data amount is 11460 bytes or more the packet won't reach its destination while with 11440 bytes everything works fine.

My current destination for testing is posttestserver.com, where I've found stored messages of several kBytes (even one slightly less than 1 MByte) so I assume that there's not a limitation on that side.

When the transmission fails the SPWF01SA1 doesn't say anything, I simply don't get the expected reply from the server.

I've also tried to split my single POST request into a sequence of AT+S.HTTPREQ commands (one for the POST header and one for just 32 bytes of data) but I couldn't make it work. Hence I assume that this splitting is not possible.

My question: is there a way with this module to send a 64 kByte block of binary data using a single POST request?

What is the recommended way to transfer large amounts of data?

Thank you.

#http-post

[Gerardo GALLUCCI · Accepted solution]

Posted on February 01, 2017 at 18:19

Ciao Luca,

I think you are facing a low RAM problem. There are not 64Kb available for a single AT command (GET, POST, REQ, or other). You need to use a lower buffer size, so, use sockets.

HTTP commands perform single operation into 3 steps:

- connect

- single write

- disconnect

You can switch to:

- SOCKON

- SOCKWRITE #1

- ...

- SOCKWRITE #n

- SOCKC

Let me know if it works (I expect so...)

BR

jerry

[Luca DC]

Posted on February 01, 2017 at 19:11

Ciao Jerry,

ok, I see that using sockets is the only way, as you say. I've just managed to transfer 16 kByte so this seems to be the solution.

I just hoped there was an easier way.

Thank you.

Luca

[Luca DC]

Posted on February 02, 2017 at 13:13

Ciao Jerry,

I'd like to ask you a clarification about the RTS signal.

I've implemented the transmission of my data using a socket. The data is split in 4 kByte chunks and sent with a sequence of AT+S.SOCKW commands. Everything is working fine.

What I see is that during each 4 KByte transmission the RTS signal is raised every 1024 bytes for ~3.5 ms. Even if I completely ignore it, all data is correctly acquired. The UART speed is the default 115200 baud.

I'd like to know if I can safely ignore this RTS glitch. I can't understand its purpose if ignoring it causes no harm.

My guess is that as soon as the UART completes the 1024th byte reception the internal 1 kByte buffer is full, RTS is raised (deasserted) and the data is transferred from the internal buffer somewhere else (another internal buffer?) starting from the first byte. The speed of the internal transfer is such that when the UART gets the 1025th byte (~87 µs after), the first bytes of the buffer have already been freed so the incoming new bytes can be safely stored. Then eventually, when the original first 1024 bytes have been transferred (~3.5 ms after), the RTS is asserted again.

Am I right? Can I assume this behavior is going to be consistent so if the UART speed is not too high the RTS signal can be ignored?

This is an important point for me because my MCU is powered at 5V and the RTS signal high level (2.5 V) is too low to be seen without a level shift. It's a 5V tolerant pin such that with a pull-up it goes up to 5V when it's not in use, but when it's working it goes from 0V to 2.5V. Bad it's not an open collector (and the same is true for the TX pin: 5V tolerant but push-pull so additional circuitry is needed). If I can safely ignore it, it would be better.

Thank you.

Luca

[Gerardo GALLUCCI]

Posted on February 03, 2017 at 10:05

Ciao Luca,

you can safely ignore RTS, but be sure to handle properly module's OK/ERROR before next SOCKW.

RTS is really mandatory in data_mode, since you have no feedback (OK/ERROR) on data transfer.

Ciao

j

[Luca DC]

Posted on February 03, 2017 at 16:22

Hi jerry,

fine, thank you very much for this good information.

Regards.

Luca