Skip to main content
Alec Davis 2021
Associate III
December 11, 2023
Question

AzureRTOS netxduo unable to download ~16K or larger image using web_http_client secure

  • December 11, 2023
  • 4 replies
  • 3372 views

Netxduo packet pool is using 51K, plenty I would have thought to download a 16K test file.

This problem was initially seen on an F417 board, and then an F429, so moved on to an H723, specifically the NUCLEO-H723ZG.

I filed a bug report at https://github.com/azure-rtos/netxduo/issues/177  but testing this with a Renesas Rx72N envision kit it works flawlessy, hence this seems like an ST problem.

 

Console output. 

After the first attempt to download the free_packets has been corrupted. normally free_packets is 4 packets less than total_packets.

Thus no further downloads happen.

 

 

 

 

 

 

MX_NetXDuo_Init: Start
MX_NetXDuo_Init: NX_APP_PACKET_POOL_SIZE = 51072
MX_NetXDuo_Init: DEFAULT_PAYLOAD_SIZE = 1536
MX_NetXDuo_Init: Started
STM32 IpAddress: 192.168.125.45
 download_test

http_web_request: Start httpbin.org

APP:before nx_dns_host_by_name_get total_packets = 32 free_packets = 28 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0

ERROR: httpbin.org nx_dns_host_by_name_get status=[0xa3]

ERROR: download_test: status =[0xa3]
download_test

http_web_request: Start httpbin.org

APP:before nx_dns_host_by_name_get total_packets = 32 free_packets = 28 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0
Client connected to HTTP server: [3.218.223.42]

nx_web_http_client_created

nx_web_http_client_response_header_callback_set
tls_metadata_size mismatch! calculated:10128 actual:18432

nx_web_http_client{_secure]_connect

http client request initialized

nx_web_http_client_request_send
Received header:
 Field name: Date (4 bytes)
 Value: Mon, 11 Dec 2023 06:20:30 GMT (29 bytes)

Received header:
 Field name: Content-Type (12 bytes)
 Value: application/octet-stream (24 bytes)

Received header:
 Field name: Content-Length (14 bytes)
 Value: 16384 (5 bytes)

Received header:
 Field name: Connection (10 bytes)
 Value: keep-alive (10 bytes)

Received header:
 Field name: Server (6 bytes)
 Value: gunicorn/19.9.0 (15 bytes)

Received header:
 Field name: Access-Control-Allow-Origin (27 bytes)
 Value: * (1 bytes)

Received header:
 Field name: Access-Control-Allow-Credentials (32 bytes)
 Value: true (4 bytes)

DONE: firmware=1 total bytes=0 errors 0
Error with http response body get: get_status=0x1

APP:Error response body get total_packets = 32 free_packets = 9 empty_pool_requests = 1530 empty_pool_suspensions = 5 invalid_packet_releases = 0

APP:before nx_web_http_client_delete total_packets = 32 free_packets = 9 empty_pool_requests = 1530 empty_pool_suspensions = 5 invalid_packet_releases = 0

APP:after nx_web_http_client_delete total_packets = 32 free_packets = 32 empty_pool_requests = 1530 empty_pool_suspensions = 5 invalid_packet_releases = 0
download_test

http_web_request: Start httpbin.org

APP:before nx_dns_host_by_name_get total_packets = 32 free_packets = 32 empty_pool_requests = 1530 empty_pool_suspensions = 5 invalid_packet_releases = 0

ERROR: httpbin.org nx_dns_host_by_name_get status=[0xa3]

ERROR: download_test: status =[0xa3]
download_test

http_web_request: Start httpbin.org

APP:before nx_dns_host_by_name_get total_packets = 32 free_packets = 32 empty_pool_requests = 1530 empty_pool_suspensions = 5 invalid_packet_releases = 0

ERROR: httpbin.org nx_dns_host_by_name_get status=[0xa3]

ERROR: download_test: status =[0xa3]
download_test

 

 

 

 

 

 

 

Attached project files "nx_httpbin.org.7z" and wireshark capture "stm32h7_httpbin_org_NX_PACKET_POOL_SIZE_32_test-16k.7z" with packet pool count set to 32 and attempting to download 16384 bytes.

 

Changing to 8192 bytes works fine as below, wireshark file "stm32h7_httpbin_org_NX_PACKET_POOL_SIZE_32_test-8k.7z" attached.

 

 

 

 

 

MX_NetXDuo_Init: Start
MX_NetXDuo_Init: NX_APP_PACKET_POOL_SIZE = 51072
MX_NetXDuo_Init: DEFAULT_PAYLOAD_SIZE = 1536
MX_NetXDuo_Init: Started
STM32 IpAddress: 192.168.125.45
 download_test

http_web_request: Start httpbin.org

APP:before nx_dns_host_by_name_get total_packets = 32 free_packets = 28 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0

ERROR: httpbin.org nx_dns_host_by_name_get status=[0xa3]

ERROR: download_test: status =[0xa3]
download_test

http_web_request: Start httpbin.org

APP:before nx_dns_host_by_name_get total_packets = 32 free_packets = 28 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0
Client connected to HTTP server: [75.101.131.185]

nx_web_http_client_created

nx_web_http_client_response_header_callback_set
tls_metadata_size mismatch! calculated:10128 actual:18432

nx_web_http_client{_secure]_connect

http client request initialized

nx_web_http_client_request_send
Received header:
 Field name: Date (4 bytes)
 Value: Mon, 11 Dec 2023 06:49:36 GMT (29 bytes)

Received header:
 Field name: Content-Type (12 bytes)
 Value: application/octet-stream (24 bytes)

Received header:
 Field name: Content-Length (14 bytes)
 Value: 8192 (4 bytes)

Received header:
 Field name: Connection (10 bytes)
 Value: keep-alive (10 bytes)

Received header:
 Field name: Server (6 bytes)
 Value: gunicorn/19.9.0 (15 bytes)

Received header:
 Field name: Access-Control-Allow-Origin (27 bytes)
 Value: * (1 bytes)

Received header:
 Field name: Access-Control-Allow-Credentials (32 bytes)
 Value: true (4 bytes)

DONE: firmware=1 total bytes=0 errors 0
DONE: firmware=1 total bytes=8192 errors 0

APP:before nx_web_http_client_delete total_packets = 32 free_packets = 28 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0

APP:after nx_web_http_client_delete total_packets = 32 free_packets = 27 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0
download_test

http_web_request: Start httpbin.org

APP:before nx_dns_host_by_name_get total_packets = 32 free_packets = 28 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0
Client connected to HTTP server: [75.101.131.185]

nx_web_http_client_created

nx_web_http_client_response_header_callback_set
tls_metadata_size mismatch! calculated:10128 actual:18432

nx_web_http_client{_secure]_connect

http client request initialized

nx_web_http_client_request_send
Received header:
 Field name: Date (4 bytes)
 Value: Mon, 11 Dec 2023 06:49:50 GMT (29 bytes)

Received header:
 Field name: Content-Type (12 bytes)
 Value: application/octet-stream (24 bytes)

Received header:
 Field name: Content-Length (14 bytes)
 Value: 8192 (4 bytes)

Received header:
 Field name: Connection (10 bytes)
 Value: keep-alive (10 bytes)

Received header:
 Field name: Server (6 bytes)
 Value: gunicorn/19.9.0 (15 bytes)

Received header:
 Field name: Access-Control-Allow-Origin (27 bytes)
 Value: * (1 bytes)

Received header:
 Field name: Access-Control-Allow-Credentials (32 bytes)
 Value: true (4 bytes)

DONE: firmware=1 total bytes=8192 errors 0

APP:before nx_web_http_client_delete total_packets = 32 free_packets = 28 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0

APP:after nx_web_http_client_delete total_packets = 32 free_packets = 27 empty_pool_requests = 0 empty_pool_suspensions = 0 invalid_packet_releases = 0

 

 

 

 

 

 

 

4 replies

Alec Davis 2021
Associate III
December 13, 2023

typo in file nx_stm32_eth_driver.c  function nx_stm32_eth_driver()

default:


/* Invalid driver request. */

/* Return the unhandled command status. */
driver_req_ptr -> nx_ip_driver_status = NX_UNHANDLED_COMMAND;

/* Default to successful return. */
driver_req_ptr -> nx_ip_driver_status = NX_DRIVER_ERROR;
}

 

It should surely should have been 

 default:


 /* Invalid driver request. */

 /* Return the unhandled command status. */
 driver_req_ptr -> nx_ip_driver_status = NX_UNHANDLED_COMMAND;

 break;
 }

 

Alec Davis 2021
Associate III
January 25, 2024

@MSG_ST

Is anyone able to verify the issue that netxduo cannot download large payloads when using nx_web_http_client_secure_connect and nx_web_http_client_response_body_get

 

The symptom is that part way through the download the nx_stm32_driver corrupts the packet_pool. Seems like Microsoft/express_logic wrote the driver based on the copyright.

 

I have provided a full example for the H723 named nx_httpbin.org.7z

Any help would be appreciated

 

Thanks

JGUIT.1
Associate II
March 20, 2024

Hello Alec

 

I find your post because I have a similar issue but on another board (b-l4s5i-iot01a) : impossible to download large file (I'm testing with 38kB). Seems on my side IP stack is stuck somewhere....

 

Did you get some help on this topic ? Have you found related information that may help ?

 

Thanks

Joel

Alec Davis 2021
Associate III
March 21, 2024

Hi Joel

You are the first to acknowledge there may be a problem.

 Just to be sure we have similar issue, I was able to use http (non secure) and download multi MB files but not with https (secure). IIRC if I assigned a huge amount of NetXDuo memory pool size or increased number of packets to 50 then secure would work. In otherwords the memory pool wasn't being exhausted.

 

@B.Montanari 

Maybe https://community.st.com/t5/stm32-mcus/how-to-implement-a-webserver-in-stm32-using-netxduo-ethernet/ta-p/641982 could be good starting point to experiment with downloading, but the linked project requires an enormousd amount of NetXDuo memory pool size @ 1536 * 100, that's 150KB of valuable RAM.

 

Alec

 

 

 

 

 

JGUIT.1
Associate II
March 21, 2024

Thanks for this first feedback Alec! 

I cannot test with http easily now so I do https only. I do small get, put and post with no problem (up to 1KB tested ok).

I have a "small" memory pool for the moment, I should try to increase probably to check (already tried to increase stack sizes, play with thread priorities etc)

150KB is huge yes. But now this remind me a question I had several months ago on a totally different configuration, on zephyr, where I needed to increase the heap size of mbedtls a lot, up to 100KB: https://github.com/zephyrproject-rtos/zephyr/discussions/65093 so maybe it s required but I still consider this is a lot of RAM ....

I will check also my netxduo driver, I m using the one from Microsoft, I need to test the one from ST. I will probably also test my application on a different nucleo board with Ethernet instead of wifi, to compare results.

We keep in touch here!

Joel

JGUIT.1
Associate II
March 21, 2024

One more point: are you using the threadx and netxduo port from ST or the mainline repositories from eclipse-threadx ? I m using the mainline repositories on my side, version 6.4.0.

Alec Davis 2021
Associate III
March 26, 2024

With the H7 I was initially testing with 6.2.1 from ST, then Microsoft suggested I try 6.3.0, I had to copy mailine netxduo github code over existing files, refer  Microsoft: Could you update your source code to 6.3.0 

So no I haven't tried 6.4.0

 

ST has not updated the packs for F4 F7 read the following thread.

Please advise when azrtos packs for F4 F7 and H7 a... - STMicroelectronics Community 

response from ST: I can confirm that X-CUBE-AZRTOS-H7 and X-CUBE-AZRTOS-F4 will be updated to support AzureRTOS 6.3.0 but not in the near future

I have no idea what ST's roadmap is.

ST has abandoned LWIP. Azrtos/ThreadX seems to be going down the same path.

threadx.io commits so far are mainly text changes from Azure RTOS to Threadx.

 

 

 

 

 

JGUIT.1
Associate II
April 8, 2024

Hello Alec

 

For your information I managed to get it working here with my B-L4S5I-IOT01A board. Downloading a 1MB file from an HTTPS server.

 

The modifications I made:

- netxduo pool size increased a lot, I haven't tried to optimize for the moment. Current definitions are the following:

 

 

#define NETX_PACKET_COUNT 128
#define NETX_PACKET_SIZE 1200 // Set the default value to 1200 since WIFI payload size (ES_WIFI_PAYLOAD_SIZE) is 1200
#define NETX_POOL_SIZE ((NETX_PACKET_SIZE + sizeof(NX_PACKET)) * NETX_PACKET_COUNT)

 

 
- Update of the firmware in my wifi module (specific to my board)
- Handling of errors currently very basic (need to improve) as discussed on github https://github.com/eclipse-threadx/netxduo/issues/261
 
Looks fine for me :)
As a remember, I'm using threadx and netxduo 6.4.0 from https://github.com/eclipse-threadx
 
Hope you will find solutions for you!
 
Joel