STM32WB55 gets stuck in UTIL_SEQ_WaitEvt() function

[tcpctrnk]

Hello, I'm working on a project started from Zigbee_APS_coord and Zigbee_APS_router examples on NUCLEO-WB55.USBDongle with STM32Cube_FW_WB_V1.21.0.

The project is based on data exchange between the coordinator and several devices acting as routers (from 2 to 10) at the APS layer. The protocol for data transmission is based on coordinator data request which wait for a router data response before passing to the next device; payload size is 66 bytes and polling rate between two coordinator requests has been set in different trials between 100 and 150 ms. 

We observed that the average response time for a single request from the time ZbApseDataReqCallback has been called to the time the incoming zigbee message is detected with the callback registered using ZbApsFilterEndpointAdd is approximately 80 ms, except for times in which some error on communication is detected (usually 0xd1 aka routing error, solved with rejoin procedure). The code snippet used to send a zigbee message on both coordinator and routers is the following:

    struct ZbApsdeDataReqT zb_aps_req_current;

    memset(&zb_aps_req_current, 0, sizeof(zb_aps_req_current));
    zb_aps_req_current.dst = *dst;
    zb_aps_req_current.profileId = profileId; 
    zb_aps_req_current.clusterId = clusterId; 
    zb_aps_req_current.srcEndpt = srcEndpt; 
    zb_aps_req_current.asdu = payload;
    zb_aps_req_current.asduLength = payloadLength;
    zb_aps_req_current.asduHandle = zigbee_app_info.aps_info.asduHandle++;
    zb_aps_req_current.txOptions |= ZB_APSDE_DATAREQ_TXOPTIONS_SECURITY;
    zb_aps_req_current.txOptions |= ZB_APSDE_DATAREQ_TXOPTIONS_ACK;

    status = ZbApsdeDataReqCallback(zigbee_app_info.zb, &zb_aps_req_current, callback, arg);

The firmware exploits the sequencer for a total of 11 tasks (including the one already present in the examples), no events have been added except the ones for zigbee stack management already included in the examples.

During normal operation and without any particular error occurrence or delay-from-start relation among different trials, the coordinator device gets stuck in UTIL_SEQ_WaitEvt() function with event being waited = EVENT_ACK_FROM_M0_EVT. This problem has been observed on different boards.

We can't reproduce this behavior at will, so the first question would be any advice on which could be the cause of the problem to define a debug strategy.

Thank you for the support.

[Ouadi]

Hello @tcpctrnk,

ZbApsdeDataReqCallback command is sent from M4 to M0 CPU through the IPCC mechanism with a system of Acknowledge, when using the polling with short period without waiting for the callback confirmation, that may have a bad effect on the task processing. 

I recommend highly to reorder your data request transmission by waiting for APSDE Data Confirmation callback before sending the next request. 

kind regards,

Ouadi

[tcpctrnk]

Thanks @Ouadi, I'll follow your recommendation and point back the results after some tests.

[tcpctrnk]

Hello @Ouadi ,

After using a flag in order to prevent new messages to be sent if the previous one hasn't been processed yet the problem still occurs.

I'll run more tests to study the problem, meanwhile do you have any other suggestion?

Also I can provide more project details if you need them.

Thank you.

[Ouadi]

Hi @tcpctrnk

Could you please provide the app_zigbee.c file and the source code that manages the transmission polling to review the implementation ? 

Regards,

Ouadi

[tcpctrnk]

Hi @Ouadi,

The project started from Zigbee_APS_coord and Zigbee_OnOff_Coord_NVM, but the project structure evolved to manage both coordinator and router code into one project (using a define to change included files). Therefore the project structure is not 1:1 with the examples. 

I'm providing samples for the coordinator side. A summary of the FW architecture is necessary as the project is still in a fast development phase and its not well documented:

1. Coordinator is controlled via UART, so it executes actions according to the command received from a master, in particular it processes data in `uart_rx_cmd_handler()` when available. After the command exchange for network startup, UART interface is used to exchange data with routers using ZB_CMD_READ_ID_STAT and ZB_CMD_WRITE_ID_OUT commands ID, making WB55 act almost as a UART-ZIGBEE bridge.
2. Besides the commands received on UART RX, coordinator monitors routers' state periodically, to communicate with the master if any problem occurred. This code has been developed very fast and it may be a little bit confusing, I'm working on replace this 'algorithm' with monitoring of the indication provided by:

ZbMsgFilterRegister(zigbee_app_info.zb, ZB_MSG_FILTER_NLME, ZB_MSG_INTERNAL_PRIO + 1, app_zb_msg_filter_cb, NULL);

However the problem described in the issue was present even before point 2. was implemented, so in my opinion the root cause is not directly related to the code composing point 2.

I'm available for any doubts or any other information which might be needed.

Thank you for your support.

[Ouadi]

Hello @tcpctrnk,

Thanks for the update,

After reviewing the code, the implementation of ZbApsdeDataReqCallback looks correct to me, now I suspect a timing management issue when performing other tasks as the permit join and Scan update periodically. 

To progress with the analysis, a complete project would be appreciated as with the provided files are missing some dependencies..

Other point, Are you able to reproduce the issue with only 2 boards  coordinator + Router, by sending the ZbApsdeDataReqCallback continiously to the same device instead of 10 ? 

Please provide also some additionnal inputs as the debug callstack and the logs.

Best regards,

Ouadi

[tcpctrnk]

Hi @Ouadi, 

Due to project constraints we decided to refactor the WB55 firmware to be lighter, this way the amount of tasks executed by the microcontroller should be reduced. I'll get back if the problem will still occur.

Thank you.

[tcpctrnk]

Hi @Ouadi,

During some tests with a lighter version of the firmware I encountered this case:

---

Other point, Are you able to reproduce the issue with only 2 boards  coordinator + Router, by sending the ZbApsdeDataReqCallback continiously to the same device instead of 10 ? 

---

So, the problem still persists, even though the scheduler seems to resume its functionality (after 1 or 2 seconds) in some cases.

I can share with you the project and a python script to test it with WB55.USBDongle and FT232 UART driver if you could provide an email.

Thank you!