STM32WB: Thread network goes down after a few minutes in BLE Thread Dynamic Mode but BLE stays active and works fine.

[Patrick Chwalek]

I created a custom application using the code from the BLE_THREAD_DYN example and the v10.0 FW BLE_THREAD_DYN stack. The BLE works fine but the thread stack goes down after a few minutes. The main differences between my code and the example is that I converted the code to use FreeRTOS but I preserved most of the logic and structure.

Upon initial system power on, the stacks both initialize and go up just fine with the device meshing with my existing thread network. However, after a few minutes the device disconnects from the thread network and the node never reconnects but the BLE remains active.

I'm troubleshooting the issue but any breadcrumbs on how to converge on the problem more quickly would be much appreciated.

Update 3/3/2021 @ 1:50PM

Within the initialization of both stacks, I commented out the BLE initialization and only ran with the Thread stack active using the v10.0 BLE_THREAD_DYN firmware. The issue persisted with the Thread stack going down after a few minutes -- the amount of minutes seems to be variable so can't seem to determine a pattern.

static void APPE_SysEvtReadyProcessing(void) {
	/* Traces channel initialization */
	TL_TRACES_Init();
 
//	APP_BLE_Init_Dyn_1();
	APP_THREAD_Init_Dyn_1();
 
//	APP_BLE_Init_Dyn_2();
	APP_THREAD_Init_Dyn_2();
	UTIL_LPM_SetOffMode(1U << CFG_LPM_APP, UTIL_LPM_ENABLE);
}

Continuing with the above, I flashed the THREAD_FTD stack while preserving the code and keeping the BLE uninitialized. Over the course of 30 minutes, the issue did not persist which means to me something is wrong with the v10.0 BLE_THREAD_DYN firmware that causes the Thread stack to drop after a few minutes, regardless if the BLE stack is initialized.

Update 3/3/2021 @ 4:30PM

After 3 hours, the Thread stack is still up, meshing, and transmitting as expected using the THREAD_FTD stack. The issue seems to be with the BLE_THREAD_DYN firmware.

[Patrick Chwalek]

I brushed off this code (BLE/Thread dynamic mode with FreeRTOS) and migrated everything to support the latest firmware (17.3). I found in a separate thread that someone discovered the linker that cube generates isn't suitable for BLE/Thread dynamic applications (_estack = 0x20024000) so I also implemented that. I can get thread working reliably but for BLE, the device advertises but the services fail to be discovered. When I connect to the device via the ST BLE Toolbox app, services fail to be discovered and automatically disconnects after a few seconds. Once disconnected, the device fails to re-advertise.  The BLE stack works fine if I comment out APP_THREAD_Init_Dyn_2 function call in APPE_SysEvtReadyProcessing. However, the moment I enable Thread, the BLE stack has the failure condition described above. Any pointers to possible solutions would be much appreciated!

[Patrick Chwalek]

For additional context as I troubleshoot, 

• I modified the BLE_THREAD_DYN example (doesn't use FreeRTOS) for the WB55 Nucleo board with all of the same clock settings as my FreeRTOS-driven project. The BLE_THREAD_DYN example works just fine with those settings.
• My app_ble file is nearly identical to the BLE_HeartRateFreeRTOS project, taking special caution to making sure I have the same FreeRTOS function call architecture
• The app_thread file is similar to the Thread_SED_Coap_FreeRTOS example, taking special caution to make sure I have the same FreeRTOS function call architecture
• The linker file in my FreeRTOS-equipped project is the same as the linker used for BLE_THREAD_DYN

Given the above, I believe the issue is with some intermediary code that can mess up the concurrent running of code on the M0 that I am just not catching or there is some other issue where the shared SRAM2 is being corrupted. There is no comprehensive app note for the BLE/THREAD dynamic code so I'm shooting in the dark a bit on the problem. 

[Patrick Chwalek]

Found a function that I think matters. You need to fill out the below function found in app_thread.c to ensure no overlapping M0 commands.

void Pre_OtCmdProcessing(void)
{
//fill in code below to ensure no overlapping messages
}

 There really should be a guide on this from ST since it seems all the hooks are there to do the FreeRTOS conversion but its left to the user to figure it all out on their own.

I've been referencing example code from the OpenThread example that uses ThreadEx. I still can't get both to run reliably.

If OpenThread succeeds in initializing, the BLE stack stops running. 

Also, I just found a BLE+Matter example that does use FreeRTOS so I began studying that implementation: https://github.com/search?q=repo%3Astm32-hotspot%2Fstm32wb-matter-device-over-thread%20MO_NOTIF_QUEUE_SIZE&type=code

[Patrick Chwalek]

After some testing, I found that I can trigger BLE advertisements via a remote OpenThread device and a Coap server running on my problem device. The problem device is able to advertise but services fail to be discovered (i.e., BLE_UserEvtRx never gets called when OT is active (no IPCC RX interrupt)). Once connected to an iPhone, the device disconnects after a short time. I can successfully trigger additional BLE advertisements with the same failure conditions. I thought the issue was with aci/hci command collision but I have binary semaphores ensuring this doesn't happen (at BLE_StatusNot and at Pre_OtCmdProcessing). 

Current commit of the OT+BLE Concurrent FreeRTOS test: https://github.com/pchwalek/buzzcam/tree/ST_OT_TEST

[Patrick Chwalek]

Going through the M0 firmware revisions, I saw this comment for v17.1 which makes me think there might potentially be another uncaught bug. However, it's just speculation on my end but it's moments like these that I wish I had access to the source code running on the M0.  

• ID 151805 : Corrected an issue on radio access between BLE and Thread that could cause a freeze in the firmware.

[Patrick Chwalek]

I don't see any HW_IPCC_Tx_Handler() calls after both the BLE and OT stacks are activated.

I have a thread that attempts to do a BLE advertisement every 33 seconds. This works but once "connected" via the ST BLE Toolbox app, I don't see any characteristics and disconnect within seconds. 

Coap messages via OT come in just fine. 

I also found that the aci_gap_set_discoverable function yields BLE_STATUS_SUCCESS but the following aci_gap_update_adv_data yields BLE_STATUS_FAILED. For the non-FreeRTOS BLE_OT_DYN example, both yield BLE_STATUS_SUCCESS

/* Start Fast or Low Power Advertising */
  ret = aci_gap_set_discoverable(ADV_IND,
                                 Min_Inter,
                                 Max_Inter,
								 GAP_PUBLIC_ADDR,
                                 NO_WHITE_LIST_USE, /* use white list */
                                 sizeof(a_LocalName),
                                 (uint8_t*) &a_LocalName,
                                 BleApplicationContext.BleApplicationContext_legacy.advtServUUIDlen,
                                 BleApplicationContext.BleApplicationContext_legacy.advtServUUID,
                                 0,
                                 0);
  
  /* Update Advertising data */
  ret = aci_gap_update_adv_data(sizeof(a_ManufData), (uint8_t*) a_ManufData);

@Remi QUINTIN if you have any pointers to possible issues, that would be much appreciated! Thanks!

[Patrick Chwalek · Accepted solution]

It appears the below is required. The buffer is never released which causes the failure experienced in this thread. 

void TL_TRACES_EvtReceived( TL_EvtPacket_t * hcievt )
{
#if(CFG_DEBUG_TRACE != 0)
  /* Call write/print function using DMA from dbg_trace */
  /* - Cast to TL_AsynchEvt_t* to get "real" payload (without Sub Evt code 2bytes),
     - (-2) to size to remove Sub Evt Code */
  DbgTraceWrite(1U, (const unsigned char *) ((TL_AsynchEvt_t *)(hcievt->evtserial.evt.payload))->payload, hcievt->evtserial.evt.plen - 2U);
#endif /* CFG_DEBUG_TRACE */
  /* Release buffer */
  TL_MM_EvtDone( hcievt );
}

I have found this referenced in AN5289 (pg 141), albeit brief. Not sure why this couldn't be featured natively within CubeMX Code instead of having the bare function weakly defined in the API, requiring the user to write the above.