Trouble using the GestureKit ULD with the VL53L8CX on a STM32U575

[Jason Mount]

Hello, I am trying to get gesture detection (swipe UP, DOWN, LEFT, RIGHT) with the VL53L8CX working on an STM32U575 running an RTOS.  I currently have a X-NUCLEO-53L8A1 plugged into a NUCLEO-U575ZI-Q board running Segger embOS in a C++ project.  I have been able to successfully configure and collect ranging data from the ToF sensor by extracting and modifying the example code in en.STSW-IMG040.zip\VL53L8CX_ULD_driver_1.2.1\Examples\ and using HAL_I2C_Mem_Write() and HAL_I2C_Mem_Read() for the implementation for the platform WrXxx() and RdXxx() functions.
However, when I try to use the GestureKit ULD in en.STSW-IMG035_M33.zip and code extracted and modified from the example in en.STSW-IMG035_F401.zip\VL53L5_GestureKit_ULD_1.5.0\Example\Gesture\gestures_example.c:
gesture_library_init_configure() fails because GW_set_frequency() fails with error -6, indicating that the frequency is invalid. The supplied frequency is the default of 20.0 (calculated from 1000.0/Params.RangingPeriod, Params.RangingPeriod is DEFAULT_GESTURE_APP_RANGING_PERIOD = 50ms).
As a quick test, I ignored the those failures and it resulted in a Hard Fault in GW_run() in HT_update_path().

Any suggestions on where to start debugging this issue?

Thank you.

[labussiy]

Hello Jason !

There is a basic check in the library on the input frequency : "if (freq <= 0) return -6"

It means that the param freq is not correct (or you are not supporting the float type).

Maybe you can try to print your param before calling the function:

MZ_TRACE("freq : %f \n",1000.0/Params.RangingPeriod);

Thanks,

Yann

[Jason Mount]

Yes, the frequency is a float with the value 20.000000.

I took a step back and verified that I could build and run the GestureKit example on the NUCLEO-F401RE using the XNUCLEO-53L8A1; it was able to recognize hand gestures as expected.  Then I started comparing the code in my project to the code in the GestureKit example.  The calls look to be the same and in the same order, the only differences seem to be that I am using the Cortex-M33 gesture library and the VL53L8CX_ULD_API (VL53L8CX_1.2.1) instead of the VL53LMZ_ULD_API (VL53LMZ_2.0.7) provided in the GestureKit.

After initializing the processor, I initialize the I2C and then, copying from LMZ_gesture_app_main(), I call:

LMZ_platform_init(&LMZDev);

vl53l8cx_init(&LMZDev);

gesture_library_init_configure();

It then fails with error -6.  Have I missed something?

I tried switching to the VL53LMZ_ULD_API to see if the older sensor firmware and driver code worked better, but the same error occurred.

[Jason Mount]

I think I may have found a work-around for now.  I still need to do more testing to verify the full operation, but this allowed me to move on with my development and all of the subsequent API calls succeeded.

I started digging into this by stepping into the assembly for GW_set_frequency() and I noticed that the frequency parameter was not being loaded into the floating point register s0 before the vcmpe comparison (s0 always contained 0):

So I tried adding an inline assembly instruction to load the s0 register before the call, and it worked:

float freq = 1000.0/Params.RangingPeriod;
asm("vmov s0, %0" :: "r"(freq));
status = GW_set_frequency(&gest_predictor, &hand_tracker, &sensor_data, freq);

I'm not sure how this would happen, is it just a bug in the libGesturesMZ_m33.a library?

[Jason Mount]

Now I'm having a similarly weird issue with SEN_set_data() in SEN_CopyRangingData().  No gestures were being recognized, and as I looked into it, SEN_CopyRangingData() always returns 0 but didn't seem to be doing it's job.  Digging further, sensor_data.ranging and sensor_data.valid are not being set by SEN_set_data() even though non-zero values are passed as parameters and sensor_data.peak is being filled, but with incorrect values (not matching the input parameter).

It's called as follows (by the way, there is a bug on line 536 of gesture_example.c, the 'status' return value isn't assigned in the original code):

				status = SEN_CopyRangingData(&sensor_data, &RangingData);
				if (status != 0)
				{
					MZ_TRACE("SEN_CopyRangingData failed : %d\n",status);
					return status;
				}

I copied the implementation from the gesture_example.c, only modifying the 2nd parameter to match the VL53L8CX driver:

// Format data from LMZ driver to Gesture algorithm
int SEN_CopyRangingData(SEN_data_t* pDest, VL53L8CX_ResultsData *pRangingData)
{
	float ranging, peak;
	bool valid;
	int idx, result, nb_targets, target_idx[3], target_status;

	if (pDest == NULL || pRangingData == NULL) return -1;

	pDest->timestamp_ms = GET_TIME_STAMP();

	for (idx = 0; idx < pDest->info.nb_zones; idx++)
	{
		select_target_index(target_idx, idx, pRangingData);

		ranging = pRangingData->distance_mm[target_idx[0]]/4.0; // Signed 14.2

		peak = pRangingData->signal_per_spad[target_idx[0]]/2048.0; // Unsigned 21.11
		nb_targets = pRangingData->nb_target_detected[idx];
		target_status = pRangingData->target_status[target_idx[0]];

		valid = (nb_targets > 0)
				&& (target_status == 5 || target_status == 6 || target_status == 9 || target_status == 10); //target_status == 4 -> wrapper on issue on L8
		result = SEN_set_data(pDest, idx, ranging, peak, valid);

		if (result != 0) return result;
	}

	return 0;
}

Can anyone confirm this strange behavior in the libGesturesMZ_m33.a library on the STM32U575?

Am I mistaken that this is the correct library to use on the STM32U575?

I'm using STSW-IMG035_M33 Gesture library for Cortex M33 version 1.9.2 available here:

https://www.st.com/en/embedded-software/stsw-img035.html

[John E KVAM]

Jason - 

I had a chat with a guy who implemented this on his U5. He tells me he had no issues. 

Of course that bothers me - a lot. 

Can I ask you to try changing your compiler flags. Perhaps turning optimization off?

It's got to be something like that, as I have two existence proofs that it works.

This is a long shot - if you are single stepping through it's probably already off. 

But it has to be some compile/build difference between your system and ours. 

.I admit this is a pure guess. 

- john

[Jason Mount]

Yes, it's really weird.  I'm using a debug build, so optimizations are off.  I opened a support request with more details. 

I was trying to get to the bottom of this, so I ported the GestureKit example from the NUCLEO-F401RE to the NUCLEO-U575ZI-Q.  To my surprise, it worked fine with the included MZ driver, but when I switched to the 8CX driver and M33 library, I could only get the TAP gesture to work.

So, apparently the original problem is some issue with my build environment.  I'm going to try to create a simple project to see if I can reproduce the issue.

But, now I'm confused at why the 8CX driver and M33 library perform worse than what I thought were the wrong driver and library for the hardware I'm using.  What am I missing?

[John E KVAM]

You took the gesture code from the STM32F401RE (Cortex M4) and ran it on your STM32U557 (CortexM33)?

And it ran???

Perhaps there is not much difference between the M4 and M33 when it comes to just doing math. 

What happened when you took the ported project, which could do 'tap' and just swapped the M33 Library? Did that do anything?

[Jason Mount · Accepted solution]

I figured out the strange issue with the library.  I created a test function that I could build and run in both environments to reproduce the problem and compared the disassembly.  I noticed that there were no floating point registers referenced in the code at all.  The other compiler defaults to soft floating point support, when I changed to hard floating point support it started working as expected.  I'm glad it was something simple.

I still have the issue with only the TAP gesture being recognized (I'm using the VL53L8CX 1.2.1 driver and M33 1.9.2 library).  As you suggested, I'll do more testing to see if swapping drivers and libraries makes a difference like it did with the GestureKit example.

[John E KVAM]

If you can get 'Taps' you should also get swipes. Try slowing down and making slow deliberate swipes. Try it from different distances. Getting too close to the sensor means your hand is not in the field of view long enough to get the 2 or 3 samples the algo needs. 

Might try practicing with the F401RE system, to get an idea of what it can do. Then move to your system. 

Glad to hear you figured out the compile issue. Who would have thought a compiler would default to soft floats when the processor had an FPU? That was a good find.

Allow me to mark your solution as "The Solution" and ask you to open another thread on your lack of swipes. 

That will keep the community page nice and tidy.

- john