VL53L4CD I2C + CircuitPython Misbehaving

[Tom_Ziegler]

Hi everyone,

I'm thoroughly flummoxed and in need of guidance.  I have a VL53L4CD rangefinder on the SparkFun breakout board that will not play nice with my Raspberry Pi.  Long story short, the rangefinder seems to be ACKing prematurely under certain conditions.  Confusingly, an identical setup has worked great in the past.

Background

Last year I used the VL53L4CD as part of a simple Raspberry Pi 4B-controlled payload.  Using the SparkFun breakout board we did some lab testing with the rangefinder connected straight to the Pi, powered by the Pi's 3V3 regulator.  We had some issues with continually bombarding the rangefinder with DataReady() requests but by waiting a few ms between queries our problems went away.  The final product involved a couple custom PCBs with a beefier 3V3 linear regulator for the non-Pi electronics and an LTC4316 I2C address translator in between the Pi and rangefinder, and the whole setup worked great.  Software was the CircuitPython library with Adafruit Blinka as a compatibility layer for the Pi 4B.  This has worked without issue for months.

Some weeks ago I started building a second model of the same system.  Made another batch of the PCBs, wired up the rangefinder, and got `[Errno 5] Input/output error`, which started me down the following rabbit hole.

 The Problem

When connected to a Raspberry Pi 4B, the rangefinder seems to get confused and ACK when it shouldn't.  I have tried this with three different Pi's and two different rangefinders, everything I had in stock this week.  Both rangefinders were recently purchased from Mouser.  The setup is as simply breadboarded as I can get, with ~1k8 pullup resistors; I have pix but since I only get three uploads for this post I'll save it for the comments.  The behavior is inconsistent, but almost always I2C communication fails during initialization of the rangefinder (though I have twice gotten a single measurement from it).  Below is an example of what I'm seeing:

Fig. 1: Failed rangefinder initialization. Yellow: SDA, Cyan: SCL

Fig. 2: Premature ACK and rangefinder holding SDA low. Yellow: SDA, Cyan: SCL

As in the captions, the yellow trace is SDA and cyan is SCL.  The current CircuitPython instruction being executed is line 194, `self._write_register(0x002D, init_seq)`.  You can see that in the decoded section of Fig. 1, where the Pi has written to register 0x002D bytes 0x12, 0x00, 0x00, 0x11 from the initialization sequence.  

As far as I can tell, the slightly lower SDA voltage is the rangefinder pulling down to write to the bus or ACK, which is the basis of my suspicion for premature ACKing.  As you can see in the second figure, the previous byte 0x00 was ACK'd successfully and the Pi moves to sending the next byte.  On the fifth bit, the rangefinder ACKs again (which will clobber a 1 if the Pi was trying to write, which is what tipped me off to this problem in the first place) and is now out of sync with the byte-stream.  The Pi interprets the rangefinder's silence as a NACK and releases SDA, but sometimes (as shown above) the rangefinder then takes SDA and holds it low until I power cycle it.

...But the Rangefinder Works!

To try and simplify the software side of things, I connected it to an Arduino Mega, using the relevant software library. I was too lazy to disable the internal pull-up resistors on the I2C pins, so the bus voltage is a smidge higher but still (barely) within the datasheet allowance.  Shown below is successful communication with the same rangefinder as in the traces above.  I have seen at least one out-of-turn ACK with the Arduino but I don't have a good picture, and it happens much more infrequently.  The one difference I see is the Arduino seems to add a couple μs of clock delay (I don't know if you call that clock stretching if it's on the master side) between bytes.  I have not yet tried banging in the same behavior in the Pi.  The message shown is part of the polling loop querying register 0x0030 to see if data are ready to be read.

Fig. 3: Successful communication with the Arduino. Yellow: SDA, Cyan: SCL

Conclusion

I really don't know what would be different between the successful rangefinder-Pi system I put together six months ago and the replica I made this week, other than a minor update in Linux version, which I have not yet tried rolling back.  Unfortunately I don't have access to the old system at the moment as it was just delivered to the international space station about ten minutes ago.  I've found some similar posts [1][2][3][4] where my hero @John E KVAM has usually helped to diagnose some I2C-related issue or another, so while I've done everything I can think of to isolate potential hardware problems I recognize I'm probably missing something. I'm pretty young and inexperienced, so any guidance or suggestions from anyone would be greatly appreciated.  I have more pictures of the setup and other weird behavior; again, this is just one example of many ways comms have failed.  

Thanks in advance!

-Thomas

[John E KVAM · Accepted solution]

Thomas - I rarely see a post with as much data as yours, well done there. 

I've used I2C almost since it was invented. And yours is a very common problem. A small glitch on the clock line, and the 'master' and 'slave' get out of sync. (Really should change those terms - how about MCU and sensor instead?)

You end up with the clock or data line held low. The 'bus stuck low' is so common that TI wrote a paper on the issue.

https://www.ti.com/jp/lit/an/scpa069/scpa069.pdf?ts=1730820610338&ref_url=https%253A%252F%252Fwww.google.com%252F

But there are lots of others. 

I'm going to blame it on your level shifter. Because it's always the level shifter. I hate those things. 

The IOVDD on the VL53L4 has a max of 3v5 and if your RPi is 3v3 I can see why you have designed one in.

As a test, could you bypass it? Or switch to a different kind?

Another common I2C issue is line length. I've found a foot is about as long as anyone should run an i2C. 

But as you have stated, it's those short spikes that are killing you. You have to get rid of them somehow. 

I'd suggest a small cap - if you aren't trying to run you I2C at the fastest rate. That might just fix you up. 

[Tom_Ziegler]

Thanks very much, @John E KVAM. I added 10 pF capacitance to SCL and now it takes on the order of minutes before the rangefinder clock de-syncs instead of ~100 us (EDIT: having upped to 47 pF, it's been running for half an hour without issue).  That's very much workable. Now that you jogged my memory, there was a difference between the flight model and the replica I just built; I was too lazy to include shielding on the replica's wire bundles, which I suspect would have added a non-negligible amount of parasitic capacitance.  :facepalm:

I'm going to go ahead and call this resolved!  I'm sorry that this forum is 90-odd percent troubleshooting communication bus issues, but I really appreciate you taking the time to do it anyway. For posterity, I'll clarify a couple more details about my setup that you brought up:

- I removed the address translator/level shifter from the test setup not long after I started trying to solve the problem, for the sake of simplicity.  I've included a picture below of my test setup with the Arduino (not pictured: additional 1k2 pull-up resistors).  Per your suggestion in a previous post I had twisted SCL and GND together. 

- Wire lengths for both test setup and the replica model were approaching 1 ft so I'm definitely on the edge of what is doable for I2C.  A good learning experience, for sure.

Fig. 1: Test setup with Arduino