Keeping data on multiple STM32s in Sync

[LMorr.3]

Hello,

I need guidance on how to best implement a packet protocol to keep the data on multiple STM32F373 in sync with each other.  If possible, I would like to reach a 'refresh rate' of 30ms.  Packet payloads for data refreshed could be up to 50 bytes.

I currently have 1 'master' uC, and it connects to about 20 slave 'module' devices over SPI as master-slave.   The master also connects to up to 12 'control panel' devices over a second SPI.  The modules gather and output data while the control panels allow the user to read and change values on the modules. 

I would like to refresh the control panels and modules as fast as possible. ( 30ms would be ideal )

The issue seems to be that polling 10 or more devices at 115200baud will result in jitter even with packets as small as a few bytes.  Having each module send ALL data on each poll cycle seems inefficient when the data has not changed, but at the same time including all the data makes the timing deterministic which is a benefit.  

I am considering using an event based protocol instead, where slave modules and control panels only send data when the data has changed, however I think if all data has changed the jitter may appear, but less frequently and more difficult to detect/troubleshoot.

I'm thinking of having control panels simply broadcast a packet to the slave instead of waiting for a response. ( that would remove the timeout retry part of my protocol potentially causing other issues )

I'm also looking at setting up a multi-master SPI but I don't think that will help with jitter.

I have always been curious about how others have addressed this.   

Thank you,

[Bob S]

You mention 115200 baud, which is a UART baud/bit rate.  But then you say you are using SPI ports everywhere.  Which is it?

If it is SPI, then there can't be any event base protocol unless you have separate interrupt lines from each slave back to the master.  You could have a command that retrieves a status byte that indicates if there is any new data.  But the master would still have to poll each slave to get the status.  On the plus side, if it IS really SPI ports, depending on how long the connections are to the slave devices, you can run that at much higher bit rates than 115200 bits/sec.

[LMorr.3]

You are right, I was working with UART as well but using SPI so I should not have mentioned 115200.  Thanks for the heads-up on being able to run SPI at a much faster rate, which will most likely solve my concern. 

The connection to the slave devices will be 4" to 6" ribbon cables. ( thinking of having 'SPI IN' 'SPI OUT' ports on each module and connect them to each other instead of having many cables going directly to each device )

I also now plan on connecting the 'control panels' directly to the slave devices instead of them going through a master uC which solves some of the issue as well.

I may also allow slaves to 'broadcast' packets to all slaves via the master, but the master would not be able to get an ack response from each slave.  I guess I could do a broadcast to all slaves and then send a second packet to each slave to get a status.  

I'm also including a USB->VirtualPort to allow a PC to connect which ads to the complexity a bit.

Thanks for the info.

[tjaekel]

You can do "broadcast" via UART or SPI:
But you have to configure as uni-directional (just Tx from master, no input on master from any slave).

It should work with UART: UART Tx goes to all slaves which do Rx (only).
The same with SPI: all MOSI goes to all slaves (and NSS selects all at the same time, or slave is "hot" vis SW NSS) - but MISO is not wired, not used (cannot be done). Use SPI in uni-directional mode.

The Master is Tx, all the slaves are Tx (only).

Now, all slaves should get the updated data at the same time (almost in sync, maybe a bit different INT latency, use INT for Rx, not polling mode).

[LMorr.3]

Could I not use SPI with TX/RX enabled on the master and slaves as bi-directional, and then broadcast to all sales by using 255 as deviceId?  All slaves would know to accept that packet but not respond back to the master.  When slaves receive non-broadcast packets addressed to their deviceId, they respond ack back to the master.  Let me know if there is a reason I should have it in uni-directional mode.  Thanks!

[tjaekel]

NO!!!! You cannot!

SPI is full-duplex, bi-directional (as 4-wire): all the time you send one byte from Master - the Slaves (and all) will send also a byte on MISO! If you combine all MISOs - a driver conflict: all Slaves want to drive now at the same time on MISO (and it conflicts).

Even you would send 1st byte with 0xFF (as indication for "broadcast") - it cannot work:
at the same time Master sends (shifts out) the 0xFF - all the slaves will also shift out a byte on MISO! Again a conflict!

In full-duplex, bi-directional mode: for every bit coming in on slave - the slave sends out a bit. CONFLICT!

It works only in uni-directional mode (a slave is never sending, the MISO signals are not used, not connected).

You might be able to implement a "SPI protocol" like:

• all starts in uni-directional mode (Slaves just listen, never respond)
• you could "enable" a particular slave (e.g. via DeviceID), via a CMD via SPI
• so that this particular slave changes now to full-duplex: it can respond now on MISO on the next following command
• BUT!:
  You have to make sure that all slaves, even they are in "Rx mode" only - do not touch (do not drive) the MISO signal, even not holding the GPIO pin high or low (all the slaves must "release the SPI bus").
• and open question: how to release a selected slave for bi-directional mode? Another command needed.
  A quite complex "protocol".

If you want to update all slaves at the same time - done by a master sending a "broadcast" it can only work in "single direction" mode (all slaves are just Rx, never a Tx).

[LMorr.3]

I see what you mean, thanks for the clarification as is is very helpful.  I mistakenly thought SPI slaves would still be able to 'listen' on MOSI even if NSS was high, and use an open-drain on the salves for the common MISO.  I see now that solution won't work.  

Maybe I can use UART instead of SPI even at the slower speed.  

I think my only options are to either chain slave devices together using 2 SPI uni-directional ports on each, or to have the master toggle the NSS but limit the number of slaves to the available GPIO or connectors on the board.    Let's see how far I get today.

UPDATE: I could use USART with a higher baud rate, and ensure the devices all use the same crystal and uC?  I could them poll devices reasonably fast enough.  Slave devices would then be able to respond back without NSS or CS.

UPDATE2: I ended up going with I2C