[Synopsys OTG] FIFO clash

[waclawek.jan]

So where exactly does the documentation say that different FIFOs can't be accessed simultaneously?

In dervice mode, an In endpoint was configured to transmit one 16-byte-long packet, and then 16 bytes (i.e. 4 words, 72 43 70 A4 D9 99 55 2E 19 7F 32 37 13 EB B6 7E) were being written to the endpoint's Tx FIFO, in "main" (and instrumentation and logging confirmed they were indeed written to the FIFO all, as 4 words), when the USB interrupt kicked in and it read from the Rx FIFO. The result on the bus was:

So what we see here? The USB core decided to transmit the first 5 bytes, 72 43 70 A4 D9, and appended the CRC calculated from the first 4 bytes of these 5. Naturally, host refused to ACK this. Core retried 2 more times, and the apparently gave up, inserting a zero-length DATA0 packet at 475 607 517ns (not expanded on that view, sorry). Then the USB core skipped 3 bytes (I'd say, in the first case it took 2 words, but used only one byte of the second word) and then transmitted one byte, with the CRC of 0 bytes (i.e. 0000). Rinse and repeat. Then core threw transfer complete interrupt and everything proceeded as usually.

I've seen similar patterns with the data packet containing 13 bytes, with CRC of the first 12 of them.

Preventing the USB interrupt to kick in solved the problem entirely; and also helped selectively disabling the particular interrupt resulting in reading from Rx FIFO (and not running any other In endpoint i.e. writing to some other Tx FIFO).

The "factory" firmware fills FIFOs only in the interrupt so it won't be hit by this. This is nice, but IMO working more by accident than by deliberate design, given the documentation doesn't appear to talk about conflicting FIFO accesses. In fact, there's nothing which would indicate, that the individual endpoints are not completely independent from each other.

I don't say this wasn't fun to find, but I am supposed to work and not having fun the whole day.

JW

[waclawek.jan]

> The rule of thumb is: Do NOT do anything in "main". ;)

It's MY program, and MY rules.

Also, I've paid for every single transistor on that chip, so I DEMAND their functionality to be adequately documented.

Weird, isn't it.

JW

[gbm]

Oh, so you want the DOEPINT register documented?! Isn't it too much? ;)

[Parawizard]

I am still battling with this. I don't quite 100% understand what is going on but it seems like this errata is worse than I first understood. I am seeing corrupt packets because of EP1 and EP0 FIFO clashing. The usb stack may put something into the EP0 fifo in anticipation of say a string descriptor that windows may only ask for a lot later. It seems almost like you would need to lock access until a transfer is complete across endpoints. Only can have one endpoint active at a time?

Even if all the calls are in the same ISR. Once the FIFO has been written isn't it possible that something happens to it during subsequence register access before it is transmitted? Like it's put it in the buffer and is waiting for the IN token.

[Parawizard]

I rewrote the area of the firmware so that the txfifo write for EP1 is in the same IRQ handler and there is no re-entry. Still the data in EP0 txfifo is corrupted! Do I need to serialize the endpoint FIFO writes? Seems nuts. Maybe either of you have an idea or is this errata worse than declared?

[Pavel A.]

> Only can have one endpoint active at a time?

This can actually have sense because USB is serial and everything is serialized on  two wires: RX or TX, for any endpoint. Nothing actually runs in parallel.  

[waclawek.jan]

At this point, I'd recommend you to start a new thread afresh, stating hardware and software used, any details which may be relevant, adding observed behaviour and explaining how is it different from the expectations.

JW

[Parawizard]

> This can actually have sense because USB is serial and everything is serialized on  two wires: RX or TX, for any endpoint. Nothing actually runs in parallel.

This is conceptually different than serializing the FIFO access though isn't it? I don't see STM doing so in their codebase. Are you saying that STM32 serializes different endpoints between fifo write and transfer complete?

[Parawizard]

Maybe I will. Is your original finding that FIFO operations can clash or is it only that interleaving OTG register access between partial FIFO writes can clash?

[Pavel A.]

(pardon Jan W. for polluting the thread)

> Are you saying that STM32 serializes different endpoints between fifo write and transfer complete?

I'm speculating that the Synopsis IP knows that there can be only one transfer on the bus at any time, either IN or OUT. Each transfer is preceded by setup phase. So the RX and TX FIFO data moves do not need to be "full duplex" and no locking is made against this in the IP. The problem indicated by Jan is that due to the Synopsis design, the STM32 software cannot easily know when it is safe to switch FIFOs (or can it?). The IP passed the validation and taped out. One of my bosses used to say, a hardware bug worked around by software is reported to the management as software bug [so software folks take the hit].

[Parawizard]

If only we had access to the OTG databook... I have asked ST directly about this issue. I have also asked them to update their errata for all affected MCU using the Synopsis OTG FS and HS controllers.

The only way I see so far that one could design this in the software side is that you would have to lock from write to transmit complete. ST has a lot more information about how the core works surely. ST doesn't even follow the reference manual and implement recommend checks before doing a TXFFLSH I think.

> One of my bosses used to say, a hardware bug worked around by software is reported to the management as software bug [so software folks take the hit].

Haha. On point.