EP0 IN transfer times out

[rudbeckia]

For learning (and suffering) purposes I am trying to write my own USB Driver in device mode. Right now I am just trying to successfully enumerate the device. So far I am able to read SETUP packets but when writing to EP0 (IN transfer) the requests seem to timeout at 5 seconds (seeing this through Wireshark). I am using a STM32F401CCU6 chip.

For example, this is what happens when a GET_DESCRIPTOR request is made:

USB URB
    [Source: host]
    [Destination: 3.0.0]
    URB id: 0xffff9d142a6f8900
    URB type: URB_SUBMIT ('S')
    URB transfer type: URB_CONTROL (0x02)
    Endpoint: 0x80, Direction: IN
    Device: 0
    URB bus id: 3
    Device setup request: relevant ('\0')
    Data: not present ('<')
    URB sec: 1688842172
    URB usec: 857980
    URB status: Operation now in progress (-EINPROGRESS) (-115)
    URB length [bytes]: 64
    Data length [bytes]: 0
    [Response in: 58]
    Interval: 0
    Start frame: 0
    Copy of Transfer Flags: 0x00000200, Dir IN
    Number of ISO descriptors: 0
Setup Data
    bmRequestType: 0x80
    bRequest: GET DESCRIPTOR (6)
    Descriptor Index: 0x00
    bDescriptorType: DEVICE (0x01)
    Language Id: no language specified (0x0000)
    wLength: 64

And the answer generated by my MCU (and code):

USB URB
    [Source: 3.0.0]
    [Destination: host]
    URB id: 0xffff9d142a6f8900
    URB type: URB_COMPLETE ('C')
    URB transfer type: URB_CONTROL (0x02)
    Endpoint: 0x80, Direction: IN
    Device: 0
    URB bus id: 3
    Device setup request: not relevant ('-')
    Data: present ('\0')
    URB sec: 1688842178
    URB usec: 31400
    URB status: No such file or directory (-ENOENT) (-2)
    URB length [bytes]: 64
    Data length [bytes]: 64
    [Request in: 57]
    [Time from request: 5.173420000 seconds]
    Unused Setup Header
    Interval: 0
    Start frame: 0
    Copy of Transfer Flags: 0x00000200, Dir IN
    Number of ISO descriptors: 0
DEVICE DESCRIPTOR
    bLength: 18
    bDescriptorType: 0x01 (DEVICE)
    bcdUSB: 0x0200
    bDeviceClass: Communications and CDC Control (0x02)
    bDeviceSubClass: 2
    bDeviceProtocol: 0
    bMaxPacketSize0: 64
    idVendor: STMicroelectronics (0x0483)
    idProduct: Virtual COM Port (0x5740)
    bcdDevice: 0x0100
    iManufacturer: 0
    iProduct: 0
    iSerialNumber: 0
    bNumConfigurations: 1
Leftover Capture Data: 000000000000000000000000000000000000000000000000000000000000000000000000…

I don't have a hardware analyzer but it seems like the write request is timing out and is not properly being ended after transmitting the 18 bytes of the device descriptor, therefore the host just fills everything with zeroes until it times outs.

This is how I write to the endpoint:

void usbWrite(uint8_t ep, void* data, uint8_t len) {
    USB_OTG_INEndpointTypeDef* endpoint = usbEpin(ep);
    volatile uint32_t* fifo = usbEpFifo(ep);

    uint16_t wordLen = (len + 3) >> 2;
    if (wordLen > endpoint->DTXFSTS) {
        return;
    }
    if ((ep != 0) && (endpoint->DIEPCTL & USB_OTG_DIEPCTL_EPENA)) {
        return;
    }

    endpoint->DIEPTSIZ = (1 << USB_OTG_DIEPTSIZ_PKTCNT_Pos) | len;
    endpoint->DIEPCTL &= ~USB_OTG_DIEPCTL_STALL;
    endpoint->DIEPCTL |= USB_OTG_DIEPCTL_EPENA | USB_OTG_DIEPCTL_CNAK;
    usbRawWrite(fifo, data, len);
}

void usbRawWrite(volatile uint32_t* fifo, void* data, uint8_t len) {
    uint32_t fifoWord;
    uint32_t* buffer = (uint32_t*)data;
    uint8_t remains = len;
    for (uint8_t idx = 0; idx < len; idx += 4, remains -= 4, buffer++) {
        switch (remains) {
            case 0:
                break;
            case 1:
                fifoWord = *buffer & 0xFF;
                *fifo = fifoWord;
                break;
            case 2:
                fifoWord = *buffer & 0xFFFF;
                *fifo = fifoWord;
                break;
            case 3:
                fifoWord = *buffer & 0xFFFFFF;
                *fifo = fifoWord;
                break;
            default:
                *fifo = *buffer;
                break;
        }
}

And here is how I am setting up the control endpoints and FIFOS:

#define USB_FIFO_EP0_SZ 0x20
#define MPSIZ_64B 0x00

OTG->GRXFSIZ = USB_FIFO_EP0_SZ;
OTG->DIEPTXF0_HNPTXFSIZ = (USB_FIFO_EP0_SZ << 16) | USB_FIFO_EP0_SZ;

USB_OTG_INEndpointTypeDef* in0 = usbEpin(0);
in0->DIEPCTL |= MPSIZ_64B | USB_OTG_DIEPCTL_SNAK;

USB_OTG_OUTEndpointTypeDef* out0 = usbEpout(0);
out0->DOEPTSIZ = (1 << USB_OTG_DOEPTSIZ_STUPCNT_Pos) | (1 << USB_OTG_DOEPTSIZ_PKTCNT_Pos) | 64;
out0->DOEPCTL |= MPSIZ_64B | USB_OTG_DOEPCTL_CNAK | USB_OTG_DOEPCTL_EPENA;

Any idea as to what I am doing wrong? It is not a board issue as Cube auto generated code works (at least it enumerates)

[waclawek.jan]

How do you handle the end-of-Tx event (DIEPINT.XFRC=1)? The SETUP IN sequence ends with Status Stage which is an OUT packet, and the you have to prepare the device to receive it (otherwise it NAKs thus host eventually times out).

"Leftover Capture Data" is IMO a red herring.

Btw. you are not supposed to clear the STALL bit in DIEPCTL manually.

JW

[rudbeckia]

Interesting, I don't handle it, so I guess I need to look into that. During  debug sessions the flow looked  something like this:

1. USB RESET
2. USB ENUMERATION DONE
3. USB RXFLVL
4. I transmit
5. USB RXFLVL

Step 5 was unexpected for me but I guess it is the Status Stage you mention. I am looking at other USB implementations and for example, on dmitrystu/libusb_stm32 a fake read is performed, as far as I can see the only thing done is GRXSTSP is popped which I already do on every read. Should I do something else? This is how I read when RXFLVL is triggered:

static void usbRead() {
    uint32_t readStatus = OTG->GRXSTSP;
    uint8_t epNum = readStatus & 0xF;
    uint16_t byteCount = (readStatus >> 4) & 0x7FF;
    if (byteCount == 0) {
        return;
    }

    switch ((readStatus & USB_OTG_GRXSTSP_PKTSTS) >> 17) {
        case PKTSTS_OUT_RX:
            break;
        case PKTSTS_SETUP_RX:
            if (epNum != 0) {
                return;
            }
            usbReadSetup(usbEpFifo(0), &controlRequest);
            usbProcessSetup(&controlRequest);
            break;
    }
}

[rudbeckia]

The value of GRXSTSP on read 5 basically denotes a SETUP transaction completed (BCNT = 0 and PKTSTS = 0100)

[waclawek.jan · Accepted solution]

Event 5. is not the Status stage; it's still Setup stage.

At the signaling level, Setup stage consists of three packets: Setup token, Data (8 byts), and ACK (from Device (a.k.a. Function in USB2.0) to Host) - see USB 2.0, 8.5.3 Control Transfers.
.What happens in the OTG machine (which consists of the "USB core", that is the part facing the bus which contains the USB transactions decoding machine; and "APB interface" which is what you as a programmer see; these run at potentially two different clocks (on STM32 it happens only in the HS interface when using external PHY) with a bunch of resynchronizers and FIFO-like interfaces i between them):

• SETUP token arrives, USB core sets itself a flag
• DATA packet arrives, USB core stores it to FIFO, stores  "SETUP data packet received" item into a mini-FIFO (top of which you see as OTG_GRXSTSP), non-empty mini-FIFO throws the RXFLVL interrupt
• you are supposed to read OTG_GRXSTSP and then the data from FIFO and store them in some memory buffer, but not process them yet
• core responds with ACK to Host, and then stores a " SETUP transaction completed" item into the mini-FIFO, that again throws RXFLVL interrupt (that's your 5. step), where reading "SETUP transaction completed" from OTG_GRXSTSP throws the OTG_DOEPINT.STUP interrupt
• handling the STUP interrupt, you now decode the arrived setup data, and decide what to do; if that packet requires you to send something to Host, you then start that In process (whatever it takes)
• when sending the last packet for In process, enable the endpoint 0 for an Out transaction, so that it is able to receive the Status stage packet, which it receives as normal 0-length data (note that Out transactions generally work in the same way as Setup, i.e. core signals you first arrival of data through the mini-FIFO, and then signals the end of transaction which in turn throws an interrupt (XFRC))

JW

[rudbeckia]

Thanks for the explanation. I have worked on it and added handlers for DOEPINT.STUP and DIEPINT.XFRC (endpoints hardcoded for the time being):

static void usbEpInterruptOut() {
    USB_OTG_OUTEndpointTypeDef *ep = usbEpout(0);
    if (ep->DOEPINT & USB_OTG_DOEPINT_STUP) {
        usbProcessSetup(&controlRequest);
    }
    ep->DOEPINT = ep->DOEPINT;
}

static void usbEpInterruptIn() {
    USB_OTG_INEndpointTypeDef *ep = usbEpin(0);
    USB_OTG_OUTEndpointTypeDef *epOut = usbEpout(0);
    if (ep->DIEPINT & USB_OTG_DIEPINT_XFRC) {
        // Write complete. Clear NAK and enable to receive STATUS OUT
        epOut->DOEPCTL |= USB_OTG_DOEPCTL_CNAK | USB_OTG_DOEPCTL_EPENA;
        epOut->DOEPTSIZ |= (1 << USB_OTG_DOEPTSIZ_PKTCNT_Pos);
    }
    ep->DIEPINT = ep->DIEPINT;
}

Also as you recommended I no longer perform any writes on my read routine. I simply read the FIFO (if required) or pop the mini FIFO (GRXSTSP). However on Wireshark everything is looking the same (I can see the descriptor but it seems it still times out).

Here is the flow that now happens upon USB being connected:

Breakpoint 1, usbReset () at ../lib/usb.c:219
219         OTGD->DCFG &= ~USB_OTG_DCFG_DAD;
halted: PC: 0x08001116

Breakpoint 2, usbAfterReset () at ../lib/usb.c:224
224     static void usbAfterReset(void) {
halted: PC: 0x0800114e

Breakpoint 3, usbRead () at ../lib/usb.c:142
142     static void usbRead() {
GRXSTSR: 18c0080
halted: PC: 0x080012fe

Breakpoint 3, usbRead () at ../lib/usb.c:142
142     static void usbRead() {
GRXSTSR: 1a80000
halted: PC: 0x080012fe

Breakpoint 4, usbWrite (ep=ep@entry=0 '\000', data=data@entry=0x20000008 <stellDeviceDescriptor>, len=len@entry=18 '\022') at ../lib/usb.c:49
49      void usbWrite(uint8_t ep, void* data, uint8_t len) {
halted: PC: 0x08001216

Breakpoint 5, usbEpInterruptIn () at ../lib/usb.c:173
173             epOut->DOEPCTL |= USB_OTG_DOEPCTL_CNAK | USB_OTG_DOEPCTL_EPENA;
halted: PC: 0x0800119e

Breakpoint 1, usbReset () at ../lib/usb.c:219
219         OTGD->DCFG &= ~USB_OTG_DCFG_DAD;
halted: PC: 0x08001116

As it can be seen, I successfully reach the DIEPINT.XFRC interrupt, but apparently I am not setting up EP0 to receive OUT packages correctly. Do you have any ideas as to what I may be doing wrong? I looked at RM0638 but can't find a concrete example or section where it states how to prepare endpoint to receive OUT packets. The RESET I get after XFRC is not inmediate, but takes around 5 seconds after the IN transfer completes.

[waclawek.jan]

Almost there.

If you did everything correctly, you should see again breakpoint 3 for the OUT token (I'm not sure if you'll see it twice, once with zero length data packet and second time upon the ACK/completion, or just the latter one), and then a OUT XFCR interrupt.

As you don't see those, you did something incorrectly. I'd try to swap

        epOut->DOEPTSIZ |= (1 << USB_OTG_DOEPTSIZ_PKTCNT_Pos);
        epOut->DOEPCTL |= USB_OTG_DOEPCTL_CNAK | USB_OTG_DOEPCTL_EPENA;

JW

[rudbeckia]

Thanks for your continuous support. You helped me a lot, specially in better understanding the micro's internal USB core flow. I was debugging a little bit more and realized that EP0.DOEPINT indicated a NAK was being sent, and an OUT package was received while EP0 was disabled, this interrupt was triggered after the IN write, this all matched your initial analysis.

Going through the datasheet once again, I found the following under Programming model - Generic non-isochronous OUT data transfers:

If there is no space in the receive FIFO, isochronous or non-isochronous data
packets are ignored and not written to the receive FIFO. Additionally, non-
isochronous OUT tokens receive a NAK handshake reply.

So I changed my RX Fifo size from 0x20 to 0x30 and it worked :downcast_face_with_sweat:. The descriptor packet no longer times out on Wirehsark, appears immediately (0.001232 seconds after request) and with the correct size (18 bytes instead of 64).

For future reference for anyone else going through this. Here is the complete code which worked for me. Far from perfect but at least gets you started with correctly providing GET_DESCRIPTOR requests.

https://github.com/crazybolillo/stell/commit/5fe772f595b7cbd70d1f89e3bd92ee4b2608b33d

Now I only need to implement GET_CONFIGURATION, SET_CONFIGURATION and a bunch of CDC stuff :face_with_tears_of_joy:.

Thanks again!