Interrupts stop on USBD CDC while usbmon shows EPROTO -71

[FastestCuber]

Occasionally after several hours our USBD CDC will get stuck in a state where "CDC_Transmit_FS" always returns "USBD_BUSY" due to the "TxState" flag never being cleared. During this time the USB IRQ "HAL_PCD_IRQHandler" is never called even though IRQs are still enabled. 

I am using "STM32Cube FW_F4 V1.25.2" but have tried copying "STM32Cube FW_F4 V1.28.2" with no luck.

MCU is STM32F405RGT6. 

The only remedy is to reset USB by calling HAL_PCD_DeInit & HAL_PCD_Stop, then reinitializing with HAL_PCD_Init, USBD_Start, and MX_USB_DEVICE_Init.

This issue can be recreated by attaching a UART (non-USB) to the board which seems to provide additional 5V power potentially causing a disturbance in the USB. Doing so recreates the issue fairly quickly, however this can also happen without UART attached after several hours. 

I also noticed that I can get into this state if I call CDC_Transmit_FS multiple times and bypass the "TxState" flag, effectively calling "USBD_CDC_TransmitPacket" multiple times. Our code does not appear to call this multiple times as it is only called from one thread. Here is a link to another STM user with similar issues. This led me to attempt calling transmit multiple times, ignoring the TxState flag. 

Question: 

1. Can an EMI disturbance on USB communication cause the USB peripheral to go into a locked state like this? (no interrupts and stuck in USBD_BUSY)

2. What registers can I check if it is in this locked state? 

3. Any remedy to this issue? i.e. Slowing down the bus, checking registers periodically, etc. 
Resetting USB causes too much disruption.

[FBL]

Hi @FastestCuber 

1. EMI-induced noise can alter the data bits, resulting in CRC errors... You can monitor MMIS bit in the
   OTG_FS_GINTSTS register to check if an illegal access occurs.
2. Figure 394 in RM shows all the interrupt hierarchy. 
3. USB protocol analyzer is extremely helpful in diagnosing and correlating USB traffic. You can attach USB traffic and we can help you.

[FastestCuber]

1. Checked the MMIS bit. It is never set.

2. I checked the state of EP1 since that seems to be where the EPROTO errors occur. Under normal operation it seems the DIEPINT1 register reads "0x2091" and clears to "0x2090" whenever the transfer completes. 

When this error occurs, the last value is 0x2090 and it never gets set to "0x2091" to indicate the transfer is complete. 

It seems the transfer is started according to the TxState flag, but it never completes and the "Bit 0 XFRC: Transfer completed interrupt" is never set which is why the TxState flag never clears to 0.

3. Attached is a wireshark capture of usbmon. I just included device 16 and the section just before it goes into this EPROTO error state. Unfortunately, I do not have a USB protocol analyzer device. 

 I'm not sure how to identify what part of the USB is locking up. It seems to be a hardware issue but how can I finesse the hardware to keep working without resetting the whole device? It seems like the transfer is hung up but is there a way to clear it and retry? 

[FBL]

Hi @FastestCuber 

Frame 126 indicates that the device did not send any data back to the host. This is likely where the problem begins, as the host is expecting data that it never receives.

The endpoint 1 might stall due to an unexpected condition, a mismatch in expected data size or some configuration is missing.

To replicate the issue on my end,  would you attach a minimum firmware example to reproduce on my reference board ? F407 Disco?

[FastestCuber]

USBD_StatusTypeDef USBD_LL_Init(USBD_HandleTypeDef *pdev)
{
  /* Init USB Ip. */
  if (pdev->id == DEVICE_FS) {
  /* Link the driver to the stack. */
  hpcd_USB_OTG_FS.pData = pdev;
  pdev->pData = &hpcd_USB_OTG_FS;

  hpcd_USB_OTG_FS.Instance = USB_OTG_FS;
  hpcd_USB_OTG_FS.Init.dev_endpoints = 4;
  hpcd_USB_OTG_FS.Init.speed = PCD_SPEED_FULL;
  hpcd_USB_OTG_FS.Init.dma_enable = DISABLE;
  hpcd_USB_OTG_FS.Init.phy_itface = PCD_PHY_EMBEDDED;
  hpcd_USB_OTG_FS.Init.Sof_enable = DISABLE;
  hpcd_USB_OTG_FS.Init.low_power_enable = DISABLE;
  hpcd_USB_OTG_FS.Init.lpm_enable = DISABLE;
  hpcd_USB_OTG_FS.Init.vbus_sensing_enable = DISABLE;
  hpcd_USB_OTG_FS.Init.use_dedicated_ep1 = DISABLE;
  if (HAL_PCD_Init(&hpcd_USB_OTG_FS) != HAL_OK)
  {
    Error_Handler( );
  }

  HAL_PCDEx_SetRxFiFo(&hpcd_USB_OTG_FS, 0x80);
  HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_FS, 0, 0x40);
  HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_FS, 1, 0x80);
  }
  return USBD_OK;
}

Here is how the device is initialized. It is used OTG_FS and CDC serial. Unfortunately, I don't have a good way to recreate it in the software as the issue seems to occur due to a hardware anomaly. Plugging in a 3.3V UART seems to power the board in addition to the normal power supply, and I believe this generates noise on the USB. The device does not reboot when I plug in the UART, but something occurs after a few minutes that causes USB to go into this locked state. 

However, if you want to get into the same EPROTO error state where Tx is unresponsive then that can be created by doing the following: 

Modify the two functions CDC_Transmit_FS and USBD_CDC_TransmitPacket to use this "forceTxStat" flag to bypass the TxState flag check. Then call CDC_Transmit_FS multiple times. This will have the same effect in terms of locking up transmit and causing the EPROTO errors. 

uint32_t forceTxStat = 1;

uint8_t CDC_Transmit_FS(uint8_t* Buf, uint16_t Len)
{
  uint8_t result = USBD_OK;
  /* USER CODE BEGIN 7 */
    if (hUsbDeviceFS.dev_state != USBD_STATE_CONFIGURED) {
        return USBD_FAIL;
    }
    USBD_CDC_HandleTypeDef *hcdc = (USBD_CDC_HandleTypeDef *)hUsbDeviceFS.pClassData;

      if (hcdc->TxState != 0 && !forceTxStat) { 
          return USBD_BUSY;
      }
    
      USBD_CDC_SetTxBuffer(&hUsbDeviceFS, Buf, Len);

      result = USBD_CDC_TransmitPacket(&hUsbDeviceFS);

  /* USER CODE END 7 */
  return result;
}


uint8_t USBD_CDC_TransmitPacket(USBD_HandleTypeDef *pdev)
{
  USBD_CDC_HandleTypeDef *hcdc = (USBD_CDC_HandleTypeDef *)pdev->pClassData;
  USBD_StatusTypeDef ret = USBD_BUSY;

  if (pdev->pClassData == NULL)
  {
    return (uint8_t)USBD_FAIL;
  }

  if (hcdc->TxState == 0U || (hcdc->TxState == 1U && forceTxStat))
  {
    /* Tx Transfer in progress */
    hcdc->TxState = 1U;

    /* Update the packet total length */
    pdev->ep_in[CDC_IN_EP & 0xFU].total_length = hcdc->TxLength;

    /* Transmit next packet */
    (void)USBD_LL_Transmit(pdev, CDC_IN_EP, hcdc->TxBuffer, hcdc->TxLength);

    ret = USBD_OK;
  }
  return (uint8_t)ret;
}

[FastestCuber]

I was able to clear the locked state for the software recreation (i.e. calling CDC_Transmit_FS multiple times ignoring the TxState flag) by using the following code: 

        USBx_INEP(1)->DIEPCTL |= USB_OTG_DIEPCTL_SNAK;
        HAL_Delay(1);  
        USBx_INEP(1)->DIEPCTL |= USB_OTG_DIEPCTL_EPDIS;
        HAL_Delay(1);  
        USBx_INEP(1)->DIEPTSIZ = 0;
        USBx_INEP(1)->DIEPINT = 0xFF;  // Clear all endpoint-specific interrupts

        uint8_t delay = 10;
        USB_OTG_FS->GRSTCTL |= (USB_OTG_GRSTCTL_TXFFLSH | (1 << 6));  // Flush TX FIFO for EP1
        while (USB_OTG_FS->GRSTCTL & USB_OTG_GRSTCTL_TXFFLSH && delay > 0)   
        {
            delay--;
            HAL_Delay(1);
        }

        USBD_CDC_HandleTypeDef *hcdc = (USBD_CDC_HandleTypeDef *)hUsbDeviceFS.pClassData;
        hcdc->TxState = 0;

However, if I use this code when in my locked state due to EMI noise then the device loses the file descriptor for the serial port. It appears that attempting to soft reset the end point causes the device to be lost on Linux side. 

Perhaps my errors are hardware issues that cannot be recovered? It would be nice to know what error has occurred though since it seems the only indication is that it returns USBD_BUSY continuously and the HAL PCD interrupts stop coming through. 

[FastestCuber]

Last piece of information I discovered, the DIEPTSIZ stays non-zero on EP1 when the transfer doesn't occur. 

Tx Sizes DIEPTSIZ tx size = 101, packet cnt = 2

The "EPENA: Endpoint enable" bit is also set for EP1 but it appears to not transfer data. 

USBx_INEP(epnum)->DIEPCTL & (USB_OTG_DIEPCTL_CNAK | USB_OTG_DIEPCTL_EPENA) = 0x80000000