After many forum searches, patching and considerable effort, I finally got good results with my CDC/ACM USB Host HS with DMA, using STM32Cube_FW_F7_V1.11.0. I thought I was past the worst of it, but long term testing has revealed infrequent hard crashes or mysterious USB stack hangs. With the help of the debugger, I discovered that the memory AFTER the my USB receive buffer was being corrupted. With the help of the MPU, I ruled out that any code running on the processor was even accessing the memory that was being corrupted. The only remaining conclusion is that the USB DMA operation was somehow overrunning the specified buffer and size options passed in, in my case, to USBH_CDC_Receive.
After reverse engineering how USB HS DMA host library is supposed to work, and referring to the rather vague and incomplete OTG_HS section of the reference manual, my guess is that the HCDMA register, which is set by the host library in USB_HC_StartXfer, and is auto-incremented as the DMA is being performed, is somehow being re-used without being rewound for a second transfer, and without transfer complete occurring in the meantime.
After patiently waiting many hours for it to crash, I did finally catch it in a state where I could observe from the USB data structure handles:
- A "data toggle error" had occurred on the input channel
- Memory after the receive buffer had been overwritten
- No receive callback had occurred, indicating the receive was still in progress
- And yet the input channel was not enabled
Basically, the stack was hung. However, in this state, simple issuing a new USBH_CDC_Receive kick-started the receive process and everything resumed working normally, except that memory after the receive buffer was corrupted, but I had left a large surplus to avoid corrupting other critical data structures in my program.