STM32F1 USB VCP Problems

[gw]

Posted on December 29, 2013 at 18:56

Hi. I'm developing an Open Source project (a JavaScript interpreter) for STM32 microcontrollers (http://www.github.com/espruino/Espruino). One part of this is a bootloader that mimic's ST's serial port bootloader, but over USB VCP - however the whole project depends on USB VCP.

It's based on ST's USB VCP example code (I recently updated from V3.4.0 to V4.0.0) and running on an STM32F103, but I'm having some big problems with it.

• On all platforms, characters sent from the PC are occasionally lost.
• On Windows a significant amount of PCs seem to just stop sending/receiving data after a while and the device and program using it have to be restarted before they'll work again. It seems like USB2 ports specifically have problems, and USB3 seem more reliable.

A few things that could be related:

• When IMR_MSK contains all flags (they weren't all set for V3.4.0 of the libs), the USB device doesn't enumerate.
• After testing it seems as if characters at the end of a USB packet are lost if it is full (>64 chars). So lowering VIRTUAL_COM_PORT_DATA_SIZE makes it even more likely that data gets lost.
• Even though VIRTUAL_COM_PORT_DATA_SIZE is 64, occasionally USB_SIL_Read returns 65!!
• I'm doing flow control using:  SetEPRxStatus(ENDP3, is_rx_buffer_not_full ? EP_RX_VALID : EP_RX_NAK); - however the RX buffer does not overflow.

The relevant code for the endpoint is here:

https://github.com/espruino/Espruino/blob/master/targetlibs/stm32f1/usb/usb_endp.c

And all the code for the bootloader is in:

https://github.com/espruino/Espruino/tree/master/targets/stm32_boot

https://github.com/espruino/Espruino/tree/master/targetlibs/stm32f1/usb

https://github.com/espruino/Espruino/tree/master/targetlibs/stm32f1/usb

Please help! This is turning into a huge problem - while I can work around the lost characters by throttling the data sent over USB, I can't get around the complete loss of data when running under Windows.If it helps, there are versions of the software available for a few of the 'HY' STM32F103 boards available on eBay, and it exhibits the same problems on all of those too. If someone can help fix this I'm more than happy to send them one of the Espruino board kits that I've made by way of a thankyou! #stm32 #usb #vcp

[gw]

Posted on January 06, 2014 at 10:41

Anyone? 

Has nobody had problems with missing characters using ST's VCP implementation?

[chen]

Posted on January 06, 2014 at 11:41

Hi

''Has nobody had problems with missing characters using ST's VCP implementation?'' I have not. ''On Windows a significant amount of PCs seem to just stop sending/receiving data after a while and the device and program using it have to be restarted before they'll work again.'' Once the USB VCP session is lost - close the com port and a simple USB disconnect/re-connect and open a new serial port session should work. I have found that you can only send 64bytes at a time, anything over 64bytes is lost.

[chen]

Posted on January 06, 2014 at 11:45

Hi

Sorry it has been months since I did this work : ''I have found that you can only send 64bytes at a time, anything over 64bytes is lost.'' is not strictly true. I could not get anything over the 64byte boundary unless the host sent a CarrageReturn. Our protocol does not send CRs so we just limited the serial packets to 64bytes at a time. Do not know what is going on in the VCP but it seems you need something to trigger the VCP to send multiple USB packets for serial data > 64bytes. (It could have been that there is something we missed in the implementation! We just wanted it to work and moved on)

[gw]

Posted on January 06, 2014 at 11:49

Thanks for the reply!

''I have found that you can only send 64bytes at a time, anything over 64bytes is lost.''This sounds like my problem as well (I understand it's not quite as simple as that!). It looks like it is probably a bug in ST's software/hardware then. Perhaps someone from ST could shed some light on this?

[tsuneo]

Posted on January 06, 2014 at 21:52

> On all platforms, characters sent from the PC are occasionally lost.

> On Windows a significant amount of PCs seem to just stop sending/receiving data after a while In your code, there are two problems which cause buffer overwrite - ie. data loss. Problem 1) The bulk OUT endpoint is driven by interrupt. If host would send the next packet while the firmware is still processing the last packet, the last packet could be overwritten by the new packet. In your protocol, the bulk endpoint should be driven by polling to prevent overwrite. Problem 2) putc() puts a character to txBuffer without checking buffer full.

https://github.com/espruino/Espruino/blob/master/targets/stm32_boot/utils.c
void putc(char charData) {
txBuffer[txHead] = charData;
txHead = (txHead+1) & BUFFERMASK;
}

Anyway, your protocol over USB doesn't fit to USB age, though it might have been good for RS a) USB is packet-oriented protocol, instead, RS232 was byte-oriendted. One-byte-by-one transfer significantly reduces USB transfer performance. b) You may do without primary error check because USB has hardware error recovery. Here is the process of your CMD_READ protocol

https://github.com/espruino/Espruino/blob/master/targets/stm32_boot/main.c
case CMD_READ: // read memory
putc(ACK);
addr = getc_blocking() << 24;
addr |= getc_blocking() << 16;
addr |= getc_blocking() << 8;
addr |= getc_blocking();
chksum = getc_blocking();
// TODO: check checksum
putc(ACK);
setLEDs(2); // green = wait for data
nBytesMinusOne = getc_blocking();
chksum = getc_blocking();
// TODO: check checksum
putc(ACK);
for (i=0;i<=nBytesMinusOne;i++)
putc(((unsigned char*)addr)[i]);
setLEDs(0); // off
break;

This protocol suggests, 1) host sends CMD_READ byte 2) device returns ACK byte 3) host sends address long word with checksum 4) device returns ACK 5) host sends number of bytes to read 6) device returns ACK 7) device returns specified number of bytes For each parameter sent by host, device returns ACK. This frequent ''handshakes'' were good to detect transfer error on RS232 age. For USB age, it7s too redundant. Your protocol should be revised as follows, 1') host sends this six bytes packet to device byte offset 0 CMD_READ 1-4 address 5 number of bytes to read No checksum is required. 2') device returns the data, split in 64-bytes packets You have to know transfer-transaction relation on USB, to design better protocol over USB. Learn it on this post.

http://forum.silabs.com/ubbthreads.php?ubb=showflat&Number=42558#Post42559

Tsuneo

[gw]

Posted on January 07, 2014 at 09:42

Hi Tsuneo,

Thanks for the advice - however the bootloader protocol is designed to be the same as ST's RS232 protocol. This means that ST's existing tools will work to flash it. I'm not worried about the speed/efficiency - I'd just like it to work reliably.Do you have any more information about how you'd suggest I drive the bulk OUT endpoint by polling? I would have thought that via IRQ was better, but if the polling works better then great.As far as problem 2, putc: In the main (non-bootloader) program, there is a check for buffer overrun - however in the bootloader, txBuffer and rxBuffer are 8kBytes and the largest amount of data ever sent in one go (without waiting for a response) is 257 bytes - so there is no chance whatsoever of an overrun.I have had some luck with the lost characters now... Anis from ST suggested that my use of SetEPRxStatus to implement some kind of flow control was wrong, and that I should just always use SetEPRXValid(ENDP3) at the end of EP3_OUT_Callback (and never set it from SOF_Callback). This has completely removed flow control now, however it does stop the lost characters (for some reason).However, it still doesn't stop the device being dropped by windows after a while. You can pretty easily break it even with Putty - by just pasting text in repeatedly. The device keeps running, the COM## device still exists in Windows and SOF_Callback keeps being called, but no data can be sent or received over USB.Fiddling with the underlying hardware (for instance adding pull-down resistors) seems to sometimes improve and sometimes worsen the situation (depending on the device). My understanding is that it should be able to recover from a certain amount of USB errors - so could it be that this error recovery is not working for some reason?

[chen]

Posted on January 07, 2014 at 10:39

Hi

''Do you have any more information about how you'd suggest I drive the bulk OUT endpoint by polling? I would have thought that via IRQ was better'' Yes, the way I solved the problem was to double/tripple buffer the data from USB/VCP. The USB driver buffers the data first. Then your ISR for sending/receiving data from the VCP needs to move the data into another buffer.

[gw]

Posted on January 07, 2014 at 10:58

That is actually what I do currently - in the ISR, the data is moved into a circular buffer. In the main (non-ISR) part of the program data is then taken out of this circular buffer and used.

[gw]

Posted on January 07, 2014 at 14:19

Ok, I think we found the problem. Anis from ST pointed me at the Errata notes:  http://www.st.com/web/en/resource/technical/document/errata_sheet/CD00197763.pdf

The minimum clock speed of APB1 for reliable USB turns out to be 13Mhz, not 8Mhz as is specified in the STM32 reference!Push the APB1 clock up above this, and it all works reliably!