Example for Azure RTOS USBX device video (UVC)

Johannes · ‎2023-05-11

Hello everyone

Does anyone have example code for Azure RTOS USBX device class video (UVC)?

I was able to add Video to my USBX demo application on Nucleo H7A3

There was sample code for CDC_ACM and MSC (mass storage), this worked fine. But for device class video, I was not able to find any example code.

CubeIDE 1.12.1 generated two files

ux_device_video.c

ux_device_video.h

But both files only contain prototypes of callback functions which need to be filled

Like this:

/**
  * @brief  USBD_VIDEO_StreamRequest
  *         This function is invoked to manage the UVC class requests.
  * @param  video_stream: Pointer to video class stream instance.
  * @param  transfer: Pointer to the transfer request.
  * @retval status
  */
UINT USBD_VIDEO_StreamRequest(UX_DEVICE_CLASS_VIDEO_STREAM *video_stream,
                              UX_SLAVE_TRANSFER *transfer)
{
   UINT status  = UX_SUCCESS;
 
  /* USER CODE BEGIN USBD_VIDEO_StreamRequest */
  UX_PARAMETER_NOT_USED(video_stream);
  UX_PARAMETER_NOT_USED(transfer);
  /* USER CODE END USBD_VIDEO_StreamRequest */
 
  return status;
}

How to fill it, that it works?

Thanks

Johannes

Johannes · ‎2023-05-16

I got a solution.

for some bizare unknown reason, it works if I change the order of the device classes.

If I change the enumeration of the device classes, so Video comes first, it works.

uint8_t UserClassInstance[USBD_MAX_CLASS_INTERFACES] = {
  CLASS_TYPE_VIDEO,
  CLASS_TYPE_MSC,
  CLASS_TYPE_CDC_ACM,
};

Now I have a compound device with

Video
Mass storage
virtual serial port

If anyone can explain, why that is, ... highly appreciated.

Johannes

mohamed.ayed · ‎2023-05-16

Hi @Johannes, as i see in your project app_usbx_device.c Tx Fifo of endpoint 4 ( EP video IN) is missed, you can see in referance project

https://github.com/STMicroelectronics/x-cube-azrtos-h7/blob/f25d657a4de4aac6dba48df28b96238479d73ca0/Projects/NUCLEO-H723ZG/Applications/USBX/Ux_Device_Video/USBX/App/app_usbx_device.c#L245

mohamed.ayed · ‎2023-05-16

Hi @Johannes , As i see in your project TxFifo Config of endpoint video (EP 4 - IN) is missed, you can check it in reference application:

https://github.com/STMicroelectronics/x-cube-azrtos-h7/blob/f25d657a4de4aac6dba48df28b96238479d73ca0/Projects/NUCLEO-H723ZG/Applications/USBX/Ux_Device_Video/USBX/App/app_usbx_device.c#L245

Johannes · ‎2023-05-16

@mohamed.ayed : I must admit, I don't understand:

If I have 4 endpoints (IN) I need one TX fifo for each endpoint?

How do these fifo indexes correspond to the endpoints?

  HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 0, 0x10);
 
  /* Set Tx FIFO 2 */
  HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 1, 0x10);
 
  /* Set Tx FIFO 3 */
  HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 2, 0x20);
 
  /* Set Tx FIFO 4 */
  HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 3, 0x80);

mohamed.ayed · ‎2023-05-16

in your project there is 5 endpoint IN and 3 endpoint OUT you can see it in ux_device_descriptor.h

endpoint IN

endpoint 0 (mandatory for each USB device ) 0x80U
#define USBD_MSC_EPIN_ADDR 0x81U
#define USBD_CDCACM_EPINCMD_ADDR 0x82U
#define USBD_CDCACM_EPIN_ADDR 0x83U
#define USBD_VIDEO_EPIN_ADDR 0x84U

endpoint OUT

endpoint 0 (mandatory for each USB device ) 0x00U
#define USBD_MSC_EPOUT_ADDR 0x01U
#define USBD_CDCACM_EPOUT_ADDR 0x03U

endpoint 1 and endpoint 3 is bidirectional

So for OUT endpoint size is regrouped in RxFiFo:

 HAL_PCDEx_SetRxFiFo(&hpcd_USB_OTG_HS, 0x200);

For IN endpoint size TxFiFo:

 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 0, 0x10);
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 1, 0x10);
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 2, 0x20);
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 4, 0x80);

please note that real value is multiplied x4

for example for RxFifo we have 0x200 so value is 512 *4 = 2048

In HS mode size of PCD USB fifo (sum of fifo value) sould be < 4096

Johannes · ‎2023-05-16

@mohamed.ayed : thank you for explaining.

Sorry, but I am still confused.

I have 5 IN-endpoints. So I need 5 TX Fifos?

does the Fifo number in HAL_PCDEx_setTxFifo have to match the endpoint number?

 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 0, 0x10); //->0x80
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 1, 0x10); //->0x81
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 2, 0x20); //->0x82
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 3, 0x20);  //->0x83
 HAL_PCDEx_SetTxFiFo(&hpcd_USB_OTG_HS, 4, 0x80);  //->0x84

Thank you for your help

Johannes

Johannes · ‎2023-05-16

I tried to more TX fifos like described above. This does not solve the problem with the video device.

I still need to move the video endpoint to the top of the list of endpoint descriptors to make it work

uint8_t UserClassInstance[USBD_MAX_CLASS_INTERFACES] = {
  CLASS_TYPE_VIDEO,
  CLASS_TYPE_MSC,
  CLASS_TYPE_CDC_ACM,
};

mohamed.ayed · ‎2023-05-16

@Johannes Yes, if you see HAL_PCDEx_SetTxFiFo API description fifo parameter (The number of Tx fifo), this is the endpoint Number .

https://github.com/STMicroelectronics/x-cube-azrtos-h7/blob/f25d657a4de4aac6dba48df28b96238479d73ca0/Drivers/STM32H7xx_HAL_Driver/Src/stm32h7xx_hal_pcd_ex.c#L70

Regards,

Mohamed

Johannes · ‎2023-05-30

I was able to implement the video demo

https://github.com/STMicroelectronics/x-cube-azrtos-h7/tree/main/Projects/NUCLEO-H723ZG/Applications/USBX/Ux_Device_Video

into my code using Nucleo H7A3

Does anyone know, how the demo data for the video is generated or encoded?

see file "stream1.h"

Are these "just" JPEG images put into a sequence of 28 images?

 
const  uint8_t  *tImagesList[] = {image0, image1, image2, image3, image4, image5, image6, image7, image8, image9, \
                                  image10, image11, image12, image13, image14, image15, image16, image17, \
                                  image18, image19, image20, image21, image22, image23, image24, image25, \
                                  image26, image27, image28};
 
uint16_t tImagesSizes[] = {JPEG0_SIZE, JPEG1_SIZE, JPEG2_SIZE, JPEG3_SIZE, JPEG4_SIZE, JPEG5_SIZE, JPEG6_SIZE, \
                           JPEG7_SIZE, JPEG8_SIZE, JPEG9_SIZE, JPEG10_SIZE, JPEG11_SIZE, JPEG12_SIZE, \
                           JPEG13_SIZE, JPEG14_SIZE, JPEG15_SIZE, JPEG16_SIZE, JPEG17_SIZE, JPEG18_SIZE, \
                           JPEG19_SIZE, JPEG20_SIZE, JPEG21_SIZE, JPEG22_SIZE, JPEG23_SIZE, JPEG24_SIZE, \
                           JPEG25_SIZE, JPEG26_SIZE, JPEG27_SIZE, JPEG28_SIZE};

mohamed.ayed · ‎2023-05-31

Hi @Johannes, To have encoded video.

1- split your video into images with ffmpeg libraries (see https://ffmpeg.org)

2- select your images and convert them to hex array you can use free tool for that.

BR

Mohamed