Does anyone know of documentation/reference manual that describes exactly how the ST-Link USB driver works on Microsoft Windows so that a custom Windows application can be written to communicate with the ST-Link USB driver to download code to a STM32H753?

[Tesla DeLorean]

Not sure ST has documented this, nor have a desire to provide support in furtherance of that. Basically it is complicated, and if you aren't working for IAR, Keil, Rowley, etc it is probably a heavy lift.

You could disassemble/review the DLL interfaces.

You could discuss with your local sales/support reps, or FAE, and gauge their keenness to support your efforts.

You could look at the means/methods used by GDB and OpenOCD, etc.

[Pavel A.]

This driver is based on WinUSB technology.

It is described in details in the Windows SDK/WDK documentation.

Unfortunately Microsoft decided to destroy their well known MSDN and Technet forums, so Stack Overflow is now the best place to get help on this.

-- pa