I haven't tried CubeMonitor-RF in 802.15.4 mode yet, BLE only... But I'll tell you what I know/understood.
1) please read the STM32CubeMonitor RF on the Cube monitor RF. https://www.st.com/resource/en/user_manual/dm00433646-stm32cubemonitorrf-software-tool-for-wireless-performance-measurements-stmicroelectronics.pdf
2) there is no VCP "transparent mode" binary yet for the USB dongle as for BLE, so that means you'll want to use same UART that the nucleo is using in Cubemonitor RF config.
3) There are 2 modes, standard mode and sniffer mode, and it seems like it needs 2 different stacks.
4) for standard mode, the documentation is refering to stm32wb5x_rfmonitor_phy802_15_4_fw.bin and it has been removed from latest firmware packages (no idea why)... But I have last seen it in STM32Cube_FW_WB_V1.8.0\Projects\STM32WB_Copro_Wireless_Binaries\STM32WB5x
5) Documentation is not up to date.
Good luck