This was much simpler than I thought it would be an is easily completed by adding the ISO to the guest and running two wizards. It is only compatible with windows 10 and later so if you are running older operating systems for labs or because you are to lazy to upgrade then you out of luck.
The official KB is here: https://pve.proxmox.com/wiki/Qemu-guest-agent#Windows
Step 1
Download the virtio drivers for WindowsTM from here https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso
Step 2
Mount the ISO as a CD/DVD on your Windows guest
Step 3
Run the virtio installer virtio-win-gt-x64 or virtio-win-gt-x86 depending on your architecture
Step 4
Run the QEMU agent installer virtio-win-guest-tools
Step 5
Shutdown the guest
In Proxmox navigate to VM ID > Options > Edit
Set the QEMU Agent option to Enabled
Restart the Guest