In order to enable VSS based backups of Hyper-V VMs using Windows Server Backup, you must add the following registry keys to register the Hyper-V VSS Writer with Windows Server Backup. Note that the WindowsServerBackup key is not created when installing the Windows Server Backup feature. This key must be created manually.

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT
\CurrentVersion\WindowsServerBackup\Application Support\{66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}

Once you have the keys listed above created, you will need to create a String Value with the following.

Name: Application Identifier
Type: REG_SZ
Value: Hyper-V

Full Article