Hyper-V on Server 2016

There are some new and changed functionalities of Hyper-V on Windows Server 2016 and Microsoft Hyper-V Server 2016. To use new features on virtual machines created with Windows Server 2012 R2 and moved or imported to a server that runs Hyper-V on Windows Server 2016, you’ll need to manually upgrade the virtual machine configuration version. Taken from TechNet, here’s what’s included in this article and whether the functionality is new or updated.

Compatible with Connected Standby (new)

When the Hyper-V role is installed on a computer that uses the Always On/Always Connected (AOAC) power model, the Connected Standby power state is now available.

Discrete device assignment (new)

This feature lets you give a virtual machine direct and exclusive access to some PCIe hardware devices. Using a device in this way bypasses the Hyper-V virtualization stack, which results in faster access.

Encryption support for the operating system disk in generation 1 virtual machines (new)

You can now protect the operating system disk using BitLocker drive encryption in generation 1 virtual machines. A new feature, key storage, creates a small, dedicated drive to store the system drive’s BitLocker key. This is done instead of using a virtual Trusted Platform Module (TPM), which is available only in generation 2 virtual machines. To decrypt the disk and start the virtual machine, the Hyper-V host must either be part of an authorized guarded fabric or have the private key from one of the virtual machine’s guardians. Key storage requires a version 8 virtual machine.

Host resource protection (new)

This feature helps prevent a virtual machine from using more than its share of system resources by looking for excessive levels of activity. This can help prevent a virtual machine’s excessive activity from degrading the performance of the host or other virtual machines. When monitoring detects a virtual machine with excessive activity, the virtual machine is given fewer resources. This monitoring and enforcement is off by default. Use Windows PowerShell to turn it on or off. To turn it on, run this command:

Set-VMProcessor -EnableHostResourceProtection $true

Hot add and remove for network adapters and memory (new)

You can now add or remove a network adapter while the virtual machine is running, without incurring downtime. This works for generation 2 virtual machines that run either Windows or Linux operating systems.

You can also adjust the amount of memory assigned to a virtual machine while it’s running, even if you haven’t enabled Dynamic Memory. This works for both generation 1 and generation 2 virtual machines, running Windows Server 2016 or Windows 10.

Hyper-V Manager improvements (updated)

  • Alternate credentials support – You can now use a different set of credentials in Hyper-V Manager when you connect to another Windows Server 2016 or Windows 10 remote host. You can also save these credentials to make it easier to log on again.
  • Manage earlier versions – With Hyper-V Manager in Windows Server 2016 and Windows 10, you can manage computers running Hyper-V on Windows Server 2012, Windows 8, Windows Server 2012 R2 and Windows 8.1.
  • Updated management protocol – Hyper-V Manager now communicates with remote Hyper-V hosts using the WS-MAN protocol, which permits CredSSP, Kerberos or NTLM authentication. When you use CredSSP to connect to a remote Hyper-V host, you can do a live migration without enabling constrained delegation in Active Directory. The WS-MAN-based infrastructure also makes it easier to enable a host for remote management. WS-MAN connects over port 80, which is open by default.

Integration services delivered through Windows Update (updated)

Updates to integration services for Windows guests are distributed through Windows Update. For service providers and private cloud hosters, this puts the control of applying updates into the hands of the tenants who own the virtual machines. Tenants can now update their Windows virtual machines with all updates, including the integration services, using a single method.


The vmguest.iso image file is no longer needed, so it isn’t included with Hyper-V on Windows Server 2016.

Linux Secure Boot (new)

Linux operating systems running on generation 2 virtual machines can now boot with the Secure Boot option enabled. Ubuntu 14.04 and later, SUSE Linux Enterprise Server 12 and later, Red Hat Enterprise Linux 7.0 and later, and CentOS 7.0 and later are enabled for Secure Boot on hosts that run Windows Server 2016. Before you boot the virtual machine for the first time, you must configure the virtual machine to use the Microsoft UEFI Certificate Authority. You can do this from Hyper-V Manager, Virtual Machine Manager, or an elevated Windows Powershell session. For Windows PowerShell, run this command:

Set-VMFirmware vmname -SecureBootTemplate MicrosoftUEFICertificateAuthority

More memory and processors for generation 2 virtual machines and Hyper-V hosts (updated)

Starting with version 8, generation 2 virtual machines can use significantly more memory and virtual processors. Hosts also can be configured with significantly more memory and virtual processors than were previously supported. These changes support new scenarios such as running e-commerce large in-memory databases for online transaction processing (OLTP) and data warehousing (DW). The Windows Server blog recently published the performance results of a virtual machine with 5.5 terabytes of memory and 128 virtual processors running 4 TB in-memory database. Performance was greater than 95% of the performance of a physical server.

Nested virtualization (new)

This feature lets you use a virtual machine as a Hyper-V host and create virtual machines within that virtualized host. This can be especially useful for development and test environments. To use nested virtualization, you’ll need:

  • To run at least Windows Server 2016 or Windows 10 on both the physical Hyper-V host and the virtualized host.
  • A processor with Intel VT-x (nested virtualization is available only for Intel processors at this time).

Networking features (new)

New networking features include:

  • Remote direct memory access (RDMA) and switch embedded teaming (SET). You can set up RDMA on network adapters bound to a Hyper-V virtual switch, regardless of whether SET is also used. SET provides a virtual switch with some of same capabilities as NIC teaming.
  • Virtual machine multi queues (VMMQ). Improves on VMQ throughput by allocating multiple hardware queues per virtual machine. The default queue becomes a set of queues for a virtual machine, and traffic is spread between the queues.
  • Quality of service (QoS) for software-defined networks. Manages the default class of traffic through the virtual switch within the default class bandwidth.

Production checkpoints (new)

Production checkpoints are “point-in-time” images of a virtual machine. These give you a way to apply a checkpoint that complies with support policies when a virtual machine runs a production workload. Production checkpoints are based on backup technology inside the guest instead of a saved state. For Windows virtual machines, the Volume Snapshot Service (VSS) is used. For Linux virtual machines, the file system buffers are flushed to create a checkpoint that’s consistent with the file system. If you’d rather use checkpoints based on saved states, choose standard checkpoints instead.


New virtual machines use production checkpoints as the default.

Rolling Hyper-V Cluster upgrade (new)

You can now add a node running Windows Server 2016 to a Hyper-V Cluster with nodes running Windows Server 2012 R2. This allows you to upgrade the cluster without downtime. The cluster runs at a Windows Server 2012 R2 feature level until you upgrade all nodes in the cluster and update the cluster functional level with the Windows PowerShell cmdlet, Update-ClusterFunctionalLevel.


After you update the cluster functional level, you can’t return it to Windows Server 2012 R2.

For a Hyper-V cluster with a functional level of Windows Server 2012 R2 with nodes running Windows Server 2012 R2 and Windows Server 2016, note the following:

  • Manage the cluster, Hyper-V, and virtual machines from a node running Windows Server 2016 or Windows 10.
  • You can move virtual machines between all of the nodes in the Hyper-V cluster.
  • To use new Hyper-V features, all nodes must run Windows Server 2016 and the cluster functional level must be updated.
  • The virtual machine configuration version for existing virtual machines isn’t upgraded. You can upgrade the configuration version only after you upgrade the cluster functional level.
  • Virtual machines that you create are compatible with Windows Server 2012 R2, virtual machine configuration level 5.

After you update the cluster functional level:

  • You can enable new Hyper-V features.
  • To make new virtual machine features available, use the Update-VmConfigurationVersion cmdlet to manually update the virtual machine configuration level.
  • You can’t add a node to the Hyper-V Cluster that runs Windows Server 2012 R2.

Hyper-V on Windows 10 doesn’t support failover clustering.


Shared virtual hard disks (updated)

You can now resize shared virtual hard disks (.vhdx files) used for guest clustering, without downtime. Shared virtual hard disks can be grown or shrunk while the virtual machine is online. Guest clusters can now also protect shared virtual hard disks by using Hyper-V Replica for disaster recovery.

Enable replication on the collection. Enabling replication on a collection is only exposed through the WMI interface. . You cannot manage replication of a collection through PowerShell cmdlet or UI. The VMs should be on hosts that are part of a Hyper-V cluster to access features that are specific to a collection. This includes Shared VHD – shared VHDs on stand-alone hosts are not supported by Hyper-V Replica.

A collection with a shared VHD but no associated guest cluster cannot create reference points for the collection (regardless of whether the shared VHD is included in the reference point creation or not).

Shielded virtual machines (new)

Shielded virtual machines use several features to make it harder for Hyper-V administrators and malware on the host to inspect, tamper with, or steal data from the state of a shielded virtual machine. Data and state is encrypted, Hyper-V administrators can’t see the video output and disks, and the virtual machines can be restricted to run only on known, healthy hosts, as determined by a Host Guardian Server.


As of Technical Preview 5, shielded virtual machines are compatible with Hyper-V Replica. To replicate a shielded virtual machine, the host you want to replicate to must be authorized to run that shielded virtual machine.

Start order priority for clustered virtual machines (new)

This feature gives you more control over which clustered virtual machines are started or restarted first. This makes it easier to start virtual machines that provide services before virtual machines that use those services. Define sets, place virtual machines in sets, and specify dependencies. Use Windows PowerShell cmdlets to manage the sets, such as New-ClusterGroupSet, Get-ClusterGroupSet, and Add-ClusterGroupSetDependency. .

Storage quality of service (QoS) (updated)

You can now create storage QoS policies on a Scale-Out File Server and assign them to one or more virtual disks on Hyper-V virtual machines. Storage performance is automatically readjusted to meet policies as the storage load fluctuates.

Virtual machine configuration file format (updated)

Virtual machine configuration files use a new format that makes reading and writing configuration data more efficient. The format also makes data corruption less likely if a storage failure occurs. Virtual machine configuration data files use a .vmcx file name extension and runtime state data files use a .vmrs file name extension.


The .vmcx file name extension indicates a binary file. Editing .vmcx or .vmrs files isn’t supported.

Virtual machine configuration version (updated)

The version represents the compatibility of the virtual machine’s configuration, saved state, and snapshot files with the version of Hyper-V. Virtual machines with version 5 are compatible with Windows Server 2012 R2 and can run on both Windows Server 2012 R2 and Windows Server 2016 . Virtual machines with versions introduced in Windows Server 2016 won’t run in Hyper-V on Windows Server 2012 R2.

If you move or import a virtual machine to a server that runs Hyper-V on Windows Server 2016 from Windows Server 2012 R2, the virtual machine’s configuration isn’t automatically updated. This means you can move the virtual machine back to a server that runs Windows Server 2012 R2. But, this also means you can’t use the new virtual machine features until you manually update the version of the virtual machine configuration.


  • After you update the version, you can’t move the virtual machine to a server that runs Windows Server 2012 R2.
  • You can’t downgrade the configuration to a previous version.
  • The Update-VMVersion cmdlet is blocked on a Hyper-V Cluster when the cluster functional level is Windows Server 2012 R2.

Virtualization-based security for generation 2 virtual machines (new)

Virtualization-based security powers features such as Device Guard and Credential Guard, offering increased protection of the operating system against exploits from malware. Virtualization based-security is available in generation 2 guest virtual machines starting with version 8.

Windows Containers (new)

Windows Containers allow many isolated applications to run on one computer system. They’re fast to build and are highly scalable and portable. Two types of container runtime are available, each with a different degree of application isolation. Windows Server Containers use namespace and process isolation. Hyper-V Containers use a light-weight virtual machine for each container.

Key features include:

  • Support for web sites and applications using HTTPS
  • Nano server can host both Windows Server and Hyper-V Containers
  • Ability to manage data through container shared folders
  • Ability to restrict container resources

Windows PowerShell Direct (new)

This gives you a way to run Windows PowerShell commands in a virtual machine from the host. Windows PowerShell Direct runs between the host and the virtual machine. This means it doesn’t require networking or firewall requirements, and it works regardless of your remote management configuration.

Windows PowerShell Direct is an alternative to the existing tools that Hyper-V administrators use to connect to a virtual machine on a Hyper-V host:

  • Remote management tools such as PowerShell or Remote Desktop
  • Hyper-V Virtual Machine Connection (VMConnect)

Those tools work well, but have trade-offs: VMConnect is reliable, but can be hard to automate. Remote PowerShell is powerful, but can be hard to set up and maintain. These trade-offs may become more important as your Hyper-V deployment grows. Windows PowerShell Direct addresses this by providing a powerful scripting and automation experience that’s as simple as using VMConnect.


Hyper-V improvements on Server 2012 R2

Hyper-V Advanced Options

Some of Hyper-V improvements on Server 2012 R2 are really going to make your life easier;

  • Compression option for Live Migration; Memory of the virtual machine being migrated is compressed and then copied over the network to the destination over a TCP/IP connection, resulting in huge performance improvements of Live Migrations. Typically live migration time with compression is a fifth of the time than when no compression is used.
  • SMB option for Live Migration; Memory of the virtual machine being migrated is compressed and then copied over the network to the destination over a SMB connection. Uses SMB Direct (RDMA). This doesn’t use compression since the processor is bypassed when using RDMA, and this gives the greatest Live Migration experience–22 seconds (RDMA) versus 38 seconds (memory compression).
  • Hyper-V Replica: Frequency of replication was only every five minutes, and now we have got a choice of 30 seconds, five minutes, and 15 minutes.
  • Extended Replication: Ability to have a Hyper-V Replica virtual machine (VM) replicated to another Hyper-V server on another site for extended disaster recovery (DR) capabilities. We can now extend this replication over to a third site.
  • Hyper-V Recovery Manager provides Windows Azure-based service to manage all Hyper-V Replication within an environment. This is just orchestration of the process and the actual replication of VMs is still utilized via the Hyper-V Replica functionality. Actual replication is still direct site-to-site and not via Windows Azure. Hyper-V Recovery Manager provides full control of the order of failover of VMs in addition to the running of scripts and even manual actions as part of the failover process.
  • Support for deduplication of VHDXs, which actually improves the performanceof the VMs (almost twice the speed). With deduplication, the process knows what the common blocks are, so it enables better caching of the most common blocks for running VMs.
  • Dynamic VHDX resizing when connected to the SCSI bus, allowing VHDX files to be resized while a VM is running, and then within a VM you can easily expand volumes to use the newly available space. It’s also possible to shrink a virtual disk, provided unpartioned space is available on a disk.
  • Support for shared VHDX files by VMS, enabling new guest clustering scenariosby allowing multiple VMs to access the same VHDX and see the VHDX as shared storage. The shared VHDX is exposed to a VM as a virtual shared SAS disk and the VHDX file can be dynamic or fixed. You can specify a disk as shared via Windows PowerShell or via the Advanced Features when adding a disk to a VM. The ability to provide a shared VHDX file is very useful, particularly in hosting environments where you would not directly expose fibre channel or iSCSI LUNs to clients (the Windows Server 2012 method to provide shared storage to VMs).
  • VM Connection improvements, allowing copy and paste via VM Connection plus audio support/printer/smart card redirection as Remote Access is now via VMBus. This is a very useful feature if a VM loses network connectivity or RDP is blocked via a firewall so administrators can’t RDP to a VM. With the new VMbus-based connection, administrators always have full access to VMs at a console level.
  • Automatic activation of VMs that are running on an activated Windows Server 2012 R2 Datacenter server (no specific channel version required). The VM doesn’t have a key at all, so no key management is required.
  • New Generation 2 VM:
    – Generation 2 VMs use UEFI, have secure boot capability, and can boot from SCSI devices and synthetic network adapters.
    – Generation 2 VMs are Windows 8/Windows 2012 or later and 64-bit only. This is because the OS needs native UEFI and must ship with Hyper-V integration drivers in-box.
    – Generation 2 provides a faster boot and install experience. Day-to-day operations are about the same.
    – It is fully supported to mix Generation 1 and Generation 2 VMs on the same host.
  • Virtual Receive Side Scaling (RSS) allows a combination of RSS and VMQ (which were mutually exclusive in Windows Server 2012) and allows a VMQ to no longer be linked to a single core, giving greater performance by spreading loads across cores. Uses a RSS hash to spread traffic processing across multiple cores.
  • Resource metering monitors incoming and outgoing storage IOPs in addition to existing CPU, memory, disk allocation, and network traffic.
  • Storage QoS allows a maximum IOPs cap for each VHDX of a VM (even when running). Minimum QoS alerting when a virtual machine disk isn’t getting required IOPs.
  • Full Linux support including dynamic memory support (add and remove), live backup (file consistency through new file freeze in Linux integration services), 64 vCPU SMP, virtual SCSI, and hot-add/resize of storage.
  • Clone a running VM and export a checkpoint (which generates a merged virtual disk for the export).
  • Cluster Shared Volumes coordinators automatically rebalanced across all nodes in the cluster.
  • Use ReFS with CSV.
  • Easier virtual network management (Software Defined Networking) and additional capabilities including in-box gateway functionality to link different virtual networks even across hybrid clouds.
  • Simple remote live monitoring of network traffic through a new graphical experience using Message Analyzer, which can collect remote and local packets.
  • Enhanced Hyper-V Extensible Switch architecture to enable coexistence with forwarding extension implementations, which previously couldn’t work with Hyper-V extensions.
  • USB pass-through, which allows USB device pass-through within certain conditions.
  • Windows Azure Pack (formally known as Windows Azure Services for Windows Server). Takes the innovation from Azure and brings it to Windows Server and System Center. Consistent portal experience to match Azure, high density web hosting, and Service bus