When I first heard that you could run nested VMs with Azure, I ran over to my laptop to deploy one of those shiny new Version 3 VMs!

Once my Host was provisioned, I got right to work.  Quickly adding the Hyper-V role and after a quick reboot, I started downloading ISOs!  And before you know it I was disappointed.  Yeah, I had a VM running, but after searching the internet for hours, I gave up.  I never could get the thing talking to the Internet.

Well, fast forward a few months and a client of mine asked if we could build a self-provisioning Nested Hyper-V Host in Azure that would pull down pre-configured VMs and start them with only one click?  I was excited. There is nothing cooler than getting to figure something out while you are getting paid.  Well, it wasn’t easy to figure out, but what I have for you here is the fruit of that labor!

If you are one of those people that likes to fast-forward to the good parts then here you go:

For those of you want that wonder what is going on under the covers, it is actually quite interesting.  This is a great example of many different types of technologies and knowledge coming together to automate complex tasks in a repeatable and insanely fast way.

This script replaced a manual process that took almost a day to build by hand, down to 15 minutes!

So, let’s take a look at what it took to build this solution because that is, in a way, even cooler than the end product which is a bang your head on the keyboard nested virtualization install process!

Plan of Attack

After thinking about this for a while, I came up with a plan of attack for this automation:

This needs to be in an ARM template.  We need to provision a Virtual Network, version 3 VM with SSD disks and a public IP address.  We will call the automation from the template and leverage the ability to have PowerShell act on our VM during the provisioning process using VM extensions.

Now during the provisioning, we are going to need a two-step automation process from a scripting perspective.  We need both a PowerShell script and a PowerShell Desired State Configuration (DSC) to get this going without any human interaction.  Remember our goal was to click on a button or execute one command and have the Host up with a running VM.

First, we need to use the PowerShell Custom Script extension to Install the Hyper-V Features and to import the xHyper-V DSC modules which allow us to configure Hyper-V.  This script will then call a reboot.  We have to do the reboot because the Hyper-V feature requires a reboot before we can call any of the PowerShell cmdlets.

Once the VM reboots, then we will call the PowerShell DSC Extension to configure the Hyper-V Host.  We will use the abilities of the DSC modules to create an Internal Switch.  Once this is completed then the script will create a new IP address and a network NAT to pass the traffic from the Nested VMs out the NIC of the Azure HOST VM.  Once this infrastructure is in place, all we need is the VM which can be downloaded from a blob storage from somewhere on the internet. After unzipping that file and placing in on the Host storage we will then simply run the New-VM Hyper-V Cmdlet.  BAM – WE have a nested VM!

Artifacts Review

So, let’s dig into what makes this all work.  It’s pretty amazing, but really there are only three text files that we need to make all of this work.

• Azure-deploy.json – ARM template which will be called to run this provisioning
• InstallHyperV.ps1 – PowerShell script which will be called first and run using the PowerShell Custom Script extension
• HyperVHostConfig.ps1 (actually packaged as a zip file) – this is the PowerShell DSC script which will be run using the PowerShell DSC Script Extension.

Azure-Deploy.json

This ARM template will create the following resources

• Virtual Machine: HYPERVHOST
• Premium Disk: HYPERVHOST_OsDisk
• Network Interface Card: HYPERVHOST-NIC
• Public IP Address: HYPERVHOST-PIP
• Virtual Network: STRGVNET

Here we see the variables section which gives you an idea of how the scripts will be called later in the resources section of the ARM template.  Notice how the PowerShell script and the DSC Zip file are called using a URL to the GitHub repo.  This could be any URL that Azure can reach on the Web or even Azure Blog Storage if the permissions are set properly.

InstallHyperV.ps1

Next, let’s look at the code in the PowerShell script.  The first part will install the NuGet package manager and then install the xHyper-V modules that will be run later by the DSC script.  After this is installed, then the Hyper-V Windows Feature is added using the Install-WindowsFeature cmdlets and then, of course, the Restart.

If you watch the deployment progress within the Azure Portal you will see the Custom script and DSC extension showing as running, transitioning and then (hopefully), Provisioning Succeeded.

The beauty of this is that even though the machine has to perform a reboot of the OS, Azure will retain the context of this provisioning process and move on to the DSC script.

HyperVHostConfig.ps1

When the machine comes back up Azure will next install and run the DSC file which is inside of the Zip file.  DSC is a different language than normal PowerShell, but you can also run PowerShell script and call cmdlets from within by using the right strategy.  So, first here we see how the xHyper-V modules that were installed in the first script are called to configure Hyper-V.

After this section has run we next see how the SetScript section will run where we can configure the New NAT Switch, adding a new IP Address and then a NAT.  Again, this is what will route our packets from any Nested VMs that are connected to this switch in Hyper-V.

Once this is complete, then you are ready to rock!

Now, I commented out the rest of the script in the GitHub repo for the automated deployment of the VM.  You can easily create a VM using Hyper-V and then zip the file up and place it in a blob to be auto-provisioned.  Just review the script and you will be able to figure it out.  The tricky part is getting the folder structure correct in your zip file for the VHD, so watch out for that.  Note, also that the IP Address for your Nested VMs will need to be on the 192.168.0.0/24 network.

 Provision HYPERVHOST and then a Nested VM

To deploy the template, click the deploy button above or over at the Repo.  This will bring you to the Azure Portal where you can update the DNS Name for your host and it will configure the HYPERVHOST for you!

Ok, now its time to build a Nested VM. Connect to HYPERVHOST using the Azure Portal.  The username and password are hardcoded in this version of the template:

User: stormtrooperio

Password: Password.!!

After you are connected, click Start and then Windows Administrative Tools, Hyper-V Manager.

Once you are in Hyper-V Manager you can create VMs.  A quick way to get started would be to download the Ubuntu Server ISO from https://www.ubuntu.com/download/server/thank-you?version=17.10.1&architecture=amd64 from within the HOST.

You will need to turn off the IE Enhanced Security Configuration from Server Manager.

After you have the ISO downloaded use these steps to get it up and running using Hyper-V Manager.  First, right-click HYPERVHOST and then click New, Virtual Machine.

Click Next, and then Provide a Name for the VM and a Location on HYPERVHOST file system for your new VM.

Stick with Generation 1 VM.

Give the VM 2 GB of RAM.

Select the NAT Switch.  Remember this was created by the PowerShell DSC xHyper-V Module.

Configure your Disk settings.

Select the Install an operating system from bootable CD/DVD-ROM and then select the ISO you downloaded. Make sure to chose the file from your download folders.

Click Finish!

Your VM will now show in the list on the HYPERVHOST.  Right-click the file and then click Settings.  Take a look at the settings, but there are no required changes.

Start the VM.

There she blows!

Double click the VM name and you will see the Ubuntu Install manager.

Follow the instructions and complete the Installer using defaults.  When you get to the network you will need to add a manual IP address.  The internal network is 192.168.0.0/24, so you can use 192.168.0.2-192.168.0.253.  Don’t worry you won’t have an issue with duplicate IPs if you use that range.  This is a private internal network only on this HYPERVHOST.  The subnet mask should be 255.255.255.0 and the default gateway is 192.168.0.1.

The DNS server should be configured as 8.8.8.8.

Follow the prompts and accept the defaults for the disk configuration.

After the Ubuntu installation has completed you will get a warning that you need to remove the ISO from the virtual DVD drive of the VM, but you can ignore this.

After the VM boots, you can login using the username and password that you provided during the install.

After the login, you can run a ping to see if the connection works.  Here I did a Ping to the Google DNS Server:

ping 8.8.8.8 -c 4

Well, I hope this post helps you get going with Nested VMs in Azure.  This could be used for almost anything you can think up with respect Dev/Test or any other number of situations.

Fork the Repo and enjoy!

https://github.com/deltadan/AzureNestedHyperV

@deltadan

About the Author