(Disclaimer: I work at gridscale.)

OpenBSD 6.4 with nested virtualization at gridscale

Since OpenBSD 6.4 was released the other day I wanted to give it a shot on our platform. Especially since we offer nested virtualization support (as a beta) I wanted to see how well the vmm(4) subsystem works on that.

OpenBSD 6.4

Creating a new IaaS-Instance I run with the defaults - except for the hardware profile set to Nested - activate Nested Virtualization (BETA). This becomes visible when you open the advanced settings.

The OpenBSD install runs smoothly. Check the dmesg here.

Nested VT

Once OpenBSD is installed, we will go ahead and give vmd(8) a shot. We grab the install64.fs as well as the bsd.rd file.

$ ftp https://ftp.bytemine.net/pub/OpenBSD/6.4/amd64/install64.f
$ ftp https://ftp.bytemine.net/pub/OpenBSD/6.4/amd64/bsd.rd

Then we prepare the host machine. For outbound NAT to work (to NAT the traffic from our guest vm running inside vmd), we add the following rule to /etc/pf.conf:

ext_if="vio0"
pass out on $ext_if from 100.64.0.0/10 to any nat-to $ext_if

And flip the sysctl responsible for ip forwarding: ip.forwarding sysctl:

doas sysctl net.inet.ip.forwarding=1

(and hardwire it in /etc/sysctl.conf to make it stick through reboots)

$ doas /usr/sbin/vmd

(add vmd_flags="" to /etc/rc.conf.local to start vmd(8) upon booting)

Create the disk for our guest and then start the vm to install. The -L-option will enable the NAT’ed networking inside the vm. Since we pass the -c option, we will connect to the serial console. This spins up a VM that boots into bsd.rd, with 1 gigabyte of main memory and two drives: the install media as well as the hard disk created in the previous step.

$ doas vmctl create host.drive -s 8G
$ doas vmctl start "vm1" -c -b bsd.rd -m 1G -L -d host.drive -d install64.fs

The corresponding vm.conf looks like this:

vm "vm1" {
    memory 1G
    disk "/data/vms/host.drive"
	local interface tap
}

Again we do a regular OpenBSD install. The dmesg within the vm is here.

As one can gather from this rather short post - getting a vm up and running with vmd(8) is a very nice experience. As usual for OpenBSD the default configuration is well thought through and now weird bells and whistles added.