LXD and VMs


Linux Containers now with VMs

by Craig Miller

Linux Containers are pretty cool technology, which I have spoken of before. But as of version 4.0, LXD also supports Virtual Machines (VM) as well.

It does this fairly transparently by just adding --vm at the end of the launch command.

LXD+VM Requirements

LXD does not run VMs natively, it only orchestrates the VM, and manages it via the standard lxc commands.

In order to run a Virtual Machine, an emulator (qemu) and Virtualizing extensions (KVM aka libvirt) are required. On my Artix distribution (like Arch but without systemd), I added those packages:

pacman -S qemu libvirt ovmf

VM Images ready to be started

Ubuntu builds linux container images at images.linuxcontainers.org. In there you will find images for not just Ubuntu, but also Fedora, Debian, Alpine, for many architectures, including arm64 and amd64 (aka x86_64). Since version 4.0, they have also put up VM images as well as the light-weight container images.

VM images are much larger than container images. Usually 10x or more. There's a reason why linux containers are considered light-weight.

Why run a VM when you can run a Container?

Good question. A container just contains the user-space part of a distribution, and though some partitioning mechanisms (like cgroups), the common kernel (of the host) keeps it all separate.

But what if the host OS doesn't have all the support items you need? By running a Virtual Machine, you are running an entire OS, including all layers of software that may be required by your application.

I found this out, when I tried to run LXD Mosaic (a web GUI for LXD). It had hidden dependencies on systemd. Since Artix is systemd-free, LXD Mosaic would not run inside a container.

As systemd worms its way more and more into the Linux distros, many other things become dependent upon its wide-ranging services. For example, Ubuntu 20.04 won't even boot as a container on a systemd-free distro. But it will boot as a VM.

Starting up the VM in LXD

Now it is time to start up our VM. LXD did a excellent job of integration into their framework. To start a container one would use the command:

lxc launch images:alpine/edge vm1

To launch it as a VM, just add --vm at the end:

lxc launch images:alpine/edge vm1 --vm

If all goes well, then a lxc ls command will show the VM up and running.

$ lxc ls
| NAME |  STATE  |       IPV4       |                     IPV6                      |      TYPE       | SNAPSHOTS |
| vm1  | RUNNING | (eth0) | fd10:111::216:3eff:fe99:816d (eth0)           | VIRTUAL-MACHINE | 0         |
|      |         |                  | 2001:eb8:8011:fd00:216:3eff:fe99:816d (eth0)  |                 |           |
| lc   | RUNNING | (eth0) | fd10:111::216:3eff:fe03:3d0f (eth0)           | CONTAINER       | 1         |
|      |         |                  | 2001:eb8:8011:fd00:216:3eff:fe03:3d0f (eth0)  |                 |           |

It is possible to get into the VM as one would do with a container:

$ lxc exec vm1 sh
~ # uname -a
Linux localhost 5.4.81-0-virt #1-Alpine SMP Thu, 03 Dec 2020 08:45:06 UTC x86_64 Linux
~ # 

Looking at my host OS

$ uname -a
Linux koa 5.9.6-artix1-1 #1 SMP PREEMPT Fri, 06 Nov 2020 12:04:06 +0000 x86_64 GNU/Linux

As you can see the VM is running a different kernel version from the host OS.


LXD continues to be an excellent framework supporting light-weight containers, but as of version 4.x, it also supports VMs. For those times when you need the complete baggage of a VM.

5 December 2020