Container Networking & the IPv6 Advantage

Wireshark

IPv6 Containers on the Network

by Craig Miller

I was recently giving a talk on Container Networking, which included LXD/Incus (Incus is the community fork of LXD), Docker and Podman.

One can think of Linux Containers as light weight virtual machines, however, they share the same Linux kernel as the host. Docker of course, is the most popular container framework, but there are others which accomplish the isolation of a specific application, or an entire file system from the host operating system.

It has been a few years, since I have played with Docker, preferring LXD because of its much better IPv6 support. To my pleasant surprise, the Docker proxy now supports IPv6, as well as IPv4.

Docker Manual IPv6 Configuration

I have been unimpressed with Docker's IPv6 support in the past. When it was added, it was all manual, like a throwback to the 1990s before DHCP, requiring the configuration of address, mask, default gateway, DNS, etc. There was no easy way to have a container pick up your router's RA (Router Advertisment) and auto configure a SLAAC (Stateless Address Auto Configuration). It was, and is still a very manual operation to get IPv6 going in Docker, and that is assuming that the Container you have selected from Docker Hub even supports IPv6.

The Docker Bridge (Really a Proxy)

Docker has had for some time a component they call a "bridge". In Classical networking a bridge is a Layer 2 device, that forwards packets based on destination MAC address. This is not how the Docker Bridge works.

The Docker bridge includes input/output port mapping, and even IP protocol conversion (IPv6<->IPv4). More on this later.

The advantage/disadvantage of the Docker framework bridge is that it creates isolation of the container from the network. For example, only certain TCP/UDP ports are forwarded into the Container.

Starting a Simple Docker Container

As you will find there are thousands of pre-made Docker containers on Docker Hub. Let's start with a relatively simple one, called containous/whoami. It just reports the IP address of the container and the remote IP address of the HTTP request.

Assuming you have Docker installed and running on your host (I used a Raspberry Pi), start this simple container with the following command:

docker run -d -p 8888:80 --name iamfoo containous/whoami

The -p parameter maps the container's port 80 to an external (on the host side) port of 8888.

Querying Your Docker Container

Now that you have whoami running, point your Web browser at the IP address of your host, including port 8888, Or use curl to see the output of whoami, like this:

$ curl http://192.168.1.100:8888/

In the above request, the host's IP address is 192.168.1.100. Great, you can see that whoami has the following output:

Hostname: 2ba0cb4e37c0
IP: 127.0.0.1
IP: ::1
IP: 172.17.0.2
RemoteAddr: 172.17.0.1:44898
GET / HTTP/1.1
Host: 192.168.1.100:8888
User-Agent: curl/7.81.0
Accept: */*

As you can see the remote address (RemoteAddr) is listed as 172.17.0.1. Clearly the Docker Proxy is doing NAT (Network Address Translation).

Docker Proxy Now Supports IPv6

Not sure in which version, but in the past few years, the Docker Proxy now also opens a listening socket on IPv6, in the above example on port 8888. netstat will show the open port on IPv6

$ sudo netstat -antp
...
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 :::8888                 :::*                    LISTEN      4932/docker-proxy

Querying Your Docker Container Over IPv6

We do a similar web request using curl but this time, we'll use the IPv6 address of the host. Since we are using a bare IPv6 address, it must be wrapped in square brackets.

$ curl http://[2001:db8::100]:8888/

Hostname: 2ba0cb4e37c0
IP: 127.0.0.1
IP: ::1
IP: 172.17.0.2
RemoteAddr: 172.17.0.1:45704
GET / HTTP/1.1
Host: [2001:db8::100]:8888
User-Agent: curl/7.81.0
Accept: */*

Congratulations, we just made a HTTP request over IPv6 to a Docker Container without having to do all the manual IPv6 configuration.

Limitations to the Docker Proxy IPv6 Support

The biggest limitation of this method, is that each Docker Container must be mapped to a unique listening port, and some how this special port has to be communicated to the user.

For example, if I want to run three (3) Docker Containers on a host all running some type of web application, I would have to map each container to a unique port, such as port 8881, 8882, 8883, and then tell everyone that to access web service 2 they have to add :8882 to the address. Not the best solution.

Utilizing an IPv6 Advantage

As an IPv6 advocate, I think IPv6 has many advantages. But even the stalwart IPv4-forever folks realize that IPv6 has many, many more addresses than IPv4.

And we can use this advantage with Docker Containers, because we can bind a container to a specific address, rather than listening to :::8888 (from netstat command above).

Binding a Container to a Specific Address

By specifying an address as part of the port mapping -p parameter, the Docker Proxy will bind, or open a listening socket to that specific address.

First we need to kill the previous whoami container, and delete it (using the docker container ID).

docker stop 2ba0cb4e37c0
docker remove 2ba0cb4e37c0

Then we will restart it providing a specific IPv6 address. Again we have to enclose the IPv6 address in square brackets, and then quotes to keep the shell from doing odd things with our square brackets

docker run -d -p "[2001:db8::100]:80:80" --name iamfoo containous/whoami

Now the docker ps command output looks like:

$ docker ps
CONTAINER ID   IMAGE               COMMAND     CREATED          STATUS         PORTS                      NAMES
d0800e8d1896   containous/whoami   "/whoami"   11 seconds ago   Up 9 seconds   2001:db8::100:80->80/tcp   iamfoo

And netstat shows:

$ sudo netat -antp
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 2001:db8::100:80        :::*                    LISTEN      4176/docker-proxy

Using the IPv6 Advantage

Of course, some would say what we did above is retrograde motion since we had a dual stack container listening on port 80, and we have made it a container listening only to IPv6, and you would be right.

You might argue, however, that we still can only have one web service running on port 80 (or 443).

This is where the IPv6 advantage comes in. We can have many, many IPv6 addresses on our host. For example, we might have ten (10) IPv6 addresses on the host, after all, there is no shortage of IPv6 addresses. So we might assign the following to our host:

2001:db8::100
2001:db8::200
2001:db8::201
2001:db8::202
...
2001:db8::208
2001:db8::209

Of course, you would set up DNS AAAA entries for each of the additional IPv6 addresses, something like:

web1    IN  AAAA    2001:db8::201
web2    IN  AAAA    2001:db8::202
...
web9    IN  AAAA    2001:db8::209

Now when you start another Docker Container, bind it to the next IPv6 address. Unfortunately, the docker command won't resolve the DNS name in the -p parameter, so you will still have use the IP address, but this can be easily fixed with a three line script:

    export HOST="web2"
    export IPADDR=$(host $HOST | awk '{print $5}')
    docker run -d -p "[$IPADDR]:80:80" containous/whoami

Now you can have multiple docker containers all listening on port 80 (or 443) running on the same host without having to go though the 1990s pain of manually configuring IPv6.

As a bonus, your users don't have to add a special port to their web request. Instead they will just use a normal URL, such as http://web2

Working with Podman

Podman is another container framework that supports OCI (Open Container Initiative). It can run Docker Containers, and I find I like it better than Docker. In the default mode it runs "rootless", which generally doesn't require root privileges or membership of a special group, to run containers. Since Podman also supports OCI, it can run Docker containers directly with the command:

    export HOST="web3"
    export IPADDR=$(host $HOST | awk '{print $5}')
    podman run -rm -p "[$IPADDR]:80:80" docker.io/containous/whoami

The IPv6 Advantage

Of course, this assumes you have a dual stack network up and running (and here's a good reason to do so). There is no need to have your entire network configured for IPv6-only.

You could use the above method of mapping Docker/Podman containers to IPv4-only addresses, but seriously, who has that many unused IPv4 addresses just laying around.

So go forth, and create as many web-services containers as you need, using an IPv6 advantage.

Notes:

29 May 2024