Tianon's Ramblings

Docker on VULTR + IPv6

2016-03-03T00:00:00-07:00

I’ve been using VULTR for a little while now and have been generally very pleased (especially with the very recent facelift the management portal received). I don’t want to waste too much time talking about it, but the “killer feature” for me (over some of their competitors like DigitalOcean) is that I can provide a raw ISO and provision my VM directly using it as I would any local VM (which also means that once my VM is up and working, I get to use the OS’s standard kernel, which is especially important for using Debian Unstable well).

Anyhow, already too much about that – let’s get to the cool stuff.

Getting right down to the beef, let’s assume I’ve got a VULTR instance already created, my OS is already installed and working, I’ve enabled IPv6 within VULTR, ensured that my VM is able to ping6 google.com (to verify at least basic routability), and have Docker version 1.10.2 installed.

For the sake of demonstration, we’ll assume that VULTR has assigned my IPv6 as follows: (available under the VM details via Settings > IPv6)

Default IP: 2001:db8::5400:00ff:fe20:2295
Network: 2001:db8::
CIDR: 64

(The astute reader may recognize RFC3849 here. 😏)

The relevant documentation which helped me get to the working state outlined below is in the “IPv6 with Docker” section.

The first step I took was creating a systemd drop-in file so that I could modify the daemon startup parameters (to include --ipv6 and --fixed-cidr-v6):

# /etc/systemd/system/docker.service.d/ipv6.conf
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// --ipv6 --fixed-cidr-v6 2001:db8::/80

I chose to use just /80 for Docker – any other reasonable prefix (assuming it is routed to your host / host network) should also work; the documentation I linked above has an example using a /125, for example.

With this half in place, I can systemctl daemon-reload and systemctl restart docker.service, and when I start a container it will be automatically assigned an IPv6 address from within that prefix. Excellent.

An important caveat to note is that this will break discovery on our host due to Docker enabling forwarding for us, so (assuming your “internet-facing” interface is named ens3 for the sake of illustration; it might just as easily be eth0, eth1, enps3, lan0, wlan0, etc) I had to sysctl net.ipv6.conf.ens3.accept_ra=2, and I added it to /etc/sysctl.d/docker-ipv6.conf for good measure (so that I don’t lose it after I reboot).

The second half of our IPv6 to containers problem is routing. The nitty-gritty details of this are discussed in the “Using NDP proxying” section of the documentation, but the gist is that my containers have IPv6 addresses, but the outside world doesn’t have a route that leads to them, and that we need to tell the kernel to respond to solicitations for our container’s IPv6 addresses appropriately.

The kernel has a mechanism for doing so (via ip -6 neigh ...), but it is limited to individual addresses and is thus not especially great for having a solution that works “magically” without further manual labor per-container.

This is where ndppd (also packaged for Debian as ndppd) came in.

# /etc/ndppd.conf
proxy ens3 {
	rule 2001:db8::/80 {
	}
}

After getting this configuration in place and restarting ndppd (systemctl restart ndppd), magic happened. My containers could ping6 google.com, and my other IPv6 hosts could ping6 the IPv6 addresses of my individual containers!

You’ve probably noted that this configuration isn’t exactly secure, since it means that each of my individual containers has a publicly routable IPv6 address, but for this specific use case, I’m OK with that! 🍦

Update (2015-03-09): thanks to Алексей Шилин for correcting my systemd drop-in file usage! ♥

DEP8 - TL;DR

2015-05-28T00:00:00-06:00

DEP stands for “Debian Enhancement Proposals”. DEP8 is about package testing, specifically post-install (as opposed to dh_auto_test which runs during package build, usually for unit tests). It’s great for integration tests, etc. that have more interesting requirements for running than unit tests normally do.

The problem is that the spec is a little bit long in the tooth for casual reading / understanding-at-a-glance.

What follows is my own personal TL;DR version.

$ cd your-package/
$ mkdir -p debian/tests
$ vim debian/tests/control

(editor of your choice)

Tests: my-test
Depends: hello, @
Restrictions: needs-root

(see the spec for more info about what these mean and what valid values are)

$ touch debian/tests/my-test
$ chmod +x debian/tests/my-test
$ vim debian/tests/my-test

(editor of your choice)

#!/bin/bash
set -e

hello

# other bits testing your actual package (installed because of "@" in "Depends:")

(again, see the spec linked above for details of how this script should behave – in general, non-zero exit code or stderr output mean failure)

$ apt-get install autopkgtest # if not already installed
$ adt-run --unbuilt-tree . --- VIRT-SERVER

Where VIRT-SERVER is one of: (as of this writing and installed by default with autopkgtest – YMMV)

DD-WRT + dns.he.net (DDNS / inadyn)

2015-05-25T00:00:00-06:00

The DD-WRT wiki hilariously has a page about this, but it’s not very helpful and account creation is entirely disabled, so here goes a blog post for my own future reference:

In the dns.he.net control panel, enable the hostname (HOSTNAME) for “dynamic dns”. Click the DDNS icon and generate a “key” (KEY), which will be used as the password for updating.

In the DD-WRT control panel, under “Setup > DDNS” (at least in Firmware: DD-WRT v24-sp2 (02/19/14) std):

DDNS Service: Custom
DYNDNS Server: dyn.dns.he.net
Username: HOSTNAME
Password: KEY
Hostname: HOSTNAME
URL: /nic/update?hostname=

Whala.

Update (2015-08-19):

ddclient.conf:

protocol=dyndns2
use=if
if=eth0
server=dyn.dns.he.net
ssl=no
login=HOSTNAME
password=KEY
HOSTNAME

The Internet's Own Boy

2014-11-22T00:00:00-07:00

I am really late coming to this train. This has been on my list since the day it was released, and today I finally found the time to sit alone and digest.

To say that it was “emotionally moving” would be a gross misrepresentation of the film. There were clearly some aspects added for dramatic effect, but if you strip away (for example) the demonization of the opposition’s actions, the proceedings remain absolutely astounding and the results entirely heartbreaking.

That a society which claims to be as advanced as ours does (technologically, morally, socially, militarily, etc) could demoralize an individual in this manner seems terminally and criminally corrupt.

When my filesystem gets corrupted, I try to desperately salvage any of the useful information (usually while trying to salvage the entire filesystem), and once my efforts have proven to have provided all the fruits they possibly can, I proceed to format the drive and begin again. Sometimes, a system needs to be rebooted. A fresh install of the operating system usually extends the life of a system by a measurable amount of time.

Perhaps reinstalling our base operating system is worth a try?

DebConf14

2014-08-30T00:00:00-06:00

I was given the opportunity to attend DebConf in Portland this year, and I must say that it took me entirely by surprise (as a first-time attendee).

Most conferences have several strong talk tracks and you end up spending a lot of time sitting in talks wondering when they’ll be over. At DebConf, there’s an entirely different dynamic, with a strong focus on what they like to call “the hallway track” (which this year has taken place a lot in the hacklabs, too). Everyone here wants to either talk about building cool stuff, or sit down and actually build some cool stuff. A large number of the talks are just launchpads for informal Q+As or actual hack sessions.

By coming, I’ve learned a lot about the Debian community and how it operates as a whole, and managed to meet a lot of very interesting and cool people (not to mention getting a bunch of them to sign my GPG key, which is also nice).

Hopefully I’ll be able to attend more DebConfs in the future, because I’ve had a great time!

Love is a Battlefield

2014-05-17T00:00:00-06:00

Docker on Gentoo can be a beautiful thing, but it can also be a challenge navigating some of the trade-offs.

The hardest decision to make, in my opinion, is which storage backend to use. Each one has ups and downs, and some of them have ups and downs that are more specific to Gentoo than others.

“aufs”

Normally for an out-of-kernel module (even a filesystem), it would be a simple matter to simply compile said module against the proper kernel sources and load it up; no harm, no foul. What’s particularly needling about AUFS is that it requires patches to the kernel proper (which, I might add, were submitted for inclusion in the kernel and rejected).

The quandary that’s most interesting about AUFS is that it’s currently the recommended Docker backend. For Ubuntu and Debian users, this isn’t a problem since the AUFS patches are included in the main kernels and so the aufs module is merely a single apt-get install away.

As you might imagine, these patches make a bit of a stir for someone who builds their own kernels (like, say, a Gentoo user), and there are two main ways to get them.

`sys-kernel/aufs-sources`

I’ll start with the easy way. If you emerge sys-kernel/aufs-sources, you’ll get sys-kernel/gentoo-sources with the AUFS patches pre-applied. Choosing this method, it’s merely a matter of making sure CONFIG_AUFS_FS is enabled in your .config and you’re good to go. If you’re already using stock sys-kernel/gentoo-sources and/or are not averse to a slight change, this will be the easiest, cleanest, and most importantly the least error-prone option by far.

`sys-fs/aufs3`

The alternative is to use sys-fs/aufs3. This package provides both the necessary kernel patches and compiles the aufs module, making it much more suitable to sys-kernel/vanilla-sources and the like. The aufs module will only load on a kernel compiled with the AUFS patches. This ebuild includes a kernel-patch use flag that will automatically apply the patches to /usr/src/linux at merge time, which is the simplest way to ensure they are applied.

Note that in my experience, this method is very human error-prone. Using sys-kernel/aufs-sources, portage tracks the patches. Using sys-fs/aufs3, it’s all up to you. I wish I could get back the lost time rebooting into a new kernel only to realize I hadn’t recompiled it again after re-emerging sys-fs/aufs3.

“btrfs”

BTRFS is fun. It’s speedy, it’s hip, it’s experimental. The obvious downside to using it as your Docker backend is that most of us don’t have our root filesystem on it, which means we either have to reinstall our OS, make a new partition/drive/loopback for Docker, or choose a different backend.

Note that if you do have BTRFS as your root filesystem, you want to make sure you do not use the AUFS backend. AUFS on top of BTRFS has lots and lots of strange issues.

“devicemapper”

The LVM/devicemapper backend is especially cool because the kernel features it requires are enabled in a wide variety of pre-compiled kernels, making this by far the easiest backend to get started with. Also, it doesn’t play foul with any known filesystems since it effectively mounts containers in loopback, avoiding potential issues with filesystems interfering.

However, unless you configure it to use a raw physical disk partition, the performance will likely leave much to be desired.

“vfs”

What we lovingly refer to as “vfs” is an interesting driver. It’s what’s used for volumes, and is essentially a reference implementation for graph drivers. It has no “copy on write” at all, and is essentially just “copy the entire rootfs for each new layer”, so is perfectly suited for volumes, but is not at all well-suited for being the general daemon backend.

Getting High on Hy

2014-04-19T00:00:00-06:00

My good friend Paul Tagliamonte recently gave a talk at PyCon 2014 about his language, Hy. It’s effectively Lisp implemented inside Python (so cleanly that Python itself doesn’t care to differentiate the result from any other Python code).

It’s a solid talk that covers a lot of good ground about some of the cool stuff you can do with Hy, and especially about the internals of exactly how Hy works, which is really fascinating stuff. There’s even a shout-out to our shared love, Docker!

The video can be found on YouTube, but I’ve also embedded it below for your viewing pleasure.

If you’d like to give Hy a try, you can check it out with try-hy, which is Hy running sandboxed on Google App Engine so you can play with it freely inside your browser.

sh/tagwords.hy:

;; python-sh from hy

(import [sh [cat grep]])
(print "Words that end with `tag`:")
(print (-> (cat "/usr/share/dict/words") (grep "-E" "tag$")))

$ hy sh/tagwords.hy
Words that end with `tag`:
Bundestag
Maytag
Reichstag
Sontag
ragtag
stag
tag