Shorewall+Docker: Two Great Tastes That Taste Great Together

As has been mentioned previously, we lurve us some Docker here at Discourse. We also lurve us some security, and I've recently been replacing our "artisinally handcrafted iptables firewall rules" with a Shorewall-managed configuration, which plays better with Puppet. Unfortunately, as it stands, like my twin three year olds, they don't always play together well.

Both Docker and Shorewall assume that nobody else is actively messing with the firewall configuration. Shorewall assumes this because it likes to completely blow away the existing firewall configuration, and replace it with a set of rules crafted from your rules files. Docker inserts NAT rules to implement its port forwarding system, amongst other things. Both make sense in isolation, but when you combine the two behaviours... FWACKOOM.

Every time you reload your Shorewall ruleset, all your Docker containers stop receiving traffic. Restarting Docker fixes it, but who wants to do that on a large-scale production infrastructure? Not me.

Luckily, Shorewall, being the awesome system that it is, has plenty of hook points (or, as it calls them, extension scripts) you can use to do funky, custom things. Such as, in this case, saving the existing Docker-related firewall rules before blowing away the firewall, and restoring them afterwards. Thanks to Docker's decision to confine most of its rules to a special chain, named DOCKER, this is quite straightforward.

There are three hooks you need to create, all in the same path.

/etc/shorewall/init and /etc/shorewall/stop have the same contents:

if iptables -t nat -L DOCKER >/dev/null 2>&1; then
    echo '*nat' >/etc/shorewall/docker_rules
    iptables -t nat -S DOCKER >>/etc/shorewall/docker_rules
    iptables -t nat -S POSTROUTING >>/etc/shorewall/docker_rules
    echo "COMMIT" >>/etc/shorewall/docker_rules

    echo '*filter' >>/etc/shorewall/docker_rules
    iptables -S DOCKER >> /etc/shorewall/docker_rules
    echo "COMMIT" >>/etc/shorewall/docker_rules
fi

/etc/shorewall/start looks like this:

if [ -f /etc/shorewall/docker_rules ]; then
    iptables-restore -n </etc/shorewall/docker_rules
    run_iptables -t nat -I PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
    run_iptables -t nat -I OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
    run_iptables -I FORWARD -o docker0 -j DOCKER
    run_iptables -I FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
    run_iptables -I FORWARD -i docker0 ! -o docker0 -j ACCEPT
    run_iptables -I FORWARD -i docker0 -o docker0 -j ACCEPT

    rm -f /etc/shorewall/docker_rules
fi

Once you've created those three files, with the above contents, when you run shorewall start or shorewall restart, your firewall will be restarted, with all your Shorewall-defined rules, and your Docker rules, all in place.