Overview

Skill Level: Any Skill Level

Ingredients

Docker basic knowledge

Step-by-step

  1. Overview

    This post describes how to use and customize the default docker0 bridge to setup the networking for docker containers. The Docker server creates and configures the host system’s docker0 interface as an Ethernet bridge inside the Linux kernel that could be used by the docker containers to communicate with each other and with the outside world, the default configuration of the docker0 works for most of the scenarios but you could customize the docker0 bridge based on your specific requirements.

    The docker0 bridge is virtual interface created by docker, it randomly chooses an address and subnet from the private range defined by RFC 1918 that are not in use on the host machine, and assigns it to docker0. All the docker containers will be connected to the docker0 bridge by default, the docker containers connnected to the docker0 bridge could use the iptables NAT rules created by docker to communicate with the outside world.

  2. How the docker0 bridge is created?

    The docker0 bridge will be created when the docker service is started.

     root@docker:~# brctl show
    bridge name bridge id STP enabled interfaces
    root@docker:~# systemctl start docker
    root@docker:~# brctl show
    bridge name bridge id STP enabled interfaces
    docker0 8000.0242f2a2ed9e no
    root@docker:~#
  3. The docker0 bridge and NAT rules under the hood

    docker0 is a Linux bridge without any real network adapter attached, and configured with ip address 172.17.0.1/16

     root@docker:~# brctl show
    bridge name bridge id STP enabled interfaces
    docker0 8000.0242f2a2ed9e no
    root@docker:~#

    root@docker:~# ifconfig docker0
    docker0 Link encap:Ethernet HWaddr 02:42:f2:a2:ed:9e
    inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
    UP BROADCAST MULTICAST MTU:1500 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

    root@docker:~#

    docker creates a network named “bridge” for docker0 Linux bridge, as shown below.

    root@docker:~# docker network ls
    NETWORK ID NAME DRIVER
    62f5d1753557 bridge bridge
    9804bb2347d2 none null
    1af92dcf5a8e host host
    root@docker:~#

    Look even further into the docker network “bridge” using command “docker network inspect”, we could see that the subnet associated with the network is 172.17.0.0/16, two docker containers “docker1” and “docker2” are connected to the bridge network with ip addresses 172.17.0.2/16 and 172.17.0.3/16.

     root@docker:~# docker network inspect bridge
    [
    {
    "Name": "bridge",
    "Id": "a6eb0368ed0cbfea8d665e3ec7720942e37a556204652f850a534fcb8863e20e",
    "Scope": "local",
    "Driver": "bridge",
    "IPAM": {
    "Driver": "default",
    "Options": null,
    "Config": [
    {
    "Subnet": "172.17.0.0/16"
    }
    ] },
    "Containers": {
    "187c7fcc684b6d7bec5092e83e07fe2d435464843b97e9675e2b06bd54c4ff05": {
    "Name": "docker2",
    "EndpointID": "facf10ae0459f3c016b578219ee0a7fa2e3ffffc073f95b00501987b7d65e06b",
    "MacAddress": "02:42:ac:11:00:03",
    "IPv4Address": "172.17.0.3/16",
    "IPv6Address": ""
    },
    "9b62a276a824961e6aa1145a05491e363c64fb7f2850197b5e19993c67d8d97c": {
    "Name": "docker1",
    "EndpointID": "70236ad7660f5be91b51fcb2fc506f9c91eccfeb6f3eee14d2a731fdbcb6a455",
    "MacAddress": "02:42:ac:11:00:02",
    "IPv4Address": "172.17.0.2/16",
    "IPv6Address": ""
    }
    },
    "Options": {
    "com.docker.network.bridge.default_bridge": "true",
    "com.docker.network.bridge.enable_icc": "true",
    "com.docker.network.bridge.enable_ip_masquerade": "true",
    "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
    "com.docker.network.bridge.name": "docker0",
    "com.docker.network.driver.mtu": "1500"
    }
    }
    ]root@docker:~#

    Docker also creates the iptables NAT rules on the docker host that could be used by the docker containers connected to docker0 bridge to connect to the outside world.

     root@docker:~# iptables -t nat -L
    Chain PREROUTING (policy ACCEPT)
    target prot opt source destination
    DOCKER all -- anywhere anywhere ADDRTYPE match dst-type LOCAL

    Chain INPUT (policy ACCEPT)
    target prot opt source destination

    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination
    DOCKER all -- anywhere !127.0.0.0/8 ADDRTYPE match dst-type LOCAL

    Chain POSTROUTING (policy ACCEPT)
    target prot opt source destination
    MASQUERADE all -- 172.17.0.0/16 anywhere

    Chain DOCKER (2 references)
    target prot opt source destination
    RETURN all -- anywhere anywhere
    root@docker:~#

  4. Connects the docker containers to docker0 bridge

    By default Docker will attach all containers to the docker0 bridge, so you do not need to specify any additional flag with docker run command to connect the docker containers to the docker0 bridge, unless the DOCKER_OPTS in docker configuration file explicitly specifies to use the other network than docker0 bridge, in this case you could use –net=bridge with docker run command to connect the containers to the docker0 bridge.

     root@docker:~# docker run -d --name docker3 --net=bridge liguangcheng/ubuntu-16.04-ppc64el
    2b4b9e7612e5593f31515e46e54ffc4ab7902ffdc6162d663f87e741b3b91117
    root@docker:~# docker inspect docker3
    ... ... ...
    "Networks": {
    "bridge": {
    "IPAMConfig": null,
    "Links": null,
    "Aliases": null,
    "NetworkID": "a6eb0368ed0cbfea8d665e3ec7720942e37a556204652f850a534fcb8863e20e",
    "EndpointID": "f0174d4c507449da55008d646bbff216064450d21d6506af1b1b0bdab4176c93",
    "Gateway": "172.17.0.1",
    "IPAddress": "172.17.0.4",
    "IPPrefixLen": 16,
    "IPv6Gateway": "",
    "GlobalIPv6Address": "",
    "GlobalIPv6PrefixLen": 0,
    "MacAddress": "02:42:ac:11:00:04"
    }

    You could use brctl show docker0 to verify if the docker container connects to the docker0 bridge correctly, here is an example:

     root@docker:~# brctl show docker0
    bridge name bridge id STP enabled interfaces
    docker0 8000.0242f2a2ed9e no veth55b0d46
    veth84dfd16
    vethc8512e4
    root@docker:~#

    There are three docker containers connect to the docker0 bridge, to verify if the docker3 is in the list:

     root@docker:~# docker exec -it docker3 ip addr
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
    valid_lft forever preferred_lft forever
    17: eth0@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:11:00:04 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.17.0.4/16 scope global eth0
    valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe11:4/64 scope link
    valid_lft forever preferred_lft forever
    root@docker:~#

    The prefix number of the eth0 in docker3 is 17, use ethtool to check the prefix number of the veths connected to the bridge docker0:

      root@docker:~# ethtool -S veth30f1f08 
    NIC statistics:
    peer_ifindex: 19
    root@docker:~# ethtool -S veth56671d5
    NIC statistics:
    peer_ifindex: 21
    root@docker:~# ethtool -S veth84dfd16
    NIC statistics:
    peer_ifindex: 17
    root@docker:~#

    veth84dfd16 is the veth peer of docker3 eth0, then we could say the docker3 is connected to the docker0 bridge correctly.

  5. Customize docker0 bridge

    The default configuration of docker0 works for most of the cases, however, you could customize the docker0 configuration per your requirements, the following options of docker0 are configurable at server startup:

    – –bip=CIDR — supply a specific IP address and netmask for the docker0 bridge, using standard CIDR notation like 192.168.1.5/24.

    – –fixed-cidr=CIDR — restrict the IP range from the docker0 subnet, using the standard CIDR notation like 172.167.1.0/28. This range must be an IPv4 range for fixed IPs (ex: 10.20.0.0/16) and must be a subset of the bridge IP range (docker0 or set using –bridge). For example with –fixed-cidr=192.168.1.0/25, IPs for your containers will be chosen from the first half of 192.168.1.0/24 subnet.

    – –mtu=BYTES — override the maximum packet length on docker0.

    These configurable parameters could be added to the docker configuration file /etc/default/docker or /etc/sysconfig/docker, here is an example:

    DOCKER_OPTS=”–bip=192.168.200.0/16 –fixed-cidr=192.168.200.0/24 –default-gateway=192.168.200.1 –mtu=1024″

    systemctl restart docker

Join The Discussion