Did you know that you can navigate the posts by swiping left and right?

Etcd clusters in 5 mins

Some notes and configurations on how to setup 3-nodes static Etcd clusters.

Say, we have three nodes, configured with the following addresses on the management segment:

  • etcd1: 172.28.128.7
  • etcd2: 172.28.128.8
  • etcd3: 172.28.128.9

A static configuration on CentOS 7

On each node, install etcd:

yum install -y etcd


Then, configure /etc/etcd/etcd.conf: For node etcd1 the relevant options to pass to the etcd command are:

ETCD_NAME=etcd1
ETCD_DATA_DIR="/var/lib/etcd/etcd1.etcd"
ETCD_LISTEN_PEER_URLS="http://172.28.128.7:2380"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.28.128.7:2380"
ETCD_INITIAL_CLUSTER="etcd1=http://172.28.128.7:2380,etcd2=http://172.28.128.8:2380,etcd3=http://172.28.128.9:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://172.28.128.7:2379"


Start the services:

systemctl start etcd


Finally, open ports tcp/2379 and tcp/2380 on the segment. Troubleshooting with journalctl -ex.

A local cluster with Docker compose

Create a Compose YAML file (for example pick up this one):

version: '2'
services:
  etcd1:
    image: quay.io/coreos/etcd
    container_name: etcd1
    command: >
        etcd -name etcd1
        -advertise-client-urls http://172.28.128.7:2379,http://172.28.128.7:4001
        -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001
        -initial-advertise-peer-urls http://172.28.128.7:2380
        -listen-peer-urls http://0.0.0.0:2380
        -initial-cluster-token etcd-cluster-1
        -initial-cluster etcd1=http://172.28.128.7:2380,etcd2=http://172.28.128.8:2380,etcd3=http://172.28.128.9:2380
        -initial-cluster-state new
    networks:
      etcd:
        ipv4_address: 172.28.128.7
  etcd2:
    image: quay.io/coreos/etcd
    container_name: etcd2
    command: >
        etcd -name etcd2
        -advertise-client-urls http://172.28.128.8:2379,http://172.28.128.8:4001
        -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001
        -initial-advertise-peer-urls http://172.28.128.8:2380
        -listen-peer-urls http://0.0.0.0:2380
        -initial-cluster-token etcd-cluster-1
        -initial-cluster etcd1=http://172.28.128.7:2380,etcd2=http://172.28.128.8:2380,etcd3=http://172.28.128.9:2380
        -initial-cluster-state new
    networks:
      etcd:
        ipv4_address: 172.28.128.8
  etcd3:
    image: quay.io/coreos/etcd
    container_name: etcd3
    command: >
        etcd -name etcd3
        -advertise-client-urls http://172.28.128.9:2379,http://172.28.128.9:4001
        -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001
        -initial-advertise-peer-urls http://172.28.128.9:2380
        -listen-peer-urls http://0.0.0.0:2380
        -initial-cluster-token etcd-cluster-1
        -initial-cluster etcd1=http://172.28.128.7:2380,etcd2=http://172.28.128.8:2380,etcd3=http://172.28.128.9:2380
        -initial-cluster-state new
    networks:
      etcd:
        ipv4_address: 172.28.128.9
networks:
  etcd:
    driver: bridge
    ipam:
      config:
      - subnet: 172.28.128.0/28 


Compose creates three Etcd instances, advertising and clustering them on the bridged network:

docker-compose -f etcd.yaml up -d

Verify the containers are up:

$ docker ps --filter status=running --filter name=etcd --format {{.ID}} {{.Names}} {{.Ports}} {{.Command}}"
5133ed70339d etcd3 2379-2380/tcp "etcd -name etcd3 -ad"
5e84fdfad10a etcd2 2379-2380/tcp "etcd -name etcd2 -ad"
2b90ee8ff266 etcd1 2379-2380/tcp "etcd -name etcd1 -ad"


Test the cluster health with:

$ docker exec -ti etcd1 etcdctl cluster-health
member 825d920a517db57 is healthy: got healthy result from http://172.28.128.7:2379
member dabc74f41c6234cd is healthy: got healthy result from http://172.28.128.8:2379
member f7056b953ac6ca92 is healthy: got healthy result from http://172.28.128.9:2379
cluster is healthy