Skip to content

Commit 8952179

Browse files
author
Matthew Fisher
committed
fix(*): make containers host-aware
Before, you had to hardcode the host's IP address and the IP address to the etcd cluster. Now that we are on CoreOS, we can make some assumptions about the system, including the default gateway and where etcd is located. The host IP address was always accessible through the default gateway, so there's a change to use netstat to grep for the ip address of the docker0 bridge. Using that, we can also assume that etcd is clustered on the host vs. a remote container on some other node, so we can just use the host IP address and a default port number to retrieve the endpoint. We should be using Fleet to manage these containers, so I removed the dependencies on certain systemd units, since fleet already considers those units to be running on each host. I also removed the dependency on deis-builder to be running before deis-controller can be started. We did this because we wanted to ensure that users did not run into issues when running a git push, but I feel that it's a poor decision to kill the controller if the builder is down. By that argument, we should kill the controller when *any* component comes down. A `fleetctl list-units` should be more than enough to ensure that the containers are running as they should be.
1 parent 4111501 commit 8952179

15 files changed

Lines changed: 72 additions & 50 deletions

File tree

‎builder/bin/boot‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ set -eo pipefail
99
# set debug based on envvar
1010
[[ $DEBUG ]] && set -x
1111

12+
# HACK: get the host IP address through the default gateway
13+
export HOST=$(netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}')
14+
1215
# configure etcd
13-
export ETCD=${ETCD:-172.17.42.1:4001}
16+
export ETCD_PORT=${ETCD_PORT:-4001}
17+
export ETCD="$HOST:$ETCD_PORT"
1418
export ETCD_PATH=${ETCD_PATH:-/deis/builder}
1519
export ETCD_TTL=${ETCD_TTL:-10}
1620

@@ -67,11 +71,10 @@ trap on_exit INT TERM EXIT
6771

6872
echo deis-builder running...
6973

70-
# publish the service to etcd using the injected HOST and PORT
74+
# publish the service to etcd using the injected PORT
7175
if [[ ! -z $PUBLISH ]]; then
7276

7377
# configure service discovery
74-
HOST=${HOST:-localhost}
7578
PORT=${PORT:-2222}
7679
PROTO=${PROTO:-tcp}
7780

‎builder/systemd/deis-builder.service‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[Unit]
22
Description=deis-builder
3-
Requires=docker.service docker-tcp.socket etcd.service
4-
After=docker.service etcd.service
53

64
[Service]
7-
ExecStart=/bin/bash -c '/usr/bin/docker start -a deis-builder || /usr/bin/docker run --name deis-builder -p 2222:22 -e PUBLISH=22 -e HOST=172.17.8.100 -e PORT=2222 -e ETCD=172.17.8.100:4001 --privileged deis/builder'
8-
ExecStop=/usr/bin/docker stop deis-builder
5+
TimeoutStartSec=20m
6+
ExecStartPre=/usr/bin/docker pull deis/builder:latest
7+
ExecStart=/usr/bin/docker run --name deis-builder -p 2222:22 -e PUBLISH=22 -e PORT=2222 --privileged deis/builder
8+
ExecStop=/usr/bin/docker rm -f deis-builder
99

1010
[Install]
1111
WantedBy=multi-user.target

‎cache/bin/boot‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ set -eo pipefail
99
# set debug based on envvar
1010
[[ $DEBUG ]] && set -x
1111

12+
# HACK: get the host IP address through the default gateway
13+
export HOST=$(netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}')
14+
1215
# configure etcd
13-
export ETCD=${ETCD:-172.17.42.1:4001}
16+
export ETCD_PORT=${ETCD_PORT:-4001}
17+
export ETCD="$HOST:$ETCD_PORT"
1418
export ETCD_PATH=${ETCD_PATH:-/deis/cache}
1519
export ETCD_TTL=${ETCD_TTL:-10}
1620

@@ -52,11 +56,10 @@ CONFD_PID=$!
5256

5357
echo deis-cache running...
5458

55-
# publish the service to etcd using the injected HOST and PORT
59+
# publish the service to etcd using the injected PORT
5660
if [[ ! -z $PUBLISH ]]; then
5761

5862
# configure service discovery
59-
HOST=${HOST:-localhost}
6063
PORT=${PORT:-6379}
6164
PROTO=${PROTO:-tcp}
6265

‎cache/systemd/deis-cache.service‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[Unit]
22
Description=deis-cache
3-
Requires=docker.service docker-tcp.socket etcd.service
4-
After=docker.service etcd.service
53

64
[Service]
7-
ExecStart=/bin/bash -c '/usr/bin/docker start -a deis-cache || /usr/bin/docker run --name deis-cache -p 6379:6379 -e PUBLISH=6379 -e HOST=172.17.8.100 -e ETCD=172.17.8.100:4001 deis/cache'
8-
ExecStop=/usr/bin/docker stop deis-cache
5+
TimeoutStartSec=20m
6+
ExecStartPre=/usr/bin/docker pull deis/cache:latest
7+
ExecStart=/usr/bin/docker run --name deis-cache -p 6379:6379 -e PUBLISH=6379 deis/cache
8+
ExecStop=/usr/bin/docker rm -f deis-cache
99

1010
[Install]
1111
WantedBy=multi-user.target

‎controller/bin/boot‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ set -eo pipefail
99
# set debug based on envvar
1010
[[ $DEBUG ]] && set -x
1111

12+
# HACK: get the host IP address through the default gateway
13+
export HOST=$(netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}')
14+
1215
# configure etcd
13-
export ETCD=${ETCD:-172.17.42.1:4001}
16+
export ETCD_PORT=${ETCD_PORT:-4001}
17+
export ETCD="$HOST:$ETCD_PORT"
1418
export ETCD_PATH=${ETCD_PATH:-/deis/controller}
1519
export ETCD_TTL=${ETCD_TTL:-10}
1620

@@ -69,11 +73,10 @@ CONFD_PID=$!
6973

7074
echo deis-controller running...
7175

72-
# publish the service to etcd using the injected HOST and PORT
76+
# publish the service to etcd using the injected PORT
7377
if [[ ! -z $PUBLISH ]]; then
7478

7579
# configure service discovery
76-
HOST=${HOST:-localhost}
7780
PORT=${PORT:-8000}
7881
PROTO=${PROTO:-tcp}
7982

‎controller/scheduler/coreos.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -199,7 +199,7 @@ def attach(self, name):
199199
200200
[Service]
201201
ExecStartPre=/usr/bin/docker pull {image}
202-
ExecStart=-/usr/bin/docker run --name {name} -P -e PORT={port} -e ETCD=172.17.42.1:4001 {image} {command}
202+
ExecStart=-/usr/bin/docker run --name {name} -P -e PORT={port} {image} {command}
203203
ExecStop=-/usr/bin/docker rm -f {name}
204204
"""
205205

@@ -238,7 +238,7 @@ def attach(self, name):
238238
239239
[Service]
240240
ExecStartPre=/usr/bin/docker pull {image}
241-
ExecStart=-/usr/bin/docker run --name {name} -p 80:80 -p 443:443 -e ETCD=172.17.42.1:4001 {image} {command}
241+
ExecStart=-/usr/bin/docker run --name {name} -p 80:80 -p 443:443 {image} {command}
242242
ExecStop=-/usr/bin/docker rm -f {name}
243243
TimeoutStartSec=10min
244244
"""
@@ -251,6 +251,6 @@ def attach(self, name):
251251
252252
[Service]
253253
ExecStartPre=/usr/bin/docker pull {image}
254-
ExecStart=-/usr/bin/docker run --name {name} -p 514:514 -e HOST=%H -e PORT=514 -e ETCD=172.17.42.1:4001 {image} {command}
254+
ExecStart=-/usr/bin/docker run --name {name} -p 514:514 -e PORT=514 {image} {command}
255255
ExecStop=-/usr/bin/docker rm -f {name}
256256
"""

‎controller/systemd/deis-controller.service‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[Unit]
22
Description=deis-controller
3-
Requires=docker.service docker-tcp.socket etcd.service deis-builder.service
4-
After=docker.service etcd.service deis-builder.service
53

64
[Service]
7-
ExecStart=/bin/bash -c '/usr/bin/docker start -a deis-controller || /usr/bin/docker run --name deis-controller -p 8000:8000 -e PUBLISH=8000 -e HOST=172.17.8.100 -e ETCD=172.17.8.100:4001 deis/controller'
8-
ExecStop=/usr/bin/docker stop deis-controller
5+
TimeoutStartSec=20m
6+
ExecStartPre=/usr/bin/docker pull deis/controller:latest
7+
ExecStart=/usr/bin/docker run --name deis-controller -p 8000:8000 -e PUBLISH=8000 deis/controller
8+
ExecStop=/usr/bin/docker rm -f deis-controller
99

1010
[Install]
1111
WantedBy=multi-user.target

‎database/bin/boot‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ set -eo pipefail
99
# set debug based on envvar
1010
[[ $DEBUG ]] && set -x
1111

12+
# HACK: get the host IP address through the default gateway
13+
export HOST=$(netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}')
14+
1215
# configure etcd
13-
export ETCD=${ETCD:-172.17.42.1:4001}
16+
export ETCD_PORT=${ETCD_PORT:-4001}
17+
export ETCD="$HOST:$ETCD_PORT"
1418
export ETCD_PATH=${ETCD_PATH:-/deis/database}
1519
export ETCD_TTL=${ETCD_TTL:-10}
1620

‎database/systemd/deis-database.service‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[Unit]
22
Description=deis-database
3-
Requires=docker.service docker-tcp.socket etcd.service
4-
After=docker.service etcd.service
53

64
[Service]
7-
ExecStart=/bin/bash -c '/usr/bin/docker start -a deis-database || /usr/bin/docker run --name deis-database -p 5432:5432 -e PUBLISH=5432 -e HOST=172.17.8.100 -e ETCD=172.17.8.100:4001 deis/database'
8-
ExecStop=/usr/bin/docker stop deis-database
5+
TimeoutStartSec=20m
6+
ExecStartPre=/usr/bin/docker pull deis/database:latest
7+
ExecStart=/usr/bin/docker run --name deis-database -p 5432:5432 -e PUBLISH=5432 deis/database
8+
ExecStop=/usr/bin/docker rm -f deis-database
99

1010
[Install]
1111
WantedBy=multi-user.target

‎logger/bin/boot‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ set -eo pipefail
99
# set debug based on envvar
1010
[[ $DEBUG ]] && set -x
1111

12+
# HACK: get the host IP address through the default gateway
13+
export HOST=$(netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}')
14+
1215
# configure etcd
13-
export ETCD=${ETCD:-172.17.42.1:4001}
16+
export ETCD_PORT=${ETCD_PORT:-4001}
17+
export ETCD="$HOST:$ETCD_PORT"
1418
export ETCD_PATH=${ETCD_PATH:-/deis/logs}
1519
export ETCD_TTL=${ETCD_TTL:-10}
1620

@@ -54,11 +58,10 @@ CONFD_PID=$!
5458

5559
echo deis-logger running...
5660

57-
# publish the service to etcd using the injected HOST and PORT
61+
# publish the service to etcd using the injected PORT
5862
if [[ ! -z $PUBLISH ]]; then
5963

6064
# configure service discovery
61-
HOST=${HOST:-localhost}
6265
PORT=${PORT:-514}
6366
PROTO=${PROTO:-tcp}
6467

0 commit comments

Comments
 (0)