Skip to content

Commit 43bc5fa

Browse files
author
Matthew Fisher
committed
fix(*): make containers host-aware
Before, you had to hardcode the host's IP address and the IP address to the etcd cluster. Now that we are on CoreOS, we can make some assumptions about the system, including the default gateway and where etcd is located. The host IP address was always accessible through the default gateway, so there's a change to use netstat to grep for the ip address of the docker0 bridge. Using that, we can also assume that etcd is clustered on the host vs. a remote container on some other node, so we can just use the host IP address and a default port number to retrieve the endpoint. We should be using Fleet to manage these containers, so I removed the dependencies on certain systemd units, since fleet already considers those units to be running on each host. I also removed the dependency on deis-builder to be running before deis-controller can be started. We did this because we wanted to ensure that users did not run into issues when running a git push, but I feel that it's a poor decision to kill the controller if the builder is down. By that argument, we should kill the controller when *any* component comes down. A `fleetctl list-units` should be more than enough to ensure that the containers are running as they should be.
1 parent 78b645e commit 43bc5fa

3 files changed

Lines changed: 13 additions & 10 deletions

File tree

bin/boot

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ set -eo pipefail
99
# set debug based on envvar
1010
[[ $DEBUG ]] && set -x
1111

12+
# HACK: get the host IP address through the default gateway
13+
export HOST=$(netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}')
14+
1215
# configure etcd
13-
export ETCD=${ETCD:-172.17.42.1:4001}
16+
export ETCD_PORT=${ETCD_PORT:-4001}
17+
export ETCD="$HOST:$ETCD_PORT"
1418
export ETCD_PATH=${ETCD_PATH:-/deis/controller}
1519
export ETCD_TTL=${ETCD_TTL:-10}
1620

@@ -69,11 +73,10 @@ CONFD_PID=$!
6973

7074
echo deis-controller running...
7175

72-
# publish the service to etcd using the injected HOST and PORT
76+
# publish the service to etcd using the injected PORT
7377
if [[ ! -z $PUBLISH ]]; then
7478

7579
# configure service discovery
76-
HOST=${HOST:-localhost}
7780
PORT=${PORT:-8000}
7881
PROTO=${PROTO:-tcp}
7982

scheduler/coreos.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -199,7 +199,7 @@ def attach(self, name):
199199
200200
[Service]
201201
ExecStartPre=/usr/bin/docker pull {image}
202-
ExecStart=-/usr/bin/docker run --name {name} -P -e PORT={port} -e ETCD=172.17.42.1:4001 {image} {command}
202+
ExecStart=-/usr/bin/docker run --name {name} -P -e PORT={port} {image} {command}
203203
ExecStop=-/usr/bin/docker rm -f {name}
204204
"""
205205

@@ -238,7 +238,7 @@ def attach(self, name):
238238
239239
[Service]
240240
ExecStartPre=/usr/bin/docker pull {image}
241-
ExecStart=-/usr/bin/docker run --name {name} -p 80:80 -p 443:443 -e ETCD=172.17.42.1:4001 {image} {command}
241+
ExecStart=-/usr/bin/docker run --name {name} -p 80:80 -p 443:443 {image} {command}
242242
ExecStop=-/usr/bin/docker rm -f {name}
243243
TimeoutStartSec=10min
244244
"""
@@ -251,6 +251,6 @@ def attach(self, name):
251251
252252
[Service]
253253
ExecStartPre=/usr/bin/docker pull {image}
254-
ExecStart=-/usr/bin/docker run --name {name} -p 514:514 -e HOST=%H -e PORT=514 -e ETCD=172.17.42.1:4001 {image} {command}
254+
ExecStart=-/usr/bin/docker run --name {name} -p 514:514 -e PORT=514 {image} {command}
255255
ExecStop=-/usr/bin/docker rm -f {name}
256256
"""

systemd/deis-controller.service

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[Unit]
22
Description=deis-controller
3-
Requires=docker.service docker-tcp.socket etcd.service deis-builder.service
4-
After=docker.service etcd.service deis-builder.service
53

64
[Service]
7-
ExecStart=/bin/bash -c '/usr/bin/docker start -a deis-controller || /usr/bin/docker run --name deis-controller -p 8000:8000 -e PUBLISH=8000 -e HOST=172.17.8.100 -e ETCD=172.17.8.100:4001 deis/controller'
8-
ExecStop=/usr/bin/docker stop deis-controller
5+
TimeoutStartSec=20m
6+
ExecStartPre=/usr/bin/docker pull deis/controller:latest
7+
ExecStart=/usr/bin/docker run --name deis-controller -p 8000:8000 -e PUBLISH=8000 deis/controller
8+
ExecStop=/usr/bin/docker rm -f deis-controller
99

1010
[Install]
1111
WantedBy=multi-user.target

0 commit comments

Comments
 (0)