Skip to content

Commit e1505f4

Browse files
author
Jonathan Chauncey
committed
feat(v2): Adding influxdb and telegraf components
1 parent d0e7edf commit e1505f4

35 files changed

Lines changed: 762 additions & 307 deletions

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
*~
22
*.swp
33
*.swo
4+
.DS_Store

‎README.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# Deis Monitor
22

3-
This project aims to provide a Prometheus based component, designed to make basic cluster monitoring simple and straight forward.
3+
## Description
4+
This repository aims to contain all the necessary components for a production quality monitoring solution that runs on top of the kubernetes cluster scheduler. It provides the [TICK](https://influxdata.com/time-series-platform/) stack which is produced by the influxdata team.
5+
6+
## Current State
7+
Currently this repo provides only 2 components (influxdb and telegraf). Telegraf is the metrics collection agent that runs using the daemon set API. For more infomation please read [this](telegraf/README.md).
8+
9+
Also provided is an influxdb container which only runs 1 instance of the database. It also does not write any data to the host filesystem so it is not a durable system right now. For more information please read [this](influxdb/README.md)
10+
411

5-
The Monitor component is under development, and not recommended for production use at this time.

‎influxdb/Dockerfile‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
#This dockerfile is based on https://github.com/jalateras/docker-influxdb
2+
FROM alpine:3.2
3+
MAINTAINER Jonathan Chauncey "<jchauncey@deis.com>"
4+
5+
ENV INFLUX_VERSION "v0.10.0-rc1"
6+
ENV GOPATH /golang
7+
ENV INFLUXDB_HOME /influxdb
8+
ENV PATH $INFLUXDB_HOME:$PATH
9+
WORKDIR /golang
10+
11+
# Expose the admin port
12+
EXPOSE 8083
13+
14+
# Expose the ssl http api port
15+
EXPOSE 8084
16+
17+
# Expose the http api port
18+
EXPOSE 8086
19+
20+
RUN \
21+
addgroup -S influxdb && \
22+
adduser -S -s /bin/bash -G influxdb influxdb
23+
24+
RUN \
25+
apk add --update bash wget git mercurial bzr go && \
26+
mkdir -p $GOPATH/src/github.com/influxdb && \
27+
cd $GOPATH/src/github.com/influxdb && \
28+
git clone https://github.com/influxdata/influxdb.git && \
29+
cd influxdb && \
30+
git checkout -b $INFLUX_VERSION && \
31+
go get -t ./... && \
32+
go build ./... && \
33+
go install ./... && \
34+
mkdir -p $INFLUXDB_HOME && \
35+
cp $GOPATH/bin/influx* $INFLUXDB_HOME/ && \
36+
chown -R influxdb:influxdb $INFLUXDB_HOME && \
37+
mkdir -p /data /logs /config && \
38+
chown -R influxdb:influxdb /data /logs /config && \
39+
apk del --purge wget git mercurial bzr go && \
40+
rm -rf /var/cache/apk/* /tmp/* /var/tmp/* $GOPATH
41+
42+
RUN mkdir -p /data
43+
VOLUME /data
44+
45+
COPY config.toml /influxdb/config.toml
46+
47+
USER influxdb
48+
CMD ["/influxdb/influxd", "-config", "/influxdb/config.toml"]

‎influxdb/Makefile‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
SHORT_NAME ?= influxdb
2+
BUILD_TAG ?= git-$(shell git rev-parse --short HEAD)
3+
DEIS_REGISTRY ?= ${DEV_REGISTRY}
4+
IMAGE_PREFIX ?= deis
5+
IMAGE_LATEST := ${DEIS_REGISTRY}${IMAGE_PREFIX}/${SHORT_NAME}:latest
6+
IMAGE := ${DEIS_REGISTRY}${IMAGE_PREFIX}/${SHORT_NAME}:${BUILD_TAG}
7+
8+
info:
9+
@echo "Build tag: ${BUILD_TAG}"
10+
@echo "Registry: ${DEIS_REGISTRY}"
11+
@echo "Image: ${IMAGE}"
12+
13+
docker-build:
14+
docker build -t $(IMAGE_LATEST) .
15+
docker tag -f $(IMAGE_LATEST) $(IMAGE)
16+
17+
docker-push:
18+
docker push ${IMAGE}
19+
20+
kube-delete: update-manifests
21+
-kubectl delete -f manifests/deis-monitor-influxdb-svc.yaml
22+
-kubectl delete -f manifests/deis-monitor-proxy-pod.yaml
23+
-kubectl delete -f manifests/deis-monitor-influxdb-rc.tmp.yaml
24+
25+
kube-create: update-manifests
26+
kubectl create -f manifests/deis-monitor-influxdb-svc.yaml
27+
kubectl create -f manifests/deis-monitor-influxdb-rc.tmp.yaml
28+
kubectl create -f manifests/deis-monitor-proxy-pod.yaml
29+
30+
kube-update: update-manifests
31+
kubectl delete -f manifests/deis-monitor-influxdb-rc.tmp.yaml
32+
kubectl create -f manifests/deis-monitor-influxdb-rc.tmp.yaml
33+
34+
update-manifests:
35+
sed 's#\(image:\) .*#\1 $(IMAGE)#' manifests/deis-monitor-influxdb-rc.yaml > manifests/deis-monitor-influxdb-rc.tmp.yaml

‎influxdb/README.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# deis-influxdb
2+
3+
## Description
4+
This is an alpine 3.2 based image for running influxdb 0.10.0-rc1. Props goes to @jalatera's alpine image from which this one is based - https://github.com/jalateras/docker-influxdb.
5+
6+
## Configuration
7+
Right now the configuration is completely static but eventually I hope to use the [envtpl](https://github.com/arschles/envtpl) project to help provide a more robust solution.
8+
9+
## Development
10+
There is a make file provided with the project that can build the image, push it to a registry, and deploy it to a kubernetes cluster.
11+
12+
### Environment Variables
13+
* `DEIS_REGISTRY` : leave blank for dockerhub otherwise provide a valid url like `quay.io`
14+
* `BUILD_TAG`: This is the tag that will be applied to the image when built
15+
* `IMAGE_PREFIX`: The account in the registry (quay.io/jchauncey)
16+
17+
### Make targets
18+
* `info`: Print out what information will be used to build the docker image
19+
* `docker-build`: Builds the docker image
20+
* `docker-push`: Pushes the image to the specified registry
21+
* `kube-delete`: Remove the service, proxy pod, and replication controller from kubernetes
22+
* `kube-create`: Create the service, proxy pod, and replication controller in kubernetes
23+
* `kube-update`: Remove the replication controller and create a new one (this is useful for publishing new images)
24+
* `update-manifests`: This is a helpful target that is used to generate a temporary manifest with the dev image set in the `image` stanza.

‎influxdb/_scripts/deploy.sh‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Build and push Docker images to Docker Hub and quay.io.
4+
#
5+
6+
cd "$(dirname "$0")" || exit 1
7+
8+
echo "Logging into docker hub account!"
9+
docker login -e="$DOCKER_EMAIL" -u="$DOCKER_USERNAME" -p="$DOCKER_PASSWORD"
10+
echo "Building docker image and pushing to docker hub!"
11+
IMAGE_PREFIX=deisci make -C .. docker-build docker-push
12+
echo "Logging into quay.io account!"
13+
docker login -e="$QUAY_EMAIL" -u="$QUAY_USERNAME" -p="$QUAY_PASSWORD" quay.io
14+
echo "Building docker image and pushing to quay.io!"
15+
DEIS_REGISTRY=quay.io/ IMAGE_PREFIX=deisci make -C .. docker-build docker-push

‎influxdb/config.toml‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
reporting-disabled = false
2+
dir = ""
3+
bind-address = ":8088"
4+
5+
[meta]
6+
enabled = true
7+
dir = "/data/meta"
8+
bind-address = ":8088"
9+
http-bind-address = ":8091"
10+
https-enabled = false
11+
https-certificate = ""
12+
retention-autocreate = true
13+
election-timeout = "1s"
14+
heartbeat-timeout = "1s"
15+
leader-lease-timeout = "500ms"
16+
commit-timeout = "50ms"
17+
cluster-tracing = false
18+
raft-promotion-enabled = true
19+
logging-enabled = true
20+
pprof-enabled = false
21+
lease-duration = "1m0s"
22+
23+
[data]
24+
enabled = true
25+
dir = "/data/db"
26+
engine = "tsm1"
27+
max-wal-size = 104857600
28+
wal-flush-interval = "10m0s"
29+
wal-partition-flush-delay = "2s"
30+
wal-dir = "/data/db/wal"
31+
wal-logging-enabled = true
32+
wal-ready-series-size = 30720
33+
wal-compaction-threshold = 0.5
34+
wal-max-series-size = 1048576
35+
wal-flush-cold-interval = "5s"
36+
wal-partition-size-threshold = 52428800
37+
query-log-enabled = true
38+
cache-max-memory-size = 524288000
39+
cache-snapshot-memory-size = 26214400
40+
cache-snapshot-write-cold-duration = "1h0m0s"
41+
compact-full-write-cold-duration = "24h0m0s"
42+
max-points-per-block = 0
43+
data-logging-enabled = true
44+
45+
[cluster]
46+
force-remote-mapping = false
47+
write-timeout = "5s"
48+
shard-writer-timeout = "5s"
49+
shard-mapper-timeout = "5s"
50+
51+
[retention]
52+
enabled = true
53+
check-interval = "30m0s"
54+
55+
[shard-precreation]
56+
enabled = true
57+
check-interval = "10m0s"
58+
advance-period = "30m0s"
59+
60+
[admin]
61+
enabled = true
62+
bind-address = ":8083"
63+
https-enabled = false
64+
https-certificate = "/etc/ssl/influxdb.pem"
65+
66+
[monitor]
67+
store-enabled = true
68+
store-database = "_internal"
69+
store-interval = "10s"
70+
71+
[subscriber]
72+
enabled = true
73+
74+
[http]
75+
enabled = true
76+
bind-address = ":8086"
77+
auth-enabled = false
78+
log-enabled = true
79+
write-tracing = false
80+
pprof-enabled = false
81+
https-enabled = false
82+
https-certificate = "/etc/ssl/influxdb.pem"
83+
84+
[collectd]
85+
enabled = false
86+
# bind-address = ":25826"
87+
# database = "collectd"
88+
# retention-policy = ""
89+
# batch-size = 5000
90+
# batch-pending = 10
91+
# batch-timeout = "10s"
92+
# read-buffer = 0
93+
# typesdb = "/usr/share/collectd/types.db"
94+
95+
[opentsdb]
96+
enabled = false
97+
bind-address = ":4242"
98+
database = "opentsdb"
99+
retention-policy = ""
100+
consistency-level = "one"
101+
tls-enabled = false
102+
certificate = "/etc/ssl/influxdb.pem"
103+
batch-size = 1000
104+
batch-pending = 5
105+
batch-timeout = "1s"
106+
log-point-errors = true
107+
108+
[continuous_queries]
109+
log-enabled = true
110+
enabled = true
111+
run-interval = "1s"
112+
113+
[hinted-handoff]
114+
enabled = true
115+
dir = "/home/influxdb/.influxdb/hh"
116+
max-size = 1073741824
117+
max-age = "168h0m0s"
118+
retry-rate-limit = 0
119+
retry-interval = "1s"
120+
retry-max-interval = "1m0s"
121+
purge-interval = "1h0m0s"

‎influxdb/manifests/deis-monitor-influxdb-rc.tmp.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
apiVersion: v1
2+
kind: ReplicationController
3+
metadata:
4+
name: deis-monitor-influxdb
5+
namespace: deis
6+
labels:
7+
heritage: deis
8+
spec:
9+
replicas: 1
10+
selector:
11+
app: deis-monitor-influxdb
12+
template:
13+
metadata:
14+
labels:
15+
app: deis-monitor-influxdb
16+
spec:
17+
containers:
18+
- name: deis-monitor-influxdb
19+
image: quay.io/jchauncey/influxdb:git-0171fdc
20+
imagePullPolicy: Always
21+
ports:
22+
- containerPort: 8083
23+
name: admin
24+
- containerPort: 8086
25+
name: transport
26+
protocol: TCP
27+
- containerPort: 8084
28+
name: ssltransport
29+
protocol: TCP

‎influxdb/manifests/deis-monitor-influxdb-rc.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
apiVersion: v1
2+
kind: ReplicationController
3+
metadata:
4+
name: deis-monitor-influxdb
5+
namespace: deis
6+
labels:
7+
heritage: deis
8+
spec:
9+
replicas: 1
10+
selector:
11+
app: deis-monitor-influxdb
12+
template:
13+
metadata:
14+
labels:
15+
app: deis-monitor-influxdb
16+
spec:
17+
containers:
18+
- name: deis-monitor-influxdb
19+
image: quay.io/jchauncey/influxdb:0.9.6.1
20+
imagePullPolicy: Always
21+
ports:
22+
- containerPort: 8083
23+
name: admin
24+
- containerPort: 8086
25+
name: transport
26+
protocol: TCP
27+
- containerPort: 8084
28+
name: ssltransport
29+
protocol: TCP

‎influxdb/manifests/deis-monitor-influxdb-svc.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
apiVersion: v1
2+
kind: Service
3+
metadata:
4+
name: deis-monitor-influxdb
5+
namespace: deis
6+
labels:
7+
heritage: deis
8+
app: deis-monitor-influxdb
9+
spec:
10+
ports:
11+
- port: 8083
12+
name: admin
13+
targetPort: admin
14+
- port: 8086
15+
name: transport
16+
targetPort: transport
17+
protocol: TCP
18+
- port: 8084
19+
name: ssltransport
20+
targetPort: ssltransport
21+
protocol: TCP
22+
selector:
23+
app: deis-monitor-influxdb

0 commit comments

Comments
 (0)