Skip to content
This repository was archived by the owner on Jun 25, 2025. It is now read-only.

Commit 6596685

Browse files
author
Jonathan Chauncey
committed
docs(readme): Update readme with new architecture
1 parent 65ed719 commit 6596685

1 file changed

Lines changed: 48 additions & 46 deletions

File tree

README.md

Lines changed: 48 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -20,50 +20,21 @@ The new v2 logger implementation has seen a simplification from the last rewrite
2020

2121
We have also decided to not use `logspout` as the mechanism to get logs from each container to the `logger` component. Now we will use [fluentd](http://fluentd.org) which is a widely supported logging framework with hundreds of plugins. This will allow the end user to configure multiple destinations such as Elastic Search and other Syslog compatible endpoints like [papertrail](http://papertrailapp.com).
2222

23-
** This image requires that the `Daemon Sets` api be available on the kubernetes cluster** For more information on running the `Daemon Sets` api see the [following](https://github.com/kubernetes/kubernetes/blob/master/docs/api.md#enabling-resources-in-the-extensions-group).
24-
25-
## Running logger v2
23+
## Configuration
2624
The following environment variables can be used to configure logger:
2725

28-
* `STORAGE_ADAPTER`: How to store logs that are sent to the logger interface. Default is `memory`
29-
* `NUMBER_OF_LINES`: How many lines to store in the ring buffer. Default is `1000`.
30-
31-
### Installation
32-
Because of the requirement of Daemon Sets we have chosen not to include the logging components in the main [Deis chart](https://github.com/deis/charts/tree/master/deis).
33-
34-
To install the logging system please do the following:
35-
36-
```
37-
$ helm repo add deis https://github.com/deis/charts
38-
$ helm install deis/deis-logger
39-
```
40-
Watch for the logging components to come up:
41-
42-
```
43-
$ kubectl get pods --namespace=deis
44-
```
45-
46-
You should see output similar to this:
47-
48-
```
49-
NAME READY STATUS RESTARTS AGE
50-
deis-builder-knypb 1/1 Running 0 1d
51-
deis-database-ldbam 1/1 Running 0 2h
52-
deis-logger-fluentd-tq187 1/1 Running 0 1d
53-
deis-logger-iwos5 1/1 Running 0 2d
54-
deis-minio-zlmk8 1/1 Running 0 3d
55-
deis-registry-bys9n 1/1 Running 0 3d
56-
deis-router-h5f0i 1/1 Running 0 3d
57-
deis-workflow-2v84b 1/1 Running 3 2d
58-
```
59-
60-
After the logging components are installed you will need to restart the `deis-workflow` pod so it can pick up the new service endpoint for the logger component.
61-
62-
```
63-
$ kubectl delete pod <workflow pod name>
64-
```
65-
66-
The replication controller should immediately restart a new pod. Now you can use the `deis logs` command for your applications.
26+
* `DEIS_NSQD_SERVICE_HOST` - default:""
27+
* `DEIS_NSQD_SERVICE_PORT_TRANSPORT` - default:4150
28+
* `NSQ_TOPIC` - default:logs
29+
* `NSQ_CHANNEL` - default:consume
30+
* `NSQ_HANDLER_COUNT` - default:30
31+
* `AGGREGATOR_STOP_TIMEOUT_SEC` - default:1
32+
* `DEIS_LOGGER_REDIS_SERVICE_HOST` - default:""
33+
* `DEIS_LOGGER_REDIS_SERVICE_PORT` - default:6379
34+
* `DEIS_LOGGER_REDIS_PASSWORD` - default:""
35+
* `DEIS_LOGGER_REDIS_DB` - default:0
36+
* `DEIS_LOGGER_REDIS_PIPELINE_LENGTH` - default:50
37+
* `DEIS_LOGGER_REDIS_PIPELINE_TIMEOUT_SECONDS` - default:1
6738

6839
## Development
6940
The only assumption this project makes about your environment is that you have a working docker host to build the image against.
@@ -87,13 +58,44 @@ DEV_REGISTRY=myhost:5000 make push
8758
```
8859

8960
### Kubernetes interactions
90-
* `DEV_REGISTRY=quay.io IMAGE_PREFIX=myaccount make kube-create`: Does a sed replacement of the image name and creates a tmp manifest file that we will use to deploy logger component to kubernetes. This will also start 2 `fluentd` daemonsets.
91-
* `make kube-delete`: This will remove all the logger components from the kubernetes cluster.
92-
* `DEV_REGISTRY=quay.io IMAGE_PREFIX=myaccount make kube-replace`: This will rebuild the binary and image, push it to the remote registry, and then replace the running components with the new version.
61+
* `make install` - Install the recently built docker image into the kubernetes cluster
62+
* `make upgrade` - Upgrade a currently installed image
63+
* `make uninstall` - Uninstall logger from a kubernetes cluster
64+
65+
### Architecture Diagram
66+
```
67+
┌────────┐
68+
│ Router │ ┌────────┐
69+
└────────┘ │ Logger │
70+
│ └────────┘
71+
Log file │
72+
│ │
73+
▼ ▼
74+
┌────────┐ ┌─────────┐ logs/metrics ┌─────┐
75+
│App Logs│──Log File──▶│ fluentd │───────topics─────▶│ NSQ │
76+
└────────┘ └─────────┘ └─────┘
77+
78+
79+
┌─────────────┐ │
80+
│ HOST │ ▼
81+
│ Telegraf │───┐ ┌────────┐
82+
└─────────────┘ │ │Telegraf│
83+
│ └────────┘
84+
┌─────────────┐ │ │
85+
│ HOST │ │ ┌───────────┐ │
86+
│ Telegraf │───┼───▶│ InfluxDB │◀────Wire ─────────┘
87+
└─────────────┘ │ └───────────┘ Protocol
88+
│ ▲
89+
┌─────────────┐ │ │
90+
│ HOST │ │ ▼
91+
│ Telegraf │───┘ ┌──────────┐
92+
└─────────────┘ │ Grafana │
93+
└──────────┘
94+
```
9395

9496
## License
9597

96-
© 2015 Engine Yard, Inc.
98+
© 2016 Engine Yard, Inc.
9799

98100
Licensed under the Apache License, Version 2.0 (the "License"); you may
99101
not use this file except in compliance with the License. You may obtain

0 commit comments

Comments
 (0)