Skip to content

Commit 43ff1b6

Browse files
author
Jonathan Chauncey
committed
docs(installing/platform-logging): Update docs for off-cluster installs
1 parent 4196ca3 commit 43ff1b6

3 files changed

Lines changed: 54 additions & 10 deletions

File tree

src/installing-workflow/index.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -41,16 +41,16 @@ the master returned by `helmc target` matches the expected cluster.
4141
## Choose Your Deployment Strategy
4242

4343
Deis Workflow includes everything it needs to run out of the box. However, these defaults are aimed at simplicity rather than
44-
production readiness. Production and staging deployments of Workflow should, at a minimum, use off-cluster storage.
45-
Which is used by Workflow components to store and backup critical data. Should an operator need to completely re-install
44+
production readiness. Production and staging deployments of Workflow should, at a minimum, use off-cluster storage
45+
which is used by Workflow components to store and backup critical data. Should an operator need to completely re-install
4646
Workflow, the required components can recover from off-cluster storage. See the documentation for [configuring object
4747
storage](configuring-object-storage.md) for more details.
4848

49-
Workflow may also be configured to use off-cluster persistence for [Postgres](configuring-postgres.md) and
50-
Redis; a deployment strategy that mirrors the "stateless" clusters from the Deis v1 PaaS.
51-
52-
Last but not least, Workflow may also use a dedicated off-cluster image registry, including Docker Hub, Quay.io, ECR or
53-
GCR for all container images. Read more about [configuring your registry](configuring-registry.md).
49+
More rigorous installations would benefit from using outside sources for the following things:
50+
* [Postgres](configuring-postgres.md) - For example AWS RDS.
51+
* [Registry](configuring-registry.md) - This includes [quay.io](https://quay.io), [dockerhub](https://hub.docker.com), [Amazon ECR](https://aws.amazon.com/ecr/), and [Google GCR](https://cloud.google.com/container-registry/).
52+
* [Redis](../managing-workflow/platform-logging.md#configuring-off-cluster-redis) - Such as AWS Elasticache
53+
* [InfluxDB](../managing-workflow/platform-monitoring.md#configuring-off-cluster-influxdb) and [Grafana](../managing-workflow/platform-monitoring.md#off-cluster-grafana)
5454

5555
## Add the Deis Chart Repository
5656

src/managing-workflow/platform-logging.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,38 @@ If the message is from the [Workflow Router](https://github.com/deis/router) we
1010

1111
Logger then acts as a consumer reading messages off of the NSQ logs topic storing those messages in a local Redis instance. When a user wants to retrieve log entries using the `deis logs` command we make an HTTP request from Controller to Logger which then fetches the appropriate data from Redis.
1212

13+
## Configuring Off Cluster Redis
14+
Even though we provide a redis instance with the default Workflow install. It is recommended that operators use a 3rd party source like Elasticache or similar offering. This way your data is durable across upgrades or outages. If you have a 3rd party Redis installation you would like to use all you need to do is set the following values in `generate_params.toml` within your chart's tpl directory.
15+
16+
* db = "0"
17+
* host = "my.host.redis"
18+
* port = "6379"
19+
* password = ""
20+
21+
You can also provide this environment variables when you run your `helm generate` command instead of editing `generate_params.toml`.
22+
23+
* LOGGER_REDIS_LOCATION="off-cluster"
24+
* DEIS_LOGGER_REDIS_DB="0"
25+
* DEIS_LOGGER_REDIS_SERVICE_HOST="my.host.redis"
26+
* DEIS_LOGGER_REDIS_SERVICE_PORT="6379"
27+
28+
The database password can also be set as a kubernetes secret using the following name: `logger-redis-creds`.
29+
30+
```
31+
apiVersion: v1
32+
kind: Secret
33+
metadata:
34+
name: logger-redis-creds
35+
namespace: deis
36+
labels:
37+
app: deis-logger-redis
38+
heritage: deis
39+
annotations:
40+
helm-keep: "true"
41+
data:
42+
password: your-base64-password-here
43+
```
44+
1345
## Debugging Logger
1446
If the `deis logs` command encounters an error it will return the following message:
1547

src/managing-workflow/platform-monitoring.md

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ We now include a monitoring stack for introspection on a running Kubernetes clus
3838
└──────────┘
3939
```
4040

41-
### Grafana
41+
## Grafana
4242

4343
Deis Workflow exposes Grafana through the router using [service annotations](https://github.com/deis/router#how-it-works). This
4444
allows users to access the Grafana UI at `http://grafana.mydomain.com`. The default username/password of
@@ -54,7 +54,10 @@ Deis Workflow monitoring does not currently write data to the host filesystem or
5454
instance fails, modified dashboards are lost. Until there is a solution to persist this, export dashboards and store
5555
them separately in version control.
5656

57-
### InfluxDB
57+
### Off Cluster Grafana
58+
It is recommended that users provide their own installation for Grafana if possible. The current deployment of Grafana within Workflow is durable across pod restarts which means custom dashboards that are created after startup will not be restored when the pod comes back up. If you wish to provide your own Grafana instance you can either set the `GRAFANA_LOCATION` environment variable when your run `helm generate` or set `grafana_location` in the generate_params.toml.
59+
60+
## InfluxDB
5861

5962
InfluxDB writes data to the host disk, however, if the InfluxDB pod dies and comes back on
6063
another host the data will not be recovered. We intend to fix this in a future release. The InfluxDB Admin UI is also
@@ -69,7 +72,16 @@ You can choose to not expose the Influx UI and API to the world by updating
6972
`$CHART_HOME/workspace/workflow-$WORKFLOW_RELEASE/manifests/deis-monitor-influxdb-ui-svc.yaml` and removing the
7073
following line - `router.deis.io/routable: "true"`.
7174

72-
### Telegraf
75+
### Off Cluster Influxdb
76+
To use off-cluster Influx please provide the following values in either the `generate_params.toml` file or as environment variables when running `helm generate`.
77+
78+
* `influxdb_location=off-cluster` - `INFLUXDB_LOCATION=off-cluster`
79+
* `url = "http://my-influxhost.com:8086"` - `INFLUXDB_SERVICE_URL="http://my-influxhost.com:8086"`
80+
* `database = "metrics"` - `INFLUXDB_DATABASE="metrics"`
81+
* `user = "InfluxUser"` - `INFLUXDB_USER="InfluxUser"`
82+
* `password = "MysuperSecurePassword"` - `INFLUXDB_PASSWORD="MysuperSecurePassword"`
83+
84+
## Telegraf
7385

7486
Telegraf is the metrics collection daemon used within the monitoring stack. It will collect and send the following metrics to InfluxDB:
7587

0 commit comments

Comments
 (0)