Skip to content

Commit 8c223b5

Browse files
author
Jonathan Chauncey
committed
docs(platform-logging): Add section on custom fluentd plugins
1 parent e744efd commit 8c223b5

2 files changed

Lines changed: 51 additions & 6 deletions

File tree

src/managing-workflow/platform-logging.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,3 +102,36 @@ Then run `helm install ./workflow --namespace deis` to install the modified char
102102
### Customizing:
103103

104104
We currently support logging information to Syslog, Elastic Search, and Sumo Logic. However, we will gladly accept pull requests that add support to other locations. For more information please visit the [fluentd repository](https://github.com/deis/fluentd).
105+
106+
107+
### Custom Fluentd Plugins
108+
109+
That are many output plugins available for [Fluentd](https://github.com/search?q=fluentd+output&ref=opensearch). But, we purposefully do not ship our Fluentd image with these installed. Instead, we provide a mechanism that allows users to install a plugin at startup time of the container and configure it.
110+
111+
If you would like to install a plugin you can set an environment variable such as the following: `FLUENTD_PLUGIN_N=some-fluentd-plugin` where N is a positive integer that is incremented for every plugin you wish to install. After you set this value you must then set the configuration text for the `FILTER` or `STORE` plugin you are installing. You can do that by setting `CUSTOM_STORE_N=configuration text` where N is the corresponding index value of the plugin you just installed.
112+
113+
Here is an example of setting the values directly in the manifest of the daemonset.
114+
115+
```
116+
env:
117+
- name: "FLUENTD_PLUGIN_1"
118+
value: "fluent-plugin-kafka"
119+
- name: "CUSTOM_STORE_1"
120+
value: |
121+
<store>
122+
@type kafka \
123+
default_topic some_topic
124+
</store>
125+
```
126+
127+
Or you could configure it using the `daemon_environment` key in the `values.yaml`:
128+
129+
```
130+
fluentd:
131+
daemon_environment:
132+
FLUENTD_PLUGIN_1: "fluent-plugin-kafka"
133+
CUSTOM_STORE_1: "|\n <store>\n @type kafka\n default_topic some_topic\n </store>"
134+
INSTALL_BUILD_TOOLS: "|\n true"
135+
```
136+
137+
For more information please see the [Custom Plugins](https://github.com/deis/fluentd#custom-plugins) section of the README.

src/managing-workflow/tuning-component-settings.md

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -112,12 +112,24 @@ BACKUPS_TO_RETAIN | number of base backups the backing store should retain (defa
112112

113113
## Customizing Fluentd
114114

115-
The following environment variables are tunable for [Fluentd][logger]:
116-
117-
Setting | Description
118-
----------------- | ---------------------------------
119-
SYSLOG_HOST_1 | The hostname of a remote syslog endpoint for shipping logs
120-
SYSLOG_PORT_1 | The port of a remote syslog endpoint for shipping logs
115+
The following values can be changed in the `values.yaml` file or by using the `--set` flag with the Helm CLI.
116+
117+
Key | Default | Description
118+
------------------| --------| ---------------------------------
119+
syslog.host | "" | Host value of a syslog endpoint
120+
syslog.port | "" | Port value of a syslog endpoint
121+
sources.start_script | false | Capture kubernetes start script logs
122+
sources.docker | false | Capture docker daemon logs
123+
sources.etcd | false | Capture etcd logs
124+
sources.kubelet | false | Capture kubelet logs
125+
sources.kube_api | false | Capture Kubernetes API logs
126+
sources.controller | false | Capture Kubernetes Controller logs
127+
sources.scheduler | false | Capture Kubernetes Scheduler logs
128+
output.disable_deis | false | Disable the Deis output plugin
129+
boot.install_build_tools | false | Install the build tools package. This is useful when using custom plugins
130+
daemon_environment | | Takes key-value pairs and turns them into environment variables.
131+
132+
For more information about the various environment variables that can be set please see the [README](https://github.com/deis/fluentd/blob/master/README.md)
121133

122134
## Customizing the Logger
123135

0 commit comments

Comments
 (0)