Skip to content

Commit 6dc4ce9

Browse files
duanhongyiduanhongyi
committed
chore(workflow): use redis stream replace nsqd
1 parent 5beb96b commit 6dc4ce9

8 files changed

Lines changed: 10 additions & 22 deletions

File tree

‎README.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@ Please see below for links and descriptions of each component:
2222
- [influxdb](https://github.com/drycc/influxdb) - The monitor database
2323
- [rabbitmq](https://github.com/drycc/rabbitmq) - RabbitMQ is a message broker used with controller celery
2424
- [minio](https://github.com/drycc/minio) - The in-cluster, ephemeral, development-only object storage system
25-
- [nsq](https://github.com/drycc/nsq) - Realtime distributed messaging platform
2625
- [workflow-cli](https://github.com/drycc/workflow-cli) - Workflow CLI `drycc`
2726

2827
We welcome your input! If you have feedback, please [submit an issue][issues]. If you'd like to participate in development, please read the "Working on Documentation" section below and [submit a pull request][prs].

‎_scripts/install.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -261,10 +261,10 @@ function install_helmbroker {
261261
--set persistence.size=${HELMBROKER_PERSISTENCE_SIZE:-5Gi} \
262262
--set persistence.storageClass=${HELMBROKER_PERSISTENCE_STORAGE_CLASS:-"openebs-kernel-nfs"} \
263263
--set platformDomain=${PLATFORM_DOMAIN} \
264-
--set certManagerEnabled=${CERT_MANAGER_ENABLED:-true} \
264+
--set certManagerEqnabled=${CERT_MANAGER_ENABLED:-true} \
265265
--set username=${HELMBROKER_USERNAME} \
266266
--set password=${HELMBROKER_PASSWORD} \
267-
--set environment.HELMBROKER_CELERY_BROKER="amqp://${RABBITMQ_USERNAME}:${RABBITMQ_PASSWORD}@drycc-rabbitmq-0.drycc-rabbitmq.drycc.svc.cluster.local:5672/drycc" \
267+
--set environment.HELMBROKER_CELERY_BROKER="amqp://${RABBITMQ_USERNAME}:${RABBITMQ_PASSWORD}@drycc-rabbitmq.drycc.svc.cluster.local:5672/drycc" \
268268
--namespace drycc --create-namespace --wait -f - <<EOF
269269
repositories:
270270
- name: drycc-helm-broker

‎charts/workflow/requirements.yaml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,6 @@ dependencies:
2626
- name: monitor
2727
version: <monitor-tag>
2828
repository: https://charts.drycc.cc/stable/monitor
29-
- name: nsqd
30-
version: <nsqd-tag>
31-
repository: https://charts.drycc.cc/stable/nsqd
3229
- name: registry
3330
version: <registry-tag>
3431
repository: https://charts.drycc.cc/stable/registry

‎charts/workflow/values.yaml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -205,9 +205,6 @@ database:
205205
username: ""
206206
password: ""
207207

208-
nsqd:
209-
replicas: 1
210-
211208
redis:
212209
# The following parameters are configured only when using an on-cluster Redis instance
213210
replicas: 1

‎src/installing-workflow/index.md‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,6 @@ $ helm install --namespace drycc \
8484
--set logger.imageRegistry=quay.io \
8585
--set minio.imageRegistry=quay.io \
8686
--set monitor.imageRegistry=quay.io \
87-
--set nsqd.imageRegistry=quay.io \
8887
--set registry.imageRegistry=quay.io \
8988
--set registry-proxy.imageRegistry=quay.io \
9089
--set global.platformDomain=drycc.cc \
@@ -124,7 +123,6 @@ drycc-redis-304849759-1f35p 1/1 Running 0 4m
124123
drycc-minio-676004970-nxqgt 1/1 Running 0 4m
125124
drycc-monitor-grafana-432627134-lnl2h 1/1 Running 0 4m
126125
drycc-monitor-telegraf-wmcmn 1/1 Running 1 4m
127-
drycc-nsqd-3597503299-6mn2x 1/1 Running 0 4m
128126
drycc-registry-756475849-lwc6b 1/1 Running 1 4m
129127
drycc-registry-proxy-96c4p 1/1 Running 0 4m
130128
drycc-rabbitmq-0 1/1 Running 0 4m

‎src/managing-workflow/platform-logging.md‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,9 @@ The logging platform is made up of 2 components - [Fluentd](https://github.com/d
44

55
[Fluentd](https://github.com/drycc/fluentd) runs on every worker node of the cluster and is deployed as a [Daemon Set](http://kubernetes.io/v1.1/docs/admin/daemons.html). The Fluentd pods capture all of the stderr and stdout streams of every container running on the host (even those not hosted directly by kubernetes). Once the log message arrives in our [custom fluentd plugin](https://github.com/drycc/fluentd/tree/main/rootfs/opt/fluentd/drycc-output) we determine where the message originated.
66

7-
If the message was from the [Workflow Controller](https://github.com/drycc/controller) or from an application deployed via workflow we send it to the logs topic on the local [NSQD](http://nsq.io) instance.
7+
If the message was from the [Workflow Controller](https://github.com/drycc/controller) or from an application deployed via workflow we send it to the logs topic on the local [Redis Stream](http://redis.io) instance.
88

9-
If the message is from the [Workflow Router](https://github.com/drycc/router) we build an Influxdb compatible message and send it to the same NSQD instance but instead place the message on the metrics topic.
10-
11-
Logger then acts as a consumer reading messages off of the NSQ logs topic storing those messages in a local Redis instance. When a user wants to retrieve log entries using the `drycc logs` command we make an HTTP request from Controller to Logger which then fetches the appropriate data from Redis.
9+
Logger then acts as a consumer reading messages off of the Redis Stream logs topic storing those messages in a local Redis instance. When a user wants to retrieve log entries using the `drycc logs` command we make an HTTP request from Controller to Logger which then fetches the appropriate data from Redis.
1210

1311
## Configuring Off Cluster Redis
1412

@@ -44,9 +42,9 @@ Error: There are currently no log messages. Please check the following things:
4442
Log file ▲
4543
│ │
4644
▼ │
47-
┌────────┐ ┌─────────┐ logs/metrics ┌─────┐
48-
│App Logs│──Log File──▶│ fluentd │───────topics─────▶│ NSQ │
49-
└────────┘ └─────────┘ └─────┘
45+
┌────────┐ ┌─────────┐ logs/metrics ┌──────────────┐
46+
│App Logs│──Log File──▶│ fluentd │───────topics─────▶│ Redis Stream │
47+
└────────┘ └─────────┘ └──────────────┘
5048
│
5149
│
5250
┌─────────────┐ │

‎src/managing-workflow/platform-monitoring.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,9 @@ We now include a monitoring stack for introspection on a running Kubernetes clus
1818
Log file ▲
1919
│ │
2020
▼ │
21-
┌────────┐ ┌─────────┐ logs/metrics ┌─────┐
22-
│App Logs│──Log File──▶│ fluentd │───────topics─────▶│ NSQ │
23-
└────────┘ └─────────┘ └─────┘
21+
┌────────┐ ┌─────────┐ logs/metrics ┌──────────────┐
22+
│App Logs│──Log File──▶│ fluentd │───────topics─────▶│ Redis Stream │
23+
└────────┘ └─────────┘ └──────────────┘
2424
│
2525
│
2626
┌─────────────┐ │

‎src/managing-workflow/upgrading-workflow.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,6 @@ drycc-monitor-telegraf-51zel 1/1 Running 1 5m
6868
drycc-monitor-telegraf-cdasg 1/1 Running 0 5m
6969
drycc-monitor-telegraf-hea6x 1/1 Running 0 5m
7070
drycc-monitor-telegraf-r7lsg 1/1 Running 0 5m
71-
drycc-nsqd-3yrg2 1/1 Running 0 5m
7271
drycc-registry-1814324048-yomz5 1/1 Running 0 5m
7372
drycc-registry-proxy-4m3o4 1/1 Running 0 5m
7473
drycc-registry-proxy-no3r1 1/1 Running 0 5m

0 commit comments

Comments
 (0)