Skip to content

Commit 8d271ca

Browse files
slackslack
authored
doc(upgrading): expand and clarify workflow upgrade instructions (#447)
* doc(upgrading): expand upgrade documentation * doc(upgrading): use env vars during upgrade * bug(upgrading): missing } * doc(upgrading): text tweaks, fresh pod output
1 parent 12fd1d7 commit 8d271ca

1 file changed

Lines changed: 103 additions & 18 deletions

File tree

‎src/managing-workflow/upgrading-workflow.md‎

Lines changed: 103 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,9 @@ intact so routing and DNS are preserved while reinstalling Workflow.
3030

3131
See the Helm Classic documentation for more detail about [keeper manifests].
3232

33-
## Process
33+
## Upgrade Process
34+
35+
### Step 1: Verify component annotations and prepare upgrade
3436

3537
To verify that the namespace, router and registry are marked as "keepers" run the following kubectl command for each component:
3638

@@ -47,43 +49,126 @@ $ kubectl --namespace=deis annotate namespace deis helm-keep=true
4749
namespace "deis" annotated
4850
```
4951

50-
To upgrade to a newer version of Workflow, run the following:
52+
Exporting environment variables for the previous and latest versions will help reduce confusion later on:
53+
54+
```
55+
$ export PREVIOUS_WORKFLOW_RELEASE=v2.2.0
56+
$ export DESIRED_WORKFLOW_RELEASE=v2.3.0
57+
```
58+
59+
### Step 2: Fetch new charts
60+
61+
Workflow charts are always published with a version number intact. The command `helmc update` updates the local chart
62+
repository to the latest set of releases.
5163

5264
```
5365
# update the charts repo
5466
$ helmc update
67+
```
68+
69+
Fetching the new chart copies the chart from the chart cache into the helmc workspace for customization.
5570

56-
# fetch your current database credentials
71+
```
72+
# fetch new chart
73+
$ helmc fetch deis/workflow-${DESIRED_WORKFLOW_RELEASE}
74+
```
75+
76+
### Step 3: Fetch credentials
77+
78+
The first time Workflow is installed, Helm automatically generates secrets for the builder and database components.
79+
When upgrading, take care to use credentials from the running Workflow installation. The following commands export the
80+
secrets to the local workstation. They will be copied into place in a later step.
81+
82+
```
83+
# fetch the current database credentials to the local workstation
5784
$ kubectl --namespace=deis get secret database-creds -o yaml > ~/active-deis-database-secret-creds.yaml
58-
# fetch the ssh key for the builder component
85+
86+
# fetch the builder component ssh keys to the local workstation
5987
$ kubectl --namespace=deis get secret builder-ssh-private-keys -o yaml > ~/active-deis-builder-secret-ssh-private-keys.yaml
88+
```
6089

61-
# fetch new chart
62-
$ helmc fetch deis/workflow-v2.3.0
90+
### Step 4: Modify and update configuration
91+
92+
Before generating the manifests for the newest release, operators should update the new `generate_params.toml` to match
93+
configuration from the **previous release**.
6394

95+
```
6496
# update your off-cluster storage configuration
65-
$ $EDITOR $(helmc home)/workspace/charts/workflow-v2.3.0/tpl/generate_params.toml
97+
$ $EDITOR $(helmc home)/workspace/charts/workflow-${DESIRED_WORKFLOW_RELEASE}/tpl/generate_params.toml
98+
```
99+
100+
```
101+
# generate templates for the new release
102+
$ helmc generate -x manifests workflow-${DESIRED_WORKFLOW_RELEASE}
103+
```
66104

67-
# generate new templates
68-
$ helmc generate -x manifests workflow-v2.3.0
105+
### Step 5: Apply secrets
69106

70-
# copy your active database secrets into the helmc workspace
107+
After generating new manifests in the previous step, copy the current secrets into place:
108+
109+
```
110+
# copy your active database secrets into the helmc workspace for the desired version
71111
$ cp ~/active-deis-database-secret-creds.yaml \
72-
$(helmc home)/workspace/charts/workflow-v2.3.0/manifests/deis-database-secret-creds.yaml
112+
$(helmc home)/workspace/charts/workflow-${DESIRED_WORKFLOW_RELEASE}/manifests/deis-database-secret-creds.yaml
73113
74-
# copy your active builder ssh keys into the helmc workspace
114+
# copy your active builder ssh keys into the helmc workspace for the desired version
75115
$ cp ~/active-deis-builder-secret-ssh-private-keys.yaml \
76-
$(helmc home)/workspace/charts/workflow-v2.3.0/manifests/deis-builder-secret-ssh-private-keys.yaml
116+
$(helmc home)/workspace/charts/workflow-${DESIRED_WORKFLOW_RELEASE}/manifests/deis-builder-secret-ssh-private-keys.yaml
117+
```
118+
119+
!!! note
120+
Make sure to copy the existing credentials manifests into the new chart
121+
location *after* `helmc generate` to preserve credentials from the running system.
122+
123+
### Step 6: Apply the Workflow upgrade
124+
125+
Helm Classic will remove all components from the previous release that are **not** marked as keepers. As of Workflow
126+
2.3 and later, the controller, registry and router will be left in-service. Traffic to applications deployed through
127+
Worfklow will continue to flow between the `uninstall` and `install` commands.
77128

129+
If Workflow is not configured to use off-cluster Postgres the Workflow API will experience a brief period of downtime
130+
while the database component recovers from backup.
131+
132+
```
78133
# uninstall workflow
79-
$ helmc uninstall workflow-v2.1.0 -n deis
134+
$ helmc uninstall workflow-${PREVIOUS_WORKFLOW_RELEASE} -n deis
80135
81-
# install workflow v2.3.0
82-
$ helmc install workflow-v2.3.0
136+
# install latest workflow release
137+
$ helmc install workflow-${DESIRED_WORKFLOW_RELEASE}
83138
```
84139

85-
Make sure to copy the existing `deis-database-secret-creds.yaml` manifest into the new chart
86-
location *after* `helmc generate` to keep database credentials intact.
140+
### Step 7: Verify upgrade
141+
142+
Verify that all components have started and passed their readiness checks:
143+
144+
```
145+
$ kubectl --namespace=deis get pods
146+
$ kd get po
147+
NAME READY STATUS RESTARTS AGE
148+
deis-builder-2448122224-3cibz 1/1 Running 0 5m
149+
deis-controller-1410285775-ipc34 1/1 Running 3 5m
150+
deis-database-e7c5z 1/1 Running 0 5m
151+
deis-logger-cgjup 1/1 Running 3 5m
152+
deis-logger-fluentd-45h7j 1/1 Running 0 5m
153+
deis-logger-fluentd-4z7lw 1/1 Running 0 5m
154+
deis-logger-fluentd-k2wsw 1/1 Running 0 5m
155+
deis-logger-fluentd-skdw4 1/1 Running 0 5m
156+
deis-logger-redis-8nazu 1/1 Running 0 5m
157+
deis-monitor-grafana-tm266 1/1 Running 0 5m
158+
deis-monitor-influxdb-ah8io 1/1 Running 0 5m
159+
deis-monitor-telegraf-51zel 1/1 Running 1 5m
160+
deis-monitor-telegraf-cdasg 1/1 Running 0 5m
161+
deis-monitor-telegraf-hea6x 1/1 Running 0 5m
162+
deis-monitor-telegraf-r7lsg 1/1 Running 0 5m
163+
deis-nsqd-3yrg2 1/1 Running 0 5m
164+
deis-registry-1814324048-yomz5 1/1 Running 0 5m
165+
deis-registry-proxy-4m3o4 1/1 Running 0 5m
166+
deis-registry-proxy-no3r1 1/1 Running 0 5m
167+
deis-registry-proxy-ou8is 1/1 Running 0 5m
168+
deis-registry-proxy-zyajl 1/1 Running 0 5m
169+
deis-router-1357759721-a3ard 1/1 Running 0 5m
170+
deis-workflow-manager-2654760652-kitf9 1/1 Running 0 5m
171+
```
87172

88173
[configuring object storage]: ../installing-workflow/configuring-object-storage.md
89174
[keeper manifests]: http://helm-classic.readthedocs.io/en/latest/awesome/#keeper-manifests

0 commit comments

Comments
 (0)