docs: iteration on aws k8s upgrade docs #4099
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
consideRatio
force-pushed
the
pr/upgrade-aws-upgrade-docs
branch
from
938e9d6 to
818d0f2
Compare
consideRatio
commented
May 19, 2024
| ```{warning} | ||
| This upgrade will cause disruptions for users and trigger alerts for | ||
| [](uptime-checks). To help other engineers, communicate that your are starting a | ||
| cluster upgrade in the `#maintenance-notices` Slack channel and setup a [snooze](uptime-checks:snoozes) |
There was a problem hiding this comment.
We shouldn't need a snooze, so that was removed. I relocated the note on communicating in slack to a dedicated step.
consideRatio
commented
May 19, 2024
Comment on lines
-11
to
-15
| ```{warning} | ||
| We haven't yet established a policy for planning and communicating maintenance | ||
| procedures to users. So preliminary, only make a k8s cluster upgrade while the | ||
| cluster is unused or that the maintenance is communicated ahead of time. | ||
| ``` |
There was a problem hiding this comment.
We still haven't, but as long as we are only very little disruptive (brief networking etc) we can probably make upgrades while clusters are active anyhow.
I opted to remove this rather than refine this. I acknowledge we should have a policy, but I'd like us to collectively iterate and find agreement after collective experience rather than me declaring and motivating one in a PR about how to make a k8s upgrade technically.
consideRatio
force-pushed
the
pr/upgrade-aws-upgrade-docs
branch
2 times, most recently
from
e9d59e2 to
090ba6c
Compare
GeorgianaElena
approved these changes
May 21, 2024
consideRatio
force-pushed
the
pr/upgrade-aws-upgrade-docs
branch
from
090ba6c to
42dee07
Compare
|
Thanks for reviewing @GeorgianaElena!! I rebased and added a commit with some adjustments after trying them out myself. |
14 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Our docs previously assumed there were no users on the user nodes, assuming a re-creation upgrade strategy could be used. This assumption is removed in this iteration.
I removed the pre-requiesite section "Consider changes to
template.jsonnet" that I now consider too out of scope to be suggested in docs to be done during a k8s upgrade.Node upgrade strategies, AWS auth, notes on version skew, and maybe something more.
Added step to overview what goes on in the cluster before upgrading. This can for example be used to rule out that something broke because of the upgrade (because it was already broken).
Upgrading multiple clusters in parallell is reasonable, and I've now made it so that the guide is easier to scale to run in parallell.
Related
This was worked in preparation for #4009. If it wasn't done now, it would be harder to handle #4009 even though it wasn't part of scope of #4009 to get this done.
Review
I think the time efficient approach is to let this be practically reviewed by merging it and then using it - iterating on it further if needed to fix issues with it.