feat: document new backup/restore procedure and webapps specific config #4791

entangled90 · 2024-12-18T13:16:18Z

Description

This PR updates the documentation related to performing backups and restoring them for self managed as a result EPIC camunda/camunda#24456.

Backups for webapps can now be performed by any webapps (operate, tasklist or optimize) and there is no need anymore to perform one for each app.

The order of the operations has also changed (source: decision document):
1 - Soft Pause on Zeebe to export - POST actuator/exporting/pause?soft=true
2 - SaaS Controller triggers WebApps and monitor it together
3 - Backup Zeebe ES Records
4 - Backup Zeebe Data
5 - Resume exporting

When should this change go live?

This is a bug fix, security concern, or something that needs urgent release support.
This is already available but undocumented and should be released within a week.
This on a specific schedule and the assignee will coordinate a release with the DevEx team. (apply hold label or convert to draft PR)
This is part of a scheduled alpha or minor. (apply alpha or minor label)
There is no urgency with this change and can be released at any time.

PR Checklist

My changes are for an already released minor and are in /versioned_docs directory.
My changes are for the next minor and are in /docs directory (aka /next/).

My changes require an Engineering review, and I've assigned the Engineering DRI or delegate.
My changes require a technical writer review, and I've assigned @camunda/tech-writers as a reviewer.

Related issues

#4790
#4717

github-actions · 2024-12-18T13:16:42Z

👋 🤖 🤔 Hello, @conceptualshark! Did you make your changes in all the right places?

These files were changed only in docs/. You might want to duplicate these changes in versioned_docs/version-8.6/.

docs/self-managed/operational-guides/backup-restore/backup-and-restore.md
docs/self-managed/operational-guides/backup-restore/optimize-backup.md
docs/self-managed/operational-guides/backup-restore/webapps-backup.md

These files were changed only in docs/. You might want to duplicate these changes in versioned_docs/version-8.7/.

docs/self-managed/operational-guides/backup-restore/backup-and-restore.md
docs/self-managed/operational-guides/backup-restore/optimize-backup.md
docs/self-managed/operational-guides/backup-restore/webapps-backup.md

You may have done this intentionally, but we wanted to point it out in case you didn't. You can read more about the versioning within our docs in our documentation guidelines.

akeller · 2025-01-06T17:33:07Z

@entangled90 is this part of alpha3? Or 8.7? I see its still in draft, but I'm not sure how quickly a tech writer would need to review this PR. Can you add the appropriate alpha or minor labels?

ChrisKujawa

Thanks @entangled90 I have smaller comments and some questions, but in general looks good 👍🏼

ChrisKujawa · 2025-01-20T11:51:41Z

docs/self-managed/operational-guides/backup-restore/backup-and-restore.md

@@ -6,7 +6,7 @@ keywords: ["backup", "backups"]
 ---

 :::note
-This release introduces breaking changes for [Operate and Tasklist](./operate-tasklist-backup.md), as well as [Optimize](./optimize-backup.md).
+This release introduces breaking changes for [Operate, Tasklist and Optimize](./webapps-backup.md).


❓ Do we need to specify an exact release (to reduce confusion)? Or is this obvious due to the version picker?

wdyt @akeller ?

Great question, given that it's a breaking change, please specify the exact release @ChrisKujawa.

docs/self-managed/operational-guides/backup-restore/backup-and-restore.md

docs/self-managed/operational-guides/backup-restore/webapps-backup.md

ChrisKujawa · 2025-01-20T12:02:29Z

docs/self-managed/operational-guides/backup-restore/webapps-backup.md

 1. The [Elasticsearch snapshot repository](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html) must be configured.
-2. Operate and Tasklist must be configured with the repository name using one of the following configuration options:
+2. All deployed webapps must be configured with the **same** repository name using one of the following configuration options.


🔧 Should we also mention that we need permissions to write ?

ChrisKujawa · 2025-01-20T12:03:07Z

docs/self-managed/operational-guides/backup-restore/webapps-backup.md

+### Index prefix
+
+:::warning breaking change
+Since version 8.7 the `indexPrefix` of all webapps must match. By default it's set to `""`, but if it was overriden, then it must done consistently across Operate, Tasklist and Optimize.


ChrisKujawa · 2025-01-20T12:05:00Z

docs/self-managed/operational-guides/backup-restore/webapps-backup.md


 ```
-POST actuator/backups
+POST actuator/backupHistory


🙃 I guess it is a bit late for this, but why is it actually backupHistory? It is backing up more than history, right? It feels like it is not syntactically correct somehow 🤔

Yes, but we never found a better term. I think we had backupZeebe and backupWebapps as proposal, but in the end the current names were chosen

ChrisKujawa · 2025-01-20T12:07:36Z

docs/self-managed/operational-guides/backup-restore/webapps-backup.md

+1. Stop all webapps.
+2. Ensure there are no webapp indices present in Elasticsearch (otherwise the restore process will fail).
 3. Iterate over all Elasticsearch snapshots included in the desired backup and restore them using the Elasticsearch restore snapshot API.
-4. Start Operate.
-
-## Backup and restore of Tasklist data
-
-Backup and restore of Tasklist may be performed in exactly the same way as [Operate data](#).
+4. Start all webapps.


❓ Should we note that Zeebe also need to be restored? Because webapp independently from Zeebe restoring will not really work right?

Yes, this is mentioned in the root document.
I think this section just expand on what "restoring webapps" means

ChrisKujawa · 2025-01-20T12:31:17Z

One thing I missed. I think we should a note to the announcements https://docs.camunda.io/docs/next/reference/announcements/announcements-870/ regarding the simplifications of backups, but also the breaking change.

npepinpe · 2025-01-20T19:52:33Z

@entangled90 - do you need my review as well as Chris', or is his enough?

entangled90 · 2025-01-21T08:39:40Z

I think Chris review is enough 👍

conceptualshark · 2025-01-21T13:37:36Z

Is this ready for a technical writing review?

entangled90 · 2025-01-22T08:47:55Z

I just realized that there are a couple of points that this PR does not address:

Identity: we are performing backup of identity indices as well, so I guess it should be mentioned, but I don't have much context about that.
Opensearch: currently only Elasticsearch is mentioned, I assume it should have already been mentioned considering that we already support it. Can we address this in a separate PR?

akeller · 2025-01-22T23:10:32Z

Hi - I see this PR is now labeled for 8.8. Please do not merge! I've added a hold label. Watch #ask-c8-documentation for when you can merge 8.8 docs items. If this is applicable to 8.7 too, adjust the labels.

entangled90 · 2025-01-23T14:40:26Z

Updated with the latest suggestions.

ChrisKujawa · 2025-02-10T12:26:16Z

@akeller @entangled90 how can we get this merged?

@akeller should we adjust it based on recent changes in the docs, or do you take care of this?

entangled90 · 2025-02-10T13:18:57Z

I added the 8.8.0-alpha1 as the functionality will be available in this version.

akeller · 2025-02-10T15:49:49Z

@akeller @entangled90 how can we get this merged?

@akeller should we adjust it based on recent changes in the docs, or do you take care of this?

@ChrisKujawa if you merge now, it will be included in 8.7 and 8.8. If you wait to merge, it will need to be (back)ported if it needs to be in 8.7 too.

There is currently a merge conflict.

relates #4790

Co-authored-by: Christopher Kujawa (Zell) <[email protected]>

akeller · 2025-02-10T22:14:55Z

Main now includes the 8.8 version as /next/. You can review & merge.

entangled90 · 2025-02-20T10:31:33Z

Can someone with write access review it?

conceptualshark

I've made a few adjustments to wording and this reads well to me. We've traditionally avoided referring to any of the components as "webapps" in C8 documentation, so if that is the method we're leaning into for describing these components, I've adjusted to "web application".

If this is introducing a breaking change, we should include an update in the 8.8 announcements as well. I think we could just make a new "Backup and restore" section there, unless there is a better section/header?

entangled90 · 2025-02-20T14:31:18Z

Yes, it's probably ok to place it under the section "Camunda Exporter" as it's in some way related to the new schema.

ChrisKujawa

We need to reiterate over it as Optimize backups will change again..

ChrisKujawa · 2025-02-20T18:44:31Z

docs/self-managed/operational-guides/backup-restore/webapps-backup.md


 :::note
-Operate must **not** be running while a backup restore is taking place.
+Operate, Tasklist and Optimize must **not** be running while a backup restore is taking place.


❌ We need to fix this.

akeller · 2025-03-03T20:37:35Z

What's the status of this PR @entangled90? Are you tracking for the next alpha release? If so, please adjust the label to alpha2.

ChrisKujawa · 2025-03-04T06:29:59Z

What's the status of this PR @entangled90? Are you tracking for the next alpha release? If so, please adjust the label to alpha2.

Carlo is on parental leave. I had no time to adjust it

akeller · 2025-03-06T21:24:31Z

What's the status of this PR @entangled90? Are you tracking for the next alpha release? If so, please adjust the label to alpha2.

Carlo is on parental leave. I had no time to adjust it

@npepinpe was there a handover for this docs PR? Can we reset the expectations for what alpha it should be in a somewhat deliverable/mergable state?

npepinpe · 2025-03-07T11:21:43Z

No, parental leave happened earlier than expected before the handover.

This targets 8.8, but we should really have it in the next alpha as the code changes it references will be included.

I will sync with Chris so we include it in what needs to be done before the handover of the epic.

entangled90 added component:zeebe Issues related with Zeebe project component:operate Issues related with Operate project component:tasklist Issues related with Tasklist project component:optimize Issues related with Optimize project labels Dec 18, 2024

entangled90 self-assigned this Dec 18, 2024

entangled90 requested a review from ChrisKujawa December 19, 2024 10:00

entangled90 force-pushed the cs-4790-new-backup-restore-procedure branch from b18d8db to 4bd252d Compare December 19, 2024 10:02

entangled90 added the target:8.7 Issues included in the 8.7 release label Jan 7, 2025

entangled90 marked this pull request as ready for review January 7, 2025 08:19

entangled90 requested review from npepinpe and Szik January 8, 2025 10:35

ChrisKujawa previously approved these changes Jan 20, 2025

View reviewed changes

entangled90 added target:8.8 Issues included in the 8.8 release and removed target:8.7 Issues included in the 8.7 release labels Jan 21, 2025

entangled90 dismissed ChrisKujawa’s stale review via 306d373 January 22, 2025 08:18

akeller added the hold This issue is parked, do not merge. label Jan 22, 2025

entangled90 force-pushed the cs-4790-new-backup-restore-procedure branch from f3cd457 to 07d87b6 Compare January 23, 2025 14:39

This was referenced Feb 10, 2025

Update backups documentation camunda/camunda#24480

Open

[EPIC]: Backups Implementation - 8.7 camunda/camunda#24456

Closed

entangled90 added the 8.8.0-alpha1 label Feb 10, 2025

akeller requested a review from a team February 10, 2025 15:50

entangled90 and others added 3 commits February 10, 2025 16:51

feat: document new backup/restore procedure and webapps specific config

0fe5138

relates #4790

doc: applied suggestions from review

39ebaf0

Co-authored-by: Christopher Kujawa (Zell) <[email protected]>

doc: change release version to 8.8 and fix management port note

c05b3e6

entangled90 force-pushed the cs-4790-new-backup-restore-procedure branch from 07d87b6 to c05b3e6 Compare February 10, 2025 15:52

akeller removed the hold This issue is parked, do not merge. label Feb 10, 2025

conceptualshark requested review from conceptualshark and removed request for a team February 20, 2025 12:31

tw review

89e0d5e

conceptualshark reviewed Feb 20, 2025

View reviewed changes

ChrisKujawa reviewed Feb 20, 2025

View reviewed changes

akeller mentioned this pull request Mar 4, 2025

Review and update backup & restore page #3266

Open

npepinpe added 8.8.0-alpha2 and removed 8.8.0-alpha1 labels Mar 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: document new backup/restore procedure and webapps specific config #4791

feat: document new backup/restore procedure and webapps specific config #4791

entangled90 commented Dec 18, 2024 •

edited

Loading

github-actions bot commented Dec 18, 2024 •

edited

Loading

akeller commented Jan 6, 2025

ChrisKujawa left a comment

ChrisKujawa Jan 20, 2025

akeller Jan 22, 2025

ChrisKujawa Jan 20, 2025

ChrisKujawa Jan 20, 2025

ChrisKujawa Jan 20, 2025

entangled90 Jan 22, 2025 •

edited

Loading

ChrisKujawa Jan 20, 2025

entangled90 Jan 22, 2025 •

edited

Loading

ChrisKujawa commented Jan 20, 2025

npepinpe commented Jan 20, 2025

entangled90 commented Jan 21, 2025

conceptualshark commented Jan 21, 2025

entangled90 commented Jan 22, 2025

akeller commented Jan 22, 2025

entangled90 commented Jan 23, 2025

ChrisKujawa commented Feb 10, 2025

entangled90 commented Feb 10, 2025

akeller commented Feb 10, 2025 •

edited

Loading

akeller commented Feb 10, 2025

entangled90 commented Feb 20, 2025

conceptualshark left a comment

entangled90 commented Feb 20, 2025

ChrisKujawa left a comment

ChrisKujawa Feb 20, 2025

akeller commented Mar 3, 2025

ChrisKujawa commented Mar 4, 2025

akeller commented Mar 6, 2025 •

edited

Loading

npepinpe commented Mar 7, 2025

feat: document new backup/restore procedure and webapps specific config #4791

Are you sure you want to change the base?

feat: document new backup/restore procedure and webapps specific config #4791

Conversation

entangled90 commented Dec 18, 2024 • edited Loading

Description

When should this change go live?

PR Checklist

Related issues

github-actions bot commented Dec 18, 2024 • edited Loading

akeller commented Jan 6, 2025

ChrisKujawa left a comment

Choose a reason for hiding this comment

ChrisKujawa Jan 20, 2025

Choose a reason for hiding this comment

akeller Jan 22, 2025

Choose a reason for hiding this comment

ChrisKujawa Jan 20, 2025

Choose a reason for hiding this comment

ChrisKujawa Jan 20, 2025

Choose a reason for hiding this comment

ChrisKujawa Jan 20, 2025

Choose a reason for hiding this comment

entangled90 Jan 22, 2025 • edited Loading

Choose a reason for hiding this comment

ChrisKujawa Jan 20, 2025

Choose a reason for hiding this comment

entangled90 Jan 22, 2025 • edited Loading

Choose a reason for hiding this comment

ChrisKujawa commented Jan 20, 2025

npepinpe commented Jan 20, 2025

entangled90 commented Jan 21, 2025

conceptualshark commented Jan 21, 2025

entangled90 commented Jan 22, 2025

akeller commented Jan 22, 2025

entangled90 commented Jan 23, 2025

ChrisKujawa commented Feb 10, 2025

entangled90 commented Feb 10, 2025

akeller commented Feb 10, 2025 • edited Loading

akeller commented Feb 10, 2025

entangled90 commented Feb 20, 2025

conceptualshark left a comment

Choose a reason for hiding this comment

entangled90 commented Feb 20, 2025

ChrisKujawa left a comment

Choose a reason for hiding this comment

ChrisKujawa Feb 20, 2025

Choose a reason for hiding this comment

akeller commented Mar 3, 2025

ChrisKujawa commented Mar 4, 2025

akeller commented Mar 6, 2025 • edited Loading

npepinpe commented Mar 7, 2025

entangled90 commented Dec 18, 2024 •

edited

Loading

github-actions bot commented Dec 18, 2024 •

edited

Loading

entangled90 Jan 22, 2025 •

edited

Loading

entangled90 Jan 22, 2025 •

edited

Loading

akeller commented Feb 10, 2025 •

edited

Loading

akeller commented Mar 6, 2025 •

edited

Loading