Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

DragonFish Backup and Restore Rehaul #169

Closed
wants to merge 45 commits into from
Closed

DragonFish Backup and Restore Rehaul #169

wants to merge 45 commits into from

Conversation

Heavybullets8
Copy link
Owner

This pull request introduces a comprehensive rehaul of the previous backup and restore system, reducing reliance on the Middlewared API and adding robust custom solutions.

New Features:

  • Importing and Exporting:

    • Export dumps and cleans the values for use on other systems.
    • Import restores these configurations for seamless recovery or migration.

  • Backup All and Restore:

    • Complete replacement for middleware Backup/Restore Operations:
      • Snapshots of ix-applications dataset and children.
      • Snapshots and rollbacks of Persistent Volume Claims (PVCs).
      • Dumps of Cloud Native PostgreSQL (CNPG) databases.
    • Restore functionality ensures complete and accurate recovery of the application state.

Issues Closed:

This update provides a robust and reliable backup and restore solution, enhancing the overall management and recovery of applications.

@Heavybullets8 Heavybullets8 self-assigned this May 23, 2024
@Heavybullets8 Heavybullets8 marked this pull request as draft May 23, 2024 01:07
@Heavybullets8
Fix locks on refresh chart status
a564016
@Heavybullets8
Refactor restore database, implement stop/start logic for restore, mo…
0a70163 
…ve stop/start to utils
@Heavybullets8
revert to not use api chart queries for now.
b59ef0e
@Heavybullets8
Implement a temporary fix for missing chart data in cache
be2e086
@Heavybullets8
Refactor api chart fetching:
532d674 
If chart data on init is not valid, refresh the chart
Add rolling refreshes for monitoring operations
Avoid redundant calls to APIChartFetcher and ChartCache.
@Heavybullets8
chore: Refactor backup and sync logic in update handler
ddc7b0f
@Heavybullets8
Refactor backup and sync logic in update handler
95c697e
@Heavybullets8
create ix_volumes dataset if needed as well
ab72cd7
@Heavybullets8
change cnpg database check for printing CNPG title
79aa840
@Heavybullets8
create on dataset not pool
333725e
@Heavybullets8
cleanup cnpg pvc's to avoid duplication
97988ac
@Heavybullets8
fetch cnpg_pvcs_to_delete differently
d963352
@Heavybullets8
handle None type for get_file return
b502e75
@Heavybullets8
clean zfsvolume objects on backup
5005537
@Heavybullets8
Refactor database backup and restore to use class inheritance
6d3ca25
@Heavybullets8
Refactor Logging (#170)
1595ed2 
Refactor logging to store logs within the program directory, and not tied to the backups themselves.
@Heavybullets8
Fix parameter pass to restore CNPG database class
bbc1f8a
@Heavybullets8
also apply cleaning on object restore
cdba9e4
@Heavybullets8
write to a temp file for kubernetes objects restore
942158e
@Heavybullets8
add logging for cleaning objects
3bafeb7
@Heavybullets8
Snapshot volumes (#171)
3acd042 
Create backups of snapshots for ix_volumes and Persistent Volumes.
Allow configuration for the maximum size for those two.
@Heavybullets8
refactor custom dataset path
df03108
@Heavybullets8
Copy link
Owner Author

https://github.com/Heavybullets8/heavy_script?tab=readme-ov-file#archived-repository-notice

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees

@Heavybullets8 Heavybullets8

Labels
backup enhancement New feature or request pvc restore rollbacks
Projects
None yet
Milestone
No milestone
1 participant
@Heavybullets8