init

.gitignore

@@ -1,2 +1,3 @@
 docker-compose.yml
-.DS_Store
+.DS_Store
+.env

Dockerfile

@@ -5,12 +5,12 @@ RUN apk add --no-cache mysql-client aws-cli curl bash
 # Set Default Environment Variables
 ENV TARGET_DATABASE_PORT=3306
 ENV NOTIFY_ENABLED=false
+ENV HEALTHCHECK_ENABLED=false
 
-# Copy Slack Alert script and make executable
-COPY resources/notify.sh /
-RUN chmod +x /notify.sh
+WORKDIR /scripts
 
-# Copy backup script and execute
-COPY resources/perform-backup.sh /
-RUN chmod +x /perform-backup.sh
-CMD ["bash", "/perform-backup.sh"]
+# Copy scripts and make them executable
+COPY scripts/ /scripts/
+RUN chmod -R +x /scripts
+
+CMD ["bash", "/scripts/backup.sh"]

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Thorsten Klein
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,152 +1,33 @@
-# kubernetes-s3-mysql-backup
+# kubernetes-backup-mysql
 
-kubernetes-s3-mysql-backup is a container image based on Alpine Linux. This container is designed to run in Kubernetes as a cronjob to perform automatic backups of MySQL databases to Amazon S3. It was created to meet my requirements for regular and automatic database backups. Having started with a relatively basic feature set, it is gradually growing to add more and more features.
+## Overview
 
-Currently, kubernetes-s3-mysql-backup supports the backing up of MySQL Databases. It can perform backups of multiple MySQL databases from a single database host. When triggered, a full database dump is performed using the `mysqldump` command for each configured database. The backup(s) are then uploaded to an Amazon S3 Bucket. kubernetes-s3-mysql-backup features Slack Integration, and can post messages into a channel detailing if the backup(s) were successful or not.
+- **Backup**: Backup MySQL databases to S3
+- **Notify**: Notify about the status of the backup, e.g. via Webhook (Slack, etc.) or Telegram
+- **Healthcheck**: Ping about the status of the backup
 
-Over time, kubernetes-s3-mysql-backup will be updated to support more features and functionality. I currently use this container as part of my Kubernetes Architecture which you can read about [here](https://benjamin.maynard.io/this-blog-now-runs-on-kubernetes-heres-the-architecture/).
+## Object Storage IAM Policy
 
-All changes are captured in the [changelog](CHANGELOG.md), which adheres to [Semantic Versioning](https://semver.org/spec/vadheres2.0.0.html).
+Here's an example IAM policy that would work for the backup job.
 
-
-## Environment Variables
-
-The below table lists all of the Environment Variables that are configurable for kubernetes-s3-mysql-backup.
-
-| Environment Variable        | Purpose                                                                                                          |
-| --------------------------- |------------------------------------------------------------------------------------------------------------------|
-| AWS_ACCESS_KEY_ID           | **(Required)** AWS IAM Access Key ID.                                                                            |
-| AWS_SECRET_ACCESS_KEY       | **(Required)** AWS IAM Secret Access Key. Should have very limited IAM permissions (see below for example) and should be configured using a Secret in Kubernetes.                                                                                                         |
-| AWS_DEFAULT_REGION          | **(Required)** Region of the S3 Bucket (e.g. eu-west-2).                                                         |
-| AWS_BUCKET_NAME             | **(Required)** The name of the S3 bucket.                                                                        |
-| AWS_BUCKET_BACKUP_PATH      | **(Required)** Path the backup file should be saved to in S3. E.g. `/database/myblog/backups`. **Do not put a trailing / or specify the filename.**                                                                                                        |
-| TARGET_DATABASE_HOST        | **(Required)** Hostname or IP address of the MySQL Host.                                                         |
-| TARGET_DATABASE_PORT        | **(Optional)** Port MySQL is listening on (Default: 3306).                                                       |
-| TARGET_DATABASE_NAMES       | **(Required)** Name of the databases to dump. This should be comma seperated (e.g. `database1,database2`).       |
-| TARGET_DATABASE_USER        | **(Required)** Username to authenticate to the database with.                                                    |
-| TARGET_DATABASE_PASSWORD    | **(Required)** Password to authenticate to the database with. Should be configured using a Secret in Kubernetes. |
-| NOTIFY_ENABLED               | **(Optional)** (true/false) Enable or disable the Slack Integration (Default False).                             |
-| NOTIFY_USERNAME              | **(Optional)** (true/false) Username to use for the Slack Integration (Default: kubernetes-s3-mysql-backup).            |
-| NOTIFY_CHANNEL               | **(Required if Slack enabled)** Slack Channel the WebHook is configured for.                                     |
-| NOTIFICATION_WEBHOOK_URL           | **(Required if Slack enabled)** What is the Slack WebHook URL to post to? Should be configured using a Secret in Kubernetes.                                                                                                                                      |
-
-
-## Slack Integration
-
-kubernetes-s3-mysql-backup supports posting into Slack after each backup job completes. The message posted into the Slack Channel varies as detailed below:
-
-* If the backup job is **SUCCESSFUL**: A generic message will be posted into the Slack Channel detailing that all database backups successfully completed.
-* If the backup job is **UNSUCCESSFUL**: A message will be posted into the Slack Channel with a detailed error message for each database that failed.
-
-In order to configure kubernetes-s3-mysql-backup to post messages into Slack, you need to create an [Incoming WebHook](https://api.slack.com/incoming-webhooks). Once generated, you can configure kubernetes-s3-mysql-backup using the environment variables detailed above.
-
-## Configuring the S3 Bucket & AWS IAM User
-
-kubernetes-s3-mysql-backup performs a backup to the same path, with the same filename each time it runs. It therefore assumes that you have Versioning enabled on your S3 Bucket. A typical setup would involve S3 Versioning, with a Lifecycle Policy.
-
-An IAM Users should be created, with API Credentials. An example Policy to attach to the IAM User (for a minimal permissions set) is as follows:
-
-```
+```json
 {
     "Version": "2012-10-17",
     "Statement": [
         {
             "Sid": "VisualEditor0",
             "Effect": "Allow",
             "Action": "s3:ListBucket",
-            "Resource": "arn:aws:s3:::<BUCKET NAME>"
+            "Resource": "arn:aws:s3:::<BUCKET_NAME>"
         },
         {
             "Sid": "VisualEditor1",
             "Effect": "Allow",
             "Action": [
                 "s3:PutObject"
             ],
-            "Resource": "arn:aws:s3:::<BUCKET NAME>/*"
+            "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
         }
     ]
 }
 ```
-
-
-## Example Kubernetes Cronjob
-
-An example of how to schedule this container in Kubernetes as a cronjob is below. This would configure a database backup to run each day at 01:00am. The AWS Secret Access Key, and Target Database Password are stored in secrets.
-
-```
-apiVersion: v1
-kind: Secret
-metadata:
-  name: AWS_SECRET_ACCESS_KEY
-type: Opaque
-data:
-  aws_secret_access_key: <AWS Secret Access Key>
----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: TARGET_DATABASE_PASSWORD
-type: Opaque
-data:
-  database_password: <Your Database Password>
----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: NOTIFICATION_WEBHOOK_URL
-type: Opaque
-data:
-  slack_webhook_url: <Your Slack WebHook URL>
----
-apiVersion: batch/v1beta1
-kind: CronJob
-metadata:
-  name: my-database-backup
-spec:
-  schedule: "0 01 * * *"
-  jobTemplate:
-    spec:
-      template:
-        spec:
-          containers:
-          - name: my-database-backup
-            image: gcr.io/maynard-io-public/kubernetes-s3-mysql-backup
-            imagePullPolicy: Always
-            env:
-              - name: AWS_ACCESS_KEY_ID
-                value: "<Your Access Key>"
-              - name: AWS_SECRET_ACCESS_KEY
-                valueFrom:
-                   secretKeyRef:
-                     name: AWS_SECRET_ACCESS_KEY
-                     key: aws_secret_access_key
-              - name: AWS_DEFAULT_REGION
-                value: "<Your S3 Bucket Region>"
-              - name: AWS_BUCKET_NAME
-                value: "<Your S3 Bucket Name>"
-              - name: AWS_BUCKET_BACKUP_PATH
-                value: "<Your S3 Bucket Backup Path>"
-              - name: TARGET_DATABASE_HOST
-                value: "<Your Target Database Host>"
-              - name: TARGET_DATABASE_PORT
-                value: "<Your Target Database Port>"
-              - name: TARGET_DATABASE_NAMES
-                value: "<Your Target Database Name(s)>"
-              - name: TARGET_DATABASE_USER
-                value: "<Your Target Database Username>"
-              - name: TARGET_DATABASE_PASSWORD
-                valueFrom:
-                   secretKeyRef:
-                     name: TARGET_DATABASE_PASSWORD
-                     key: database_password
-              - name: NOTIFY_ENABLED
-                value: "<true/false>"
-              - name: NOTIFY_CHANNEL
-                value: "#chatops"
-              - name: NOTIFICATION_WEBHOOK_URL
-                valueFrom:
-                   secretKeyRef:
-                     name: NOTIFICATION_WEBHOOK_URL
-                     key: slack_webhook_url
-          restartPolicy: Never
-```

resources/perform-backup.sh → scripts/backup.sh

@@ -1,5 +1,15 @@
 #!/bin/bash
 
+echo "Starting database backup process at $(date +'%d-%m-%Y %H:%M:%S')."
+echo -e "Config: \n\tDatabase Host: $TARGET_DATABASE_HOST \n\tDatabase Port: $TARGET_DATABASE_PORT \n\tDatabase User: $TARGET_DATABASE_USER \n\tDatabase Names: $TARGET_DATABASE_NAMES \n\tAWS Bucket URI: $AWS_BUCKET_URI \n\tAWS Bucket Backup Path: $AWS_BUCKET_BACKUP_PATH"
+
+# Exit if any of the required environment variables are not set
+if [ -z "$TARGET_DATABASE_HOST" ] || [ -z "$TARGET_DATABASE_PORT" ] || [ -z "$TARGET_DATABASE_USER" ] || [ -z "$TARGET_DATABASE_PASSWORD" ] || [ -z "$TARGET_DATABASE_NAMES" ] || [ -z "$AWS_BUCKET_URI" ] || [ -z "$AWS_BUCKET_BACKUP_PATH" ]
+then
+    echo -e "ERROR: One or more required environment variables are not set. Exiting with status code 1."
+    exit 1
+fi
+
 # Set the has_failed variable to false. This will change if any of the subsequent database backups/uploads fail.
 has_failed=false
 backup_name=""
@@ -49,6 +59,12 @@ then
         /notify.sh "One or more backups on database host $TARGET_DATABASE_HOST failed. The error details are included below:" "$logcontents"
     fi
 
+    if [ "$HEALTHCHECK_ENABLED" = true ]
+    then
+        # Send healthcheck alert
+        /healthcheck.sh "fail"
+    fi
+
     echo -e "kubernetes-s3-mysql-backup encountered 1 or more errors. Exiting with status code 1."
     exit 1
 else
@@ -58,5 +74,13 @@ else
         /notify.sh "Backup created: $backup_tar_name"
     fi
 
+    if [ "$HEALTHCHECK_ENABLED" = true ]
+    then
+        # Send healthcheck alert
+        /healthcheck.sh
+    fi
+
+    echo -e "All database backups completed successfully. Exiting with status code 0."
+
     exit 0
 fi

scripts/healthcheck.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+if [ "$1" = "fail" ]; then
+    curl -m 10 --retry 5 -X POST -d "message=Backup failed for $TARGET_DATABASE_HOST" "$HEALTHCHECK_URL/fail"
+else
+    curl -m 10 --retry 5 -X POST -d "message=Backup succeeded for $TARGET_DATABASE_HOST" "$HEALTHCHECK_URL"
+fi