-
Notifications
You must be signed in to change notification settings - Fork 21
/
backup.html.md.erb
164 lines (118 loc) · 10.2 KB
/
backup.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
---
title: Back Up MySQL for Pivotal Cloud Foundry®
owner: MySQL
---
## <a id="automated-backups"></a>Automated Backups ##
In versions of MySQL for Pivotal Cloud Foundry® (PCF) 1.7.0.0 and higher, automated backups can be enabled in the "Configure Backups" section. Backups has the following features:
- Periodically create and upload backup artifacts suitable for restoring the complete set of database instances allocated in the service.
- No locks, no downtime. The only affect on the serving systems is the amount of I/O required to copy the database and log files off of the VM.
- Includes a metadata file which contains the critical details of the backup artifact, including the effective calendar time of the backup.
- Backup artifacts are encrypted within the MySQL for PCF cluster of VMs; unencrypted data is never transported outside of the MySQL for PCF deployment.
- Backups are currently stored on **Amazon S3** only. Other storage targets will be added over time.
Backups are enabled by default. If you do not wish for this deployment to be backed up, you must disable backups in three steps:
1. Select "Disable Backups" in the Configure Backup section.
1. Switch to "Resource Config," section, and change the number of instances for `Backup Prepare Node` from 1 to 0.
1. Return to the Installation Dashboard, and click `Apply Changes`.
### <a id="configuring-step-by-step"></a>Configuring Backups Step-by-Step ###
For each step, refer to the screenshot above.
1. Select "Enable Backups"
1. Enter the bucket name where backup artifacts will be uploaded. Using the S3 credentials provided, MySQL for PCF will automatically create the bucket if it does not exist.
1. Enter a folder name for this cluster's backups to be stored. Again, the folder will be automatically created if it does not exist. **It is important to use this folder exclusively for this cluster's backup artifacts.** Mixing the backup artifacts from different clusters within a single folder can cause confusion and possible inadvertent loss of backup artifacts.
1. Provide an AWS Access Key and Secret Access Key in the following fields. We recommend that you create an [IAM](https://aws.amazon.com/iam/) credential that only has access to this bucket.
1. Specify how often you'd like backups to occur in the final field, "Cron Schedule." The [syntax definition](http://godoc.org/github.com/robfig/cron) is similar to traditional cron, plus easy features such as, `@every 1d` to back up once per day.
<p class="note"><strong>Note</strong>: For large databases, the default storage for the `Backup Prepare Node` may not be sufficient. In order to compress and encrypt backup artifacts, the VM must be configured with twice the amount of ephemeral disk space as the persistent disk space of the `MySQL Server` nodes.</p>
### <a id="understanding-metadata"></a>Understanding Backup Metadata ###
Along with each release, MySQL for PCF will upload a `mysql-backup-XXXXXXXXXX.txt` file. The contents of that file will look something like this:
```
uuid = dfe9fcdd-7d0f-11e5-93b3-0695a7b9771f
name =
tool_name = innobackupex
tool_command = --user=root --password=... --stream=tar tmp/
tool_version = 1.5.1-xtrabackup
ibbackup_version = xtrabackup version 2.2.10 based on MySQL server 5.6.22 Linux (x86_64) (revision id: )
server_version = 10.0.21-MariaDB-wsrep
start_time = 2015-10-28 01:04:40
end_time = 2015-10-28 01:04:43
lock_time = 1
binlog_pos =
innodb_from_lsn = 0
innodb_to_lsn = 1730899
partial = N
incremental = N
format = tar
compact = N
compressed = N
encrypted = N
```
Within this file, the most important items are the `start_time` and the `server_version` entries. Transactions that have not been completed at the start of the backup effort will **not** be present in the restored artifact.
**Note**: Both `compressed` and `encrypted` show as `N` in this file, yet the artifact uploaded by MySQL for PCF is both. This is a known defect, and will be fixed in a future release.
### <a id="restoring"></a>Restoring a Backup Artifact ###
MySQL for PCF is normally HA, in that it keeps at least two complete copies of the data. In most cases, if a cluster is still able to connect to persistent storage, you can restore a cluster to health using the [bootstrap process](bootstrapping.html). Before resorting to a database restore, please check with support to be sure that your existing cluster is beyond help.
The DR backups feature of MySQL for PCF is primarily intended as a way to recover data to the same Pivotal Cloud Foundry® Foundation from which the data was backed up. This process replaces 100% of the data and state of a running MySQL for PCF cluster. This is especially relevant with regard to service instances and bindings. Due to the way in which service instances are defined it is not currently possible to restore a MySQL for PCF database to a different Foundation.
In the event of a total cluster loss,the process to restore a backup artifact to a MySQL for PCF cluster is entirely manual. Follow these steps to use the offsite backups to restore your cluster to its previous state.
- Discover the encryption keys in the `Credentials` tab of the MySQL for PCF tile.
1. If necessary, install the same version of the MySQL for PCF product in Ops Manager.
1. Reduce the size of the MySQL for PCF cluster to a single node
1. Select the MySQL for PCF tile > Settings > Resource Config > MySQL Server > Instances, and set to 1
1. Click "Save"
1. Click "Back to Installation Dashboard," then "Apply Changes"
1. Wait for deployment to succeed
1. Prepare the first node for restoration
1. [ssh](https://docs.pivotal.io/pivotalcf/customizing/trouble-advanced.html#ssh) onto the Ops Manager Director
1. From there, use BOSH to [ssh](https://docs.pivotal.io/pivotalcf/customizing/trouble-advanced.html#bosh-ssh) onto first mysql job
- IP address can be found in MySQL for PCF > Status > MySQL Server
- VM credentials can be found in MySQL for PCF > Credentials > MySQL Server > Vm Credentials
1. Become super user<br>
`sudo su`
1. Pause the local database server<br>
`monit stop all`
1. Confirm that all jobs are listed as 'not monitored'<br>
`watch monit summary`
1. Delete the existing MySQL data which is stored on disk<br>
`rm -rf /var/vcap/store/mysql/*`
1. Restore the backup
1. Move the compressed backup (named e.g. `mysql-backup.tar.bzip2`) to the node (e.g. via `scp`)
1. Decrypt the file using the `gpg` command<br>
`gpg --decrypt mysql-backup-1444362715.tar.gpg`
1. Expand the backup artifact into the data director of MySQL<br>
`tar xvjf mysql-backup.tar.bzip2 --directory=/var/vcap/store/mysql`
1. MySQL process expects data directory to be owned by a particular user<br>
`chown -R vcap:vcap /var/vcap/store/mysql`
1. `monit start all`
1. `watch monit summary` until all jobs are listed as 'running'
1. Exit out of the MySQL node
1. Increase the size of the cluster back to three
1. MySQL for PCF tile > Settings > Resource Config > MySQL Server > Instances to 3
1. Click "Save"
1. "Back to Installation Dashboard" > Apply Changes
1. Wait for deployment to succeed
---
## <a id="manual-process"></a>Manual Process ##
If you do not wish to use the automated facility included in MySQL for PCF, you can still [perform backups manually](#manual-backup). If you need to restore from a manual backup, follow the instructions in the [manual restore](#manual-restore) section.
### <a id="prerequisites"></a>Prerequisites #####
* Locate the IP address for the MySQL node in the **Status** tab.
<%= image_tag("./mysql-server-ip.png") %>
* Locate the root password for the MySQL server in the **Credentials** tab.
<%= image_tag("./mysql-root-password.png") %>
### <a id="manual-backup"></a>Manual Backup ###
This backup acquires a global read lock on all tables, but does not hold it for the entire duration of the dump.
<p class="note"><strong>Note</strong>: The procedure to manually backup MySQL below does not cover steps that are specific to a particular IaaS. It is important to tailor these instructions based on your infrastructure and configuration requirements. </p>
1. Spin up a virtual machine (VM) to use for your MySQL backup and restore. Ensure the VM has sufficient disk space to receive the data dump. Configure network access to your MySQL for PCF deployment.
1. Install the [mysqldump](https://mariadb.com/kb/en/mariadb/mysqldump/) client of your choice.
1. In a terminal window, execute the following command. To backup **all** databases in the MySQL deployment, use `--all-databases`:
<pre class="terminal">$ mysqldump -u root -p -h $MYSQL_NODE_IP --all-databases > user_databases.sql</pre>
To backup a single database, specify the database name:
<pre class="terminal">$ mysqldump -u root -p -h $MYSQL_NODE_IP $DB_NAME > user_databases.sql</pre>
<p class="note"><strong>Note</strong>: The backup data you obtain from running the command above is not encrypted nor compressed. Pivotal recommends encrypting the backup artifact before storing it off-site.</p>
1. Store the backup in a different location than the MySQL primary storage, preferably off-site.
### <a id="manual-restore"></a>Manual Restore ###
Restoring from a backup is the same whether one or multiple databases were backed up. Executing the SQL dump will drop, recreate, and refill the specified databases and tables.
<p class="note"><strong>Warning:</strong> Restoring a database will delete all data that existed in the database prior to the restore. Restoring a database using a full backup artifact, produced by `mysqldump --all-databases` for example, will replace all data and user permissions.</p>
1. From the same VM you used to perform the manual backup, or a similar VM, restore from the data dump:
<pre class="terminal">$ mysql -u root -p -h $MYSQL\_NODE\_IP < user_databases.sql</pre>
1. Re-apply user privileges:
<pre class="terminal">$ mysql -u root -p -h $MYSQL\_NODE\_IP -e "FLUSH PRIVILEGES"</pre>
This command tells the cluster to re-load user permissions using the data that has just been restored.
### <a id="examples"></a>Examples ###
More examples can be found in the [MariaDB documentation](http://mariadb.com/kb/en/mariadb/mysqldump/#examples).