Skip to content

Commit

Permalink
Merge pull request #312 from databacker/rename-config-fields
Browse files Browse the repository at this point in the history
configuration fields to camelCase
  • Loading branch information
deitch committed May 10, 2024
2 parents fda8c89 + 39e684d commit 738cc8f
Show file tree
Hide file tree
Showing 4 changed files with 49 additions and 48 deletions.
43 changes: 22 additions & 21 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ The environment variables, CLI flag options and config file options are similar,
due to variances in how the various are structured. As a general rule:

* Environment variables are all uppercase, with words separated by underscores, and most start with `DB_DUMP`. For example, `DB_DUMP_FREQ=60`.
* CLI flags are all lowercase, with words separated by hyphens. Since the CLI has sub-commands, the `dump-` and `restore-` are unnecessary. For example, `mysql-backup dump --frequency=60` or `mysql-backup restore --target=/foo/file.gz`.
* CLI flags are all lowercase, with words separated by hyphens, a.k.a. kebab-case. Since the CLI has sub-commands, the `dump-` and `restore-` are unnecessary. For example, `mysql-backup dump --frequency=60` or `mysql-backup restore --target=/foo/file.gz`.
* Config file keys are camelCase, for example, `dump.maxAllowedPacket=6000`.

For example, the following are equivalent.

Expand Down Expand Up @@ -69,7 +70,7 @@ The following are the environment variables, CLI flags and configuration file op
| password for the database | BR | `pass` | `DB_PASS` | `database.credentials.password` | |
| names of databases to dump, comma-separated | B | `include` | `DB_NAMES` | `dump.include` | all databases in the server |
| names of databases to exclude from the dump | B | `exclude` | `DB_NAMES_EXCLUDE` | `dump.exclude` | |
| do not include `USE <database>;` statement in the dump | B | `no-database-name` | `NO_DATABASE_NAME` | `dump.no-database-name` | `false` |
| do not include `USE <database>;` statement in the dump | B | `no-database-name` | `NO_DATABASE_NAME` | `dump.noDatabaseName` | `false` |
| restore to a specific database | R | `restore --database` | `RESTORE_DATABASE` | `restore.database` | |
| how often to do a dump or prune, in minutes | BP | `dump --frequency` | `DB_DUMP_FREQ` | `dump.schedule.frequency` | `1440` (in minutes), i.e. once per day |
| what time to do the first dump or prune | BP | `dump --begin` | `DB_DUMP_BEGIN` | `dump.schedule.begin` | `0`, i.e. immediately |
Expand All @@ -79,19 +80,19 @@ The following are the environment variables, CLI flags and configuration file op
| where to put the dump file; see [backup](./backup.md) | BP | `dump --target` | `DB_DUMP_TARGET` | `dump.targets` | |
| where the restore file exists; see [restore](./restore.md) | R | `restore --target` | `DB_RESTORE_TARGET` | `restore.target` | |
| replace any `:` in the dump filename with `-` | BP | `dump --safechars` | `DB_DUMP_SAFECHARS` | `database.safechars` | `false` |
| AWS access key ID, used only if a target does not have one | BRP | `aws-access-key-id` | `AWS_ACCESS_KEY_ID` | `dump.targets[s3-target].credentials.access-key-id` | |
| AWS secret access key, used only if a target does not have one | BRP | `aws-secret-access-key` | `AWS_SECRET_ACCESS_KEY` | `dump.targets[s3-target].credentials.secret-access-key` | |
| AWS access key ID, used only if a target does not have one | BRP | `aws-access-key-id` | `AWS_ACCESS_KEY_ID` | `dump.targets[s3-target].accessKeyId` | |
| AWS secret access key, used only if a target does not have one | BRP | `aws-secret-access-key` | `AWS_SECRET_ACCESS_KEY` | `dump.targets[s3-target].secretAccessKey` | |
| AWS default region, used only if a target does not have one | BRP | `aws-region` | `AWS_REGION` | `dump.targets[s3-target].region` | |
| alternative endpoint URL for S3-interoperable systems, used only if a target does not have one | BR | `aws-endpoint-url` | `AWS_ENDPOINT_URL` | `dump.targets[s3-target].endpoint` | |
| SMB username, used only if a target does not have one | BRP | `smb-user` | `SMB_USER` | `dump.targets[smb-target].credentials.username` | |
| SMB password, used only if a target does not have one | BRP | `smb-pass` | `SMB_PASS` | `dump.targets[smb-target].credentials.password` | |
| SMB username, used only if a target does not have one | BRP | `smb-user` | `SMB_USER` | `dump.targets[smb-target].username` | |
| SMB password, used only if a target does not have one | BRP | `smb-pass` | `SMB_PASS` | `dump.targets[smb-target].password` | |
| compression to use, one of: `bzip2`, `gzip` | BP | `compression` | `DB_DUMP_COMPRESSION` | `dump.compression` | `gzip` |
| when in container, run the dump or restore with `nice`/`ionice` | BR | `` | `NICE` | `` | `false` |
| filename to save the target backup file | B | `dump --filename-pattern` | `DB_DUMP_FILENAME_PATTERN` | `dump.filename-pattern` | |
| directory with scripts to execute before backup | B | `dump --pre-backup-scripts` | `DB_DUMP_PRE_BACKUP_SCRIPTS` | `dump.scripts.pre-backup` | in container, `/scripts.d/pre-backup/` |
| directory with scripts to execute after backup | B | `dump --post-backup-scripts` | `DB_DUMP_POST_BACKUP_SCRIPTS` | `dump.scripts.post-backup` | in container, `/scripts.d/post-backup/` |
| directory with scripts to execute before restore | R | `restore --pre-restore-scripts` | `DB_DUMP_PRE_RESTORE_SCRIPTS` | `restore.pre-restore-scripts` | in container, `/scripts.d/pre-restore/` |
| directory with scripts to execute after restore | R | `restore --post-restore-scripts` | `DB_DUMP_POST_RESTORE_SCRIPTS` | `restore.post-restore-scripts` | in container, `/scripts.d/post-restore/` |
| filename to save the target backup file | B | `dump --filename-pattern` | `DB_DUMP_FILENAME_PATTERN` | `dump.filenamePattern` | |
| directory with scripts to execute before backup | B | `dump --pre-backup-scripts` | `DB_DUMP_PRE_BACKUP_SCRIPTS` | `dump.scripts.preBackup` | in container, `/scripts.d/pre-backup/` |
| directory with scripts to execute after backup | B | `dump --post-backup-scripts` | `DB_DUMP_POST_BACKUP_SCRIPTS` | `dump.scripts.postBackup` | in container, `/scripts.d/post-backup/` |
| directory with scripts to execute before restore | R | `restore --pre-restore-scripts` | `DB_DUMP_PRE_RESTORE_SCRIPTS` | `restore.scripts.preRestore` | in container, `/scripts.d/pre-restore/` |
| directory with scripts to execute after restore | R | `restore --post-restore-scripts` | `DB_DUMP_POST_RESTORE_SCRIPTS` | `restore.scripts.postRestore` | in container, `/scripts.d/post-restore/` |
| retention policy for backups | BP | `dump --retention` | `RETENTION` | `prune.retention` | Infinite |

## Configuration File
Expand Down Expand Up @@ -125,24 +126,24 @@ for details of each.
* `include`: list of tables to include
* `exclude`: list of tables to exclude
* `safechars`: safe characters in filename
* `no-database-name`: remove `USE <database>` from dumpfile
* `noDatabaseName`: remove `USE <database>` from dumpfile
* `schedule`: the schedule configuration
* `frequency`: the frequency of the schedule
* `begin`: the time to begin the schedule
* `cron`: the cron schedule
* `once`: run once and exit
* `compression`: the compression to use
* `compact`: compact the dump
* `max-allowed-packet`: max packet size
* `filename-pattern`: the filename pattern
* `maxAllowedPacket`: max packet size
* `filenamePattern`: the filename pattern
* `scripts`:
* `pre-backup`: path to directory with pre-backup scripts
* `post-backup`: path to directory with post-backup scripts
* `preBackup`: path to directory with pre-backup scripts
* `postBackup`: path to directory with post-backup scripts
* `targets`: list of names of known targets, defined in the `targets` section, where to save the backup
* `restore`: the restore configuration
* `scripts`:
* `pre-restore`: path to directory with pre-restore scripts
* `post-restore`: path to directory with post-restore scripts
* `preRestore`: path to directory with pre-restore scripts
* `postRestore`: path to directory with post-restore scripts
* `database`: the database configuration
* `server`: host:port
* `port`: port (deprecated)
Expand All @@ -154,12 +155,12 @@ for details of each.
* `targets`: target configurations, each of which can be reference by other sections. Key is the name of the target that is referenced elsewhere. Each one has the following structure:
* `type`: the type of target, one of: file, s3, smb
* `url`: the URL of the target
* `details`: access details for the target, depends on target type:
* `spec`: access details for the target, depends on target type:
* Type s3:
* `region`: the region
* `endpoint`: the endpoint
* `access-key-id`: the access key ID (s3)
* `secret-access-key`: the secret access key (s3)
* `accessKeyId`: the access key ID (s3)
* `secretAccessKey`: the secret access key (s3)
* Type smb:
* `domain`: the domain (smb)
* `username`: the username (smb)
Expand Down
20 changes: 10 additions & 10 deletions pkg/config/local.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,12 @@ type Dump struct {
Include []string `yaml:"include"`
Exclude []string `yaml:"exclude"`
Safechars bool `yaml:"safechars"`
NoDatabaseName bool `yaml:"no-database-name"`
NoDatabaseName bool `yaml:"noDatabaseName"`
Schedule Schedule `yaml:"schedule"`
Compression string `yaml:"compression"`
Compact bool `yaml:"compact"`
MaxAllowedPacket int `yaml:"max-allowed-packet"`
FilenamePattern string `yaml:"filename-pattern"`
MaxAllowedPacket int `yaml:"maxAllowedPacket"`
FilenamePattern string `yaml:"filenamePattern"`
Scripts BackupScripts `yaml:"scripts"`
Targets []string `yaml:"targets"`
}
Expand All @@ -48,17 +48,17 @@ type Schedule struct {
}

type BackupScripts struct {
PreBackup string `yaml:"pre-backup"`
PostBackup string `yaml:"post-backup"`
PreBackup string `yaml:"preBackup"`
PostBackup string `yaml:"postBackup"`
}

type Restore struct {
Scripts RestoreScripts `yaml:"scripts"`
}

type RestoreScripts struct {
PreRestore string `yaml:"pre-restore"`
PostRestore string `yaml:"post-restore"`
PreRestore string `yaml:"preRestore"`
PostRestore string `yaml:"postRestore"`
}

type Database struct {
Expand All @@ -76,7 +76,7 @@ type Telemetry struct {
remote.Connection
// BufferSize is the size of the buffer for telemetry messages. It keeps BufferSize messages
// in memory before sending them remotely. The default of 0 is the same as 1, i.e. send every message.
BufferSize int `yaml:"buffer-size"`
BufferSize int `yaml:"bufferSize"`
}

var _ yaml.Unmarshaler = &Target{}
Expand Down Expand Up @@ -159,8 +159,8 @@ func (s S3Target) Storage() (storage.Storage, error) {
}

type AWSCredentials struct {
AccessKeyId string `yaml:"access-key-id"`
SecretAccessKey string `yaml:"secret-access-key"`
AccessKeyId string `yaml:"accessKeyId"`
SecretAccessKey string `yaml:"secretAccessKey"`
}

type SMBTarget struct {
Expand Down
18 changes: 9 additions & 9 deletions sample-configs/local.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ spec:
- table3
- table4
safechars: true # defaults to false
no-database-name: false # remove the `USE <database>` statement from backup files, defaults to false
noDatabaseName: false # remove the `USE <database>` statement from backup files, defaults to false
# schedule to dump, can use one of: cron, frequency, once. If frequency is set, begin will be checked
schedule:
once: true # run only once and exit; ignores all other scheduling. Defaults to false
Expand All @@ -28,11 +28,11 @@ spec:
begin: +25 # What time to do the first dump. Must be in one of two formats: Absolute: HHMM, e.g. `2330` or `0415`; or Relative: +MM, i.e. how many minutes after starting the container, e.g. `+0` (immediate), `+10` (in 10 minutes), or `+90` in an hour and a half"
compression: gzip # defaults to gzip
compact: true # defaults to false
max-allowed-packet: 4194304 # defaults to 4194304
filename-pattern: db_backup_{{ .now }}.{{ .compression }}
maxAllowedPacket: 4194304 # defaults to 4194304
filenamePattern: db_backup_{{ .now }}.{{ .compression }}
scripts:
pre-backup: /path/to/prescripts/
post-backup: /path/to/postscripts/
preBackup: /path/to/prescripts/
postBackup: /path/to/postscripts/
# list of named targets to backup to, from the index below
targets:
- s3
Expand All @@ -42,8 +42,8 @@ spec:

restore:
scripts:
pre-restore: /path/to/prescripts/
post-restore: /path/to/postscripts/
preRestore: /path/to/prescripts/
postRestore: /path/to/postscripts/

# database configuration
database:
Expand All @@ -61,8 +61,8 @@ spec:
details:
region: us-west-1
endpoint: https://s3.us-west-1.amazonaws.com
access-key-id: access_key_id
secret-access-key: secret_access_key
accessKeyId: access_key_id
secretAccessKey: secret_access_key
file:
type: file
url: file:///tmp/databackup
Expand Down
16 changes: 8 additions & 8 deletions sample-configs/telemetry.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,18 +35,18 @@ spec:
- table3
- table4
safechars: true # defaults to false
no-database-name: false # remove the `USE <database>` statement from backup files, defaults to false
noDatabaseName: false # remove the `USE <database>` statement from backup files, defaults to false
# schedule to dump, can use one of: cron, frequency, once. If frequency is set, begin will be checked
schedule:
once: true # run only once and exit; ignores all other scheduling. Defaults to false
cron: "0 10 * * *"
frequency: 1440 # in minutes
begin: 25 # minutes from initialization
compression: gzip # defaults to gzip
filename-pattern: db_backup_{{ .now }}.{{ .compression }}
filenamePattern: db_backup_{{ .now }}.{{ .compression }}
scripts:
pre-backup: /path/to/prescripts/
post-backup: /path/to/postscripts/
preBackup: /path/to/prescripts/
postBackup: /path/to/postscripts/
# list of named targets to backup to, from the index below
targets:
- s3
Expand All @@ -56,8 +56,8 @@ spec:

restore:
scripts:
pre-restore: /path/to/prescripts/
post-restore: /path/to/postscripts/
preRestore: /path/to/prescripts/
postRestore: /path/to/postscripts/

# database configuration
database:
Expand All @@ -75,8 +75,8 @@ spec:
region: us-west-1
endpoint: https://s3.us-west-1.amazonaws.com
credentials:
access-key-id: access_key_id
secret-access-key: secret_access_key
accessKeyId: access_key_id
secretAccessKey: secret_access_key
file:
type: file
url: file:///tmp/databackup
Expand Down

0 comments on commit 738cc8f

Please sign in to comment.