Releases: vitessio/vitess
Vitess v16.0.4
Release of Vitess v16.0.4
The entire changelog for this release can be found here.
The release includes 11 merged Pull Requests.
Thanks to all our contributors: @GuptaManan100, @app/vitess-bot, @dbussink, @rohit-nayak-ps, @shlomi-noach, @systay
Vitess v17.0.1
Release of Vitess v17.0.1
The entire changelog for this release can be found here.
The release includes 23 merged Pull Requests.
Thanks to all our contributors: @GuptaManan100, @app/vitess-bot, @frouioui, @mattlord, @shlomi-noach
Vitess v16.0.3
Release of Vitess v16.0.3
The entire changelog for this release can be found here.
The release includes 38 merged Pull Requests.
Thanks to all our contributors: @GuptaManan100, @app/github-actions, @app/vitess-bot, @frouioui, @harshit-gangal, @shlomi-noach, @systay
Vitess v15.0.4
Release of Vitess v15.0.4
The entire changelog for this release can be found here.
The release includes 33 merged Pull Requests.
Thanks to all our contributors: @GuptaManan100, @app/vitess-bot, @frouioui, @harshit-gangal, @shlomi-noach, @systay
Vitess v17.0.0
Release of Vitess v17.0.0
Summary
Table of Contents
- Major Changes
- Breaking Changes
- VTTablet: Initializing all replicas with super_read_only
- Default Local Cell Preference for TabletPicker
- Dedicated stats for VTGate Prepare operations
- VTAdmin web migrated from create-react-app to vite
- Keyspace name validation in TopoServer
- Shard name validation in TopoServer
- Compression CLI flags removed from vtctld and vtctldclient binaries
- VtctldClient command RestoreFromBackup will now use the correct context
- VTTablet Restore Metrics
- New command line flags and behavior
- New stats
- Online DDL
- VReplication
- VTTablet
- VTGate
- Deprecations and Deletions
- Breaking Changes
Major Changes
Breaking Changes
Default Local Cell Preference for TabletPicker
We added options to the TabletPicker
that allow for specifying a cell preference in addition to making the default behavior to give priority to the local cell and any alias it belongs to. We are also introducing a new way to select tablet type preference which should eventually replace the in_order:
hint currently used as a prefix for tablet types. The signature for creating a new TabletPicker
now looks like:
func NewTabletPicker(
ctx context.Context,
ts *topo.Server,
cells []string,
localCell, keyspace, shard, tabletTypesStr string,
options TabletPickerOptions,
) (*TabletPicker, error) {...}
Where ctx, localCell, option are all new parameters.
option
is of type TabletPickerOptions
and includes two fields, CellPreference
and TabletOrder
.
CellPreference
: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the clientTabletOrder
: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client
See PR 12282 Description for examples on how this changes cell picking behavior.
Default TLS version changed for vtgr
When using TLS with vtgr
, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.
vtgr
is now deprecated as part of v17.0.0
, please see the deprecation notice.
Dedicated stats for VTGate Prepare operations
Prior to v17 Vitess incorrectly combined stats for VTGate Execute
and Prepare
operations under a single stats key (Execute
). In v17 Execute
and Prepare
operations generate stats under independent stats keys.
Here is a (condensed) example of stats output:
{
"VtgateApi": {
"Histograms": {
"Execute.src.primary": {
"500000": 5
},
"Prepare.src.primary": {
"100000000": 0
}
}
},
"VtgateApiErrorCounts": {
"Execute.src.primary.INVALID_ARGUMENT": 3,
"Execute.src.primary.ALREADY_EXISTS": 1
}
}
VTAdmin web migrated to vite
Previously, VTAdmin web used the Create React App framework to test, build, and serve the application. In v17, Create React App has been removed, and Vite is used in its place. Some of the main changes include:
- Vite uses
VITE_*
environment variables instead ofREACT_APP_*
environment variables - Vite uses
import.meta.env
in place ofprocess.env
- Vitest is used in place of Jest for testing
- Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
public/index.html
has been moved to root directory in web/vtadmin
Keyspace name validation in TopoServer
Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.
Keyspace names are restricted to using only ASCII characters, digits and _
and -
. TopoServer's GetKeyspace
and CreateKeyspace
methods return an error if given an invalid name.
Shard name validation in TopoServer
Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.
Shard names are restricted to using only ASCII characters, digits and _
and -
. TopoServer's GetShard
and CreateShard
methods return an error if given an invalid name.
Compression CLI flags remove from vtctld and vtctldclient binaries
The CLI flags below were mistakenly added to vtctld
and vtctldclient
in v15. In v17, they are no longer present in those binaries.
--compression-engine-name
--compression-level
--external-compressor
--external-compressor-extension
--external-decompressor
VtctldClient command RestoreFromBackup will now use the correct context
The VtctldClient command RestoreFromBackup
initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp.
Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 PR#12830,
this behavior was changed to use a context with a timeout of action_timeout
. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough
time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes.
VTTablet Restore Metrics
As part of the VTTablet Sidecar Schema Maintenance Refactor in v16.0.0, we dropped the local_metadata
table from the sidecar database schema. This table was storing a couple of metrics related to restores from backup.
They have now been re-introduced as metrics that can be accessed from /debug/vars
.
Vttablet's transaction throttler now also throttles DML outside of BEGIN; ...; COMMIT;
blocks
Prior to v17, vttablet
's transaction throttler (enabled with --enable-tx-throttler
) would only throttle requests done inside an explicit transaction, i.e., a BEGIN; ...; COMMIT;
block.
In v17 PR#13040, this behavior was being changed so that it also throttles work outside of explicit transactions for INSERT/UPDATE/DELETE/LOAD
queries.
New command line flags and behavior
Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size
Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes
.
In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size
and --builtinbackup-file-write-buffer-size
.
--builtinbackup-file-read-buffer-size
: read files using an IO buffer of this many bytes. Golang defaults are used when set to0
.--builtinbackup-file-write-buffer-size
: write files using an IO buffer of this many bytes. Golang defaults are used when set to0
. (default2097152
)
These flags are applicable to the following programs:
vtbackup
vtctld
vttablet
vttestserver
Manifest backup external decompressor command
Add a new builtin/xtrabackup flag --manifest-external-decompressor
. When set the value of that flag is stored in the manifest field ExternalDecompressor
. This manifest field may be consulted when decompressing a backup that was c...
Vitess v17.0.0-rc2
Release of Vitess v17.0.0-rc2
Summary
Table of Contents
- Major Changes
- Breaking Changes
- Default Local Cell Preference for TabletPicker
- Dedicated stats for VTGate Prepare operations
- VTAdmin web migrated from create-react-app to vite
- Keyspace name validation in TopoServer
- Shard name validation in TopoServer
- Compression CLI flags removed from vtctld and vtctldclient binaries
- VtctldClient command RestoreFromBackup will now use the correct context
- New command line flags and behavior
- New stats
- Online DDL
- VReplication
- VTTablet
- VTGate
- Deprecations and Deletions
- Breaking Changes
Major Changes
Breaking Changes
Default Local Cell Preference for TabletPicker
We added options to the TabletPicker
that allow for specifying a cell preference in addition to making the default behavior to give priority to the local cell and any alias it belongs to. We are also introducing a new way to select tablet type preference which should eventually replace the in_order:
hint currently used as a prefix for tablet types. The signature for creating a new TabletPicker
now looks like:
func NewTabletPicker(
ctx context.Context,
ts *topo.Server,
cells []string,
localCell, keyspace, shard, tabletTypesStr string,
options TabletPickerOptions,
) (*TabletPicker, error) {...}
Where ctx, localCell, option are all new parameters.
option
is of type TabletPickerOptions
and includes two fields, CellPreference
and TabletOrder
.
CellPreference: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the client
TabletOrder`: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client
See PR 12282 Description for examples on how this changes cell picking behavior.
Default TLS version changed for vtgr
When using TLS with vtgr
, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.
Dedicated stats for VTGate Prepare operations
Prior to v17 Vitess incorrectly combined stats for VTGate Execute and Prepare operations under a single stats key (Execute
). In v17 Execute and Prepare operations generate stats under independent stats keys.
Here is a (condensed) example of stats output:
{
"VtgateApi": {
"Histograms": {
"Execute.src.primary": {
"500000": 5
},
"Prepare.src.primary": {
"100000000": 0
}
}
},
"VtgateApiErrorCounts": {
"Execute.src.primary.INVALID_ARGUMENT": 3,
"Execute.src.primary.ALREADY_EXISTS": 1
}
}
VTAdmin web migrated to vite
Previously, VTAdmin web used the Create React App framework to test, build, and serve the application. In v17, Create React App has been removed, and Vite is used in its place. Some of the main changes include:
- Vite uses
VITE_*
environment variables instead ofREACT_APP_*
environment variables - Vite uses
import.meta.env
in place ofprocess.env
- Vitest is used in place of Jest for testing
- Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
public/index.html
has been moved to root directory in web/vtadmin
Keyspace name validation in TopoServer
Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.
Keyspace names are restricted to using only ASCII characters, digits and _
and -
. TopoServer's GetKeyspace
and CreateKeyspace
methods return an error if given an invalid name.
Shard name validation in TopoServer
Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.
Shard names are restricted to using only ASCII characters, digits and _
and -
. TopoServer's GetShard
and CreateShard
methods return an error if given an invalid name.
Compression CLI flags remove from vtctld and vtctldclient binaries
The CLI flags below were mistakenly added to vtctld
and vtctldclient
in v15. In v17, they are no longer present in those binaries.
--compression-engine-name
--compression-level
--external-compressor
--external-compressor-extension
--external-decompressor
VtctldClient command RestoreFromBackup will now use the correct context
The VtctldClient command RestoreFromBackup initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp.
Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 PR#12830,
this behavior was changed to use a context with a timeout of action_timeout
. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough
time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes.
Vttablet's transaction throttler now also throttles DML outside of BEGIN; ...; COMMIT;
blocks
Prior to v17, vttablet
's transaction throttler (enabled with --enable-tx-throttler
) would only throttle requests done inside an explicit transaction, i.e., a BEGIN; ...; COMMIT;
block.
In v17 PR#13040, this behavior was being changed so that it also throttles work outside of explicit transactions for INSERT/UPDATE/DELETE/LOAD
queries.
New command line flags and behavior
Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size
Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes.
In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size
and --builtinbackup-file-write-buffer-size
.
--builtinbackup-file-read-buffer-size
: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.--builtinbackup-file-write-buffer-size
: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)
These flags are applicable to the following programs:
vtbackup
vtctld
vttablet
vttestserver
Manifest backup external decompressor command
Add a new builtin/xtrabackup flag --manifest-external-decompressor
. When set the value of that flag is stored in the manifest field ExternalDecompressor
. This manifest field may be consulted when decompressing a backup that was compressed with an external command.
This feature enables the following flow:
- Take a backup using an external compressor
Backup --compression-engine=external \ --external-compressor=zstd \ --manifest-external-decompressor="zstd -d"
- Restore that backup with a mere
Restore
command, without having to specify--external-decompressor
.
vttablet --throttler-config-via-topo
This flag was introduced in v16 and defaulted to false
. In v17 it defaults to true
, and there is no need to supply it.
Note that this flag overrides --enable-lag-throttler
and --throttle-threshold
, ...
Vitess v17.0.0-rc1
Release of Vitess v17.0.0-rc1
Summary
Table of Contents
- Major Changes
- Breaking Changes
- Default Local Cell Preference for TabletPicker
- Dedicated stats for VTGate Prepare operations
- VTAdmin web migrated from create-react-app to vite
- Keyspace name validation in TopoServer
- Shard name validation in TopoServer
- Compression CLI flags removed from vtctld and vtctldclient binaries
- VtctldClient command RestoreFromBackup will now use the correct context
- New command line flags and behavior
- New stats
- Online DDL
- VReplication
- VTTablet
- VTGate
- Deprecations and Deletions
- Breaking Changes
Major Changes
Breaking Changes
Default Local Cell Preference for TabletPicker
We added options to the TabletPicker
that allow for specifying a cell preference in addition to making the default behavior to give priority to the local cell and any alias it belongs to. We are also introducing a new way to select tablet type preference which should eventually replace the in_order:
hint currently used as a prefix for tablet types. The signature for creating a new TabletPicker
now looks like:
func NewTabletPicker(
ctx context.Context,
ts *topo.Server,
cells []string,
localCell, keyspace, shard, tabletTypesStr string,
options TabletPickerOptions,
) (*TabletPicker, error) {...}
Where ctx, localCell, option are all new parameters.
option
is of type TabletPickerOptions
and includes two fields, CellPreference
and TabletOrder
.
CellPreference: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the client
TabletOrder`: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client
See PR 12282 Description for examples on how this changes cell picking behavior.
Default TLS version changed for vtgr
When using TLS with vtgr
, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.
Dedicated stats for VTGate Prepare operations
Prior to v17 Vitess incorrectly combined stats for VTGate Execute and Prepare operations under a single stats key (Execute
). In v17 Execute and Prepare operations generate stats under independent stats keys.
Here is a (condensed) example of stats output:
{
"VtgateApi": {
"Histograms": {
"Execute.src.primary": {
"500000": 5
},
"Prepare.src.primary": {
"100000000": 0
}
}
},
"VtgateApiErrorCounts": {
"Execute.src.primary.INVALID_ARGUMENT": 3,
"Execute.src.primary.ALREADY_EXISTS": 1
}
}
VTAdmin web migrated to vite
Previously, VTAdmin web used the Create React App framework to test, build, and serve the application. In v17, Create React App has been removed, and Vite is used in its place. Some of the main changes include:
- Vite uses
VITE_*
environment variables instead ofREACT_APP_*
environment variables - Vite uses
import.meta.env
in place ofprocess.env
- Vitest is used in place of Jest for testing
- Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
public/index.html
has been moved to root directory in web/vtadmin
Keyspace name validation in TopoServer
Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.
Keyspace names are restricted to using only ASCII characters, digits and _
and -
. TopoServer's GetKeyspace
and CreateKeyspace
methods return an error if given an invalid name.
Shard name validation in TopoServer
Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.
Shard names are restricted to using only ASCII characters, digits and _
and -
. TopoServer's GetShard
and CreateShard
methods return an error if given an invalid name.
Compression CLI flags remove from vtctld and vtctldclient binaries
The CLI flags below were mistakenly added to vtctld
and vtctldclient
in v15. In v17, they are no longer present in those binaries.
--compression-engine-name
--compression-level
--external-compressor
--external-compressor-extension
--external-decompressor
VtctldClient command RestoreFromBackup will now use the correct context
The VtctldClient command RestoreFromBackup initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp.
Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 PR#12830,
this behavior was changed to use a context with a timeout of action_timeout
. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough
time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes.
Vttablet's transaction throttler now also throttles DML outside of BEGIN; ...; COMMIT;
blocks
Prior to v17, vttablet
's transaction throttler (enabled with --enable-tx-throttler
) would only throttle requests done inside an explicit transaction, i.e., a BEGIN; ...; COMMIT;
block.
In v17 PR#13040, this behavior was being changed so that it also throttles work outside of explicit transactions for INSERT/UPDATE/DELETE/LOAD
queries.
New command line flags and behavior
Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size
Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes.
In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size
and --builtinbackup-file-write-buffer-size
.
--builtinbackup-file-read-buffer-size
: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.--builtinbackup-file-write-buffer-size
: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)
These flags are applicable to the following programs:
vtbackup
vtctld
vttablet
vttestserver
Manifest backup external decompressor command
Add a new builtin/xtrabackup flag --manifest-external-decompressor
. When set the value of that flag is stored in the manifest field ExternalDecompressor
. This manifest field may be consulted when decompressing a backup that was compressed with an external command.
This feature enables the following flow:
- Take a backup using an external compressor
Backup --compression-engine=external \ --external-compressor=zstd \ --manifest-external-decompressor="zstd -d"
- Restore that backup with a mere
Restore
command, without having to specify--external-decompressor
.
vttablet --throttler-config-via-topo
This flag was introduced in v16 and defaulted to false
. In v17 it defaults to true
, and there is no need to supply it.
Note that this flag overrides --enable-lag-throttler
and --throttle-threshold
, which now give warnings, and will be removed in v18.
New stats
####...
Vitess v16.0.2
Release of Vitess v16.0.2
Known Issues
Schema-initialization stuck on semi-sync ACKs while upgrading to v16.0.2
During upgrades from <= v15.x.x
to v16.0.2
, as part of PromoteReplica
call, the schema-init realizes that there are schema diffs to apply and ends up writing to the database if semi-sync is enabled, all of these writes get blocked indefinitely.
Eventually, PromoteReplica
fails, and this fails the entire PRS call.
A fix for this issue was merged on release-16.0
in PR#13441, read the corresponding bug report to learn more.
This issue is fixed in v16.0.3
and later patch releases.
Broken downgrade from v17.x.x when super_read_only turned on by default
In v17.x.x
super_read_only
is turned on by default meaning that downgrading from v17
to v16.0.2
breaks due to init_db.sql
needing write access.
This issue is fixed in >= v16.0.3
thanks to PR #13525
Major Changes
Upgrade to go1.20.3
Vitess v16.0.2
now runs on go1.20.3
.
Below is a summary of this Go patch release. You can learn more here.
go1.20.3 (released 2023-04-04) includes security fixes to the go/parser, html/template, mime/multipart, net/http, and net/textproto packages, as well as bug fixes to the compiler, the linker, the runtime, and the time package. See the Go 1.20.3 milestone on our issue tracker for details.
EffectiveCallerId in Vtgate gRPC calls
A new flag grpc-use-static-authentication-callerid
is added to gate the behavior introduced in #12050.
Earlier, we used to automatically set immediateCallerID to user from static authentication context that overrode the EffectiveCallerId.
Shard name validation in TopoServer
Prior to v16.0.2, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.
Shard names may no longer contain the forward slash ("/") character, and TopoServer's CreateShard
method returns an error if given such a name.
The entire changelog for this release can be found here.
The release includes 24 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @ajm188, @frouioui, @github-actions[bot], @harshit-gangal, @mattlord, @systay, @vitess-bot[bot]
Vitess v16.0.1
Release of Vitess v16.0.1
Known Issues
Schema-initialization stuck on semi-sync ACKs while upgrading to v16.0.1
During upgrades from <= v15.x.x
to v16.0.1
, as part of PromoteReplica
call, the schema-init realizes that there are schema diffs to apply and ends up writing to the database if semi-sync is enabled, all of these writes get blocked indefinitely.
Eventually, PromoteReplica
fails, and this fails the entire PRS call.
A fix for this issue was merged on release-16.0
in PR#13441, read the corresponding bug report to learn more.
This issue is fixed in v16.0.3
and later patch releases.
Broken downgrade from v17.x.x when super_read_only turned on by default
In v17.x.x
super_read_only
is turned on by default meaning that downgrading from v17
to v16.0.1
breaks due to init_db.sql
needing write access.
This issue is fixed in >= v16.0.3
thanks to PR #13525
Major Changes
Upgrade to go1.20.2
Vitess v16.0.1
now runs on go1.20.2
.
Below is a summary of this Go patch release. You can learn more here.
go1.20.2 (released 2023-03-07) includes a security fix to the crypto/elliptic package, as well as bug fixes to the compiler, the covdata command, the linker, the runtime, and the crypto/ecdh, crypto/rsa, crypto/x509, os, and syscall packages.
Keyspace name validation in TopoServer
Prior to v16.0.1, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.
Keyspace names may no longer contain the forward slash ("/") character, and TopoServer's GetKeyspace
and CreateKeyspace
methods return an error if given such a name.
The entire changelog for this release can be found here.
The release includes 39 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @ajm188, @frouioui, @github-actions[bot], @mattlord, @rohit-nayak-ps, @rsajwani, @shlomi-noach, @systay, @vitess-bot[bot]
Vitess v15.0.3
Release of Vitess v15.0.3
The entire changelog for this release can be found here.
The release includes 52 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @ajm188, @dbussink, @deepthi, @frouioui, @harshit-gangal, @mattlord, @rsajwani, @shlomi-noach, @systay, @vitess-bot[bot], @vmg