Skip to content

Releases: vitessio/vitess

Vitess v16.0.4

18 Aug 12:44
87ea735
Compare
Choose a tag to compare

Release of Vitess v16.0.4

The entire changelog for this release can be found here.

The release includes 11 merged Pull Requests.

Thanks to all our contributors: @GuptaManan100, @app/vitess-bot, @dbussink, @rohit-nayak-ps, @shlomi-noach, @systay

Vitess v17.0.1

28 Jul 13:49
a4e751f
Compare
Choose a tag to compare

Release of Vitess v17.0.1

The entire changelog for this release can be found here.

The release includes 23 merged Pull Requests.

Thanks to all our contributors: @GuptaManan100, @app/vitess-bot, @frouioui, @mattlord, @shlomi-noach

Vitess v16.0.3

28 Jul 13:33
a93cb55
Compare
Choose a tag to compare

Release of Vitess v16.0.3

The entire changelog for this release can be found here.

The release includes 38 merged Pull Requests.

Thanks to all our contributors: @GuptaManan100, @app/github-actions, @app/vitess-bot, @frouioui, @harshit-gangal, @shlomi-noach, @systay

Vitess v15.0.4

28 Jul 13:50
72d943e
Compare
Choose a tag to compare

Release of Vitess v15.0.4

The entire changelog for this release can be found here.

The release includes 33 merged Pull Requests.

Thanks to all our contributors: @GuptaManan100, @app/vitess-bot, @frouioui, @harshit-gangal, @shlomi-noach, @systay

Vitess v17.0.0

27 Jun 13:10
70a9466
Compare
Choose a tag to compare

Release of Vitess v17.0.0

Summary

Table of Contents

Major Changes

Breaking Changes

Default Local Cell Preference for TabletPicker

We added options to the TabletPicker that allow for specifying a cell preference in addition to making the default behavior to give priority to the local cell and any alias it belongs to. We are also introducing a new way to select tablet type preference which should eventually replace the in_order: hint currently used as a prefix for tablet types. The signature for creating a new TabletPicker now looks like:

func NewTabletPicker(
	ctx context.Context,
	ts *topo.Server,
	cells []string,
	localCell, keyspace, shard, tabletTypesStr string,
	options TabletPickerOptions,
) (*TabletPicker, error) {...}

Where ctx, localCell, option are all new parameters.

option is of type TabletPickerOptions and includes two fields, CellPreference and TabletOrder.

  • CellPreference: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the client
  • TabletOrder: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client

See PR 12282 Description for examples on how this changes cell picking behavior.

Default TLS version changed for vtgr

When using TLS with vtgr, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.

vtgr is now deprecated as part of v17.0.0, please see the deprecation notice.

Dedicated stats for VTGate Prepare operations

Prior to v17 Vitess incorrectly combined stats for VTGate Execute and Prepare operations under a single stats key (Execute). In v17 Execute and Prepare operations generate stats under independent stats keys.

Here is a (condensed) example of stats output:

{
  "VtgateApi": {
    "Histograms": {
      "Execute.src.primary": {
        "500000": 5
      },
      "Prepare.src.primary": {
        "100000000": 0
      }
    }
  },
  "VtgateApiErrorCounts": {
    "Execute.src.primary.INVALID_ARGUMENT": 3,
    "Execute.src.primary.ALREADY_EXISTS": 1
  }
}

VTAdmin web migrated to vite

Previously, VTAdmin web used the Create React App framework to test, build, and serve the application. In v17, Create React App has been removed, and Vite is used in its place. Some of the main changes include:

  • Vite uses VITE_* environment variables instead of REACT_APP_* environment variables
  • Vite uses import.meta.env in place of process.env
  • Vitest is used in place of Jest for testing
  • Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
  • public/index.html has been moved to root directory in web/vtadmin

Keyspace name validation in TopoServer

Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.

Keyspace names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetKeyspace and CreateKeyspace methods return an error if given an invalid name.

Shard name validation in TopoServer

Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.

Shard names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetShard and CreateShard methods return an error if given an invalid name.

Compression CLI flags remove from vtctld and vtctldclient binaries

The CLI flags below were mistakenly added to vtctld and vtctldclient in v15. In v17, they are no longer present in those binaries.

  • --compression-engine-name
  • --compression-level
  • --external-compressor
  • --external-compressor-extension
  • --external-decompressor

VtctldClient command RestoreFromBackup will now use the correct context

The VtctldClient command RestoreFromBackup initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp.
Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 PR#12830,
this behavior was changed to use a context with a timeout of action_timeout. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough
time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes.

VTTablet Restore Metrics

As part of the VTTablet Sidecar Schema Maintenance Refactor in v16.0.0, we dropped the local_metadata table from the sidecar database schema. This table was storing a couple of metrics related to restores from backup.
They have now been re-introduced as metrics that can be accessed from /debug/vars.

Vttablet's transaction throttler now also throttles DML outside of BEGIN; ...; COMMIT; blocks

Prior to v17, vttablet's transaction throttler (enabled with --enable-tx-throttler) would only throttle requests done inside an explicit transaction, i.e., a BEGIN; ...; COMMIT; block.
In v17 PR#13040, this behavior was being changed so that it also throttles work outside of explicit transactions for INSERT/UPDATE/DELETE/LOAD queries.

New command line flags and behavior

Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size

Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes.

In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size.

  • --builtinbackup-file-read-buffer-size: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.
  • --builtinbackup-file-write-buffer-size: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)

These flags are applicable to the following programs:

  • vtbackup
  • vtctld
  • vttablet
  • vttestserver

Manifest backup external decompressor command

Add a new builtin/xtrabackup flag --manifest-external-decompressor. When set the value of that flag is stored in the manifest field ExternalDecompressor. This manifest field may be consulted when decompressing a backup that was c...

Read more

Vitess v17.0.0-rc2

13 Jun 13:37
4de9709
Compare
Choose a tag to compare
Vitess v17.0.0-rc2 Pre-release
Pre-release

Release of Vitess v17.0.0-rc2

Summary

Table of Contents

Major Changes

Breaking Changes

Default Local Cell Preference for TabletPicker

We added options to the TabletPicker that allow for specifying a cell preference in addition to making the default behavior to give priority to the local cell and any alias it belongs to. We are also introducing a new way to select tablet type preference which should eventually replace the in_order: hint currently used as a prefix for tablet types. The signature for creating a new TabletPicker now looks like:

func NewTabletPicker(
	ctx context.Context,
	ts *topo.Server,
	cells []string,
	localCell, keyspace, shard, tabletTypesStr string,
	options TabletPickerOptions,
) (*TabletPicker, error) {...}

Where ctx, localCell, option are all new parameters.

option is of type TabletPickerOptions and includes two fields, CellPreference and TabletOrder.
CellPreference: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the client TabletOrder`: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client

See PR 12282 Description for examples on how this changes cell picking behavior.

Default TLS version changed for vtgr

When using TLS with vtgr, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.

Dedicated stats for VTGate Prepare operations

Prior to v17 Vitess incorrectly combined stats for VTGate Execute and Prepare operations under a single stats key (Execute). In v17 Execute and Prepare operations generate stats under independent stats keys.

Here is a (condensed) example of stats output:

{
  "VtgateApi": {
    "Histograms": {
      "Execute.src.primary": {
        "500000": 5
      },
      "Prepare.src.primary": {
        "100000000": 0
      }
    }
  },
  "VtgateApiErrorCounts": {
    "Execute.src.primary.INVALID_ARGUMENT": 3,
    "Execute.src.primary.ALREADY_EXISTS": 1
  }
}

VTAdmin web migrated to vite

Previously, VTAdmin web used the Create React App framework to test, build, and serve the application. In v17, Create React App has been removed, and Vite is used in its place. Some of the main changes include:

  • Vite uses VITE_* environment variables instead of REACT_APP_* environment variables
  • Vite uses import.meta.env in place of process.env
  • Vitest is used in place of Jest for testing
  • Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
  • public/index.html has been moved to root directory in web/vtadmin

Keyspace name validation in TopoServer

Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.

Keyspace names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetKeyspace and CreateKeyspace methods return an error if given an invalid name.

Shard name validation in TopoServer

Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.

Shard names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetShard and CreateShard methods return an error if given an invalid name.

Compression CLI flags remove from vtctld and vtctldclient binaries

The CLI flags below were mistakenly added to vtctld and vtctldclient in v15. In v17, they are no longer present in those binaries.

  • --compression-engine-name
  • --compression-level
  • --external-compressor
  • --external-compressor-extension
  • --external-decompressor

VtctldClient command RestoreFromBackup will now use the correct context

The VtctldClient command RestoreFromBackup initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp.
Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 PR#12830,
this behavior was changed to use a context with a timeout of action_timeout. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough
time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes.

Vttablet's transaction throttler now also throttles DML outside of BEGIN; ...; COMMIT; blocks

Prior to v17, vttablet's transaction throttler (enabled with --enable-tx-throttler) would only throttle requests done inside an explicit transaction, i.e., a BEGIN; ...; COMMIT; block.
In v17 PR#13040, this behavior was being changed so that it also throttles work outside of explicit transactions for INSERT/UPDATE/DELETE/LOAD queries.

New command line flags and behavior

Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size

Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes.

In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size.

  • --builtinbackup-file-read-buffer-size: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.
  • --builtinbackup-file-write-buffer-size: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)

These flags are applicable to the following programs:

  • vtbackup
  • vtctld
  • vttablet
  • vttestserver

Manifest backup external decompressor command

Add a new builtin/xtrabackup flag --manifest-external-decompressor. When set the value of that flag is stored in the manifest field ExternalDecompressor. This manifest field may be consulted when decompressing a backup that was compressed with an external command.

This feature enables the following flow:

  1. Take a backup using an external compressor
     Backup --compression-engine=external \
            --external-compressor=zstd \
            --manifest-external-decompressor="zstd -d"
    
  2. Restore that backup with a mere Restore command, without having to specify --external-decompressor.

vttablet --throttler-config-via-topo

This flag was introduced in v16 and defaulted to false. In v17 it defaults to true, and there is no need to supply it.

Note that this flag overrides --enable-lag-throttler and --throttle-threshold, ...

Read more

Vitess v17.0.0-rc1

06 Jun 14:36
d0caade
Compare
Choose a tag to compare
Vitess v17.0.0-rc1 Pre-release
Pre-release

Release of Vitess v17.0.0-rc1

Summary

Table of Contents

Major Changes

Breaking Changes

Default Local Cell Preference for TabletPicker

We added options to the TabletPicker that allow for specifying a cell preference in addition to making the default behavior to give priority to the local cell and any alias it belongs to. We are also introducing a new way to select tablet type preference which should eventually replace the in_order: hint currently used as a prefix for tablet types. The signature for creating a new TabletPicker now looks like:

func NewTabletPicker(
	ctx context.Context,
	ts *topo.Server,
	cells []string,
	localCell, keyspace, shard, tabletTypesStr string,
	options TabletPickerOptions,
) (*TabletPicker, error) {...}

Where ctx, localCell, option are all new parameters.

option is of type TabletPickerOptions and includes two fields, CellPreference and TabletOrder.
CellPreference: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the client TabletOrder`: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client

See PR 12282 Description for examples on how this changes cell picking behavior.

Default TLS version changed for vtgr

When using TLS with vtgr, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.

Dedicated stats for VTGate Prepare operations

Prior to v17 Vitess incorrectly combined stats for VTGate Execute and Prepare operations under a single stats key (Execute). In v17 Execute and Prepare operations generate stats under independent stats keys.

Here is a (condensed) example of stats output:

{
  "VtgateApi": {
    "Histograms": {
      "Execute.src.primary": {
        "500000": 5
      },
      "Prepare.src.primary": {
        "100000000": 0
      }
    }
  },
  "VtgateApiErrorCounts": {
    "Execute.src.primary.INVALID_ARGUMENT": 3,
    "Execute.src.primary.ALREADY_EXISTS": 1
  }
}

VTAdmin web migrated to vite

Previously, VTAdmin web used the Create React App framework to test, build, and serve the application. In v17, Create React App has been removed, and Vite is used in its place. Some of the main changes include:

  • Vite uses VITE_* environment variables instead of REACT_APP_* environment variables
  • Vite uses import.meta.env in place of process.env
  • Vitest is used in place of Jest for testing
  • Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
  • public/index.html has been moved to root directory in web/vtadmin

Keyspace name validation in TopoServer

Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.

Keyspace names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetKeyspace and CreateKeyspace methods return an error if given an invalid name.

Shard name validation in TopoServer

Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.

Shard names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetShard and CreateShard methods return an error if given an invalid name.

Compression CLI flags remove from vtctld and vtctldclient binaries

The CLI flags below were mistakenly added to vtctld and vtctldclient in v15. In v17, they are no longer present in those binaries.

  • --compression-engine-name
  • --compression-level
  • --external-compressor
  • --external-compressor-extension
  • --external-decompressor

VtctldClient command RestoreFromBackup will now use the correct context

The VtctldClient command RestoreFromBackup initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp.
Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 PR#12830,
this behavior was changed to use a context with a timeout of action_timeout. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough
time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes.

Vttablet's transaction throttler now also throttles DML outside of BEGIN; ...; COMMIT; blocks

Prior to v17, vttablet's transaction throttler (enabled with --enable-tx-throttler) would only throttle requests done inside an explicit transaction, i.e., a BEGIN; ...; COMMIT; block.
In v17 PR#13040, this behavior was being changed so that it also throttles work outside of explicit transactions for INSERT/UPDATE/DELETE/LOAD queries.

New command line flags and behavior

Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size

Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes.

In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size.

  • --builtinbackup-file-read-buffer-size: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.
  • --builtinbackup-file-write-buffer-size: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)

These flags are applicable to the following programs:

  • vtbackup
  • vtctld
  • vttablet
  • vttestserver

Manifest backup external decompressor command

Add a new builtin/xtrabackup flag --manifest-external-decompressor. When set the value of that flag is stored in the manifest field ExternalDecompressor. This manifest field may be consulted when decompressing a backup that was compressed with an external command.

This feature enables the following flow:

  1. Take a backup using an external compressor
     Backup --compression-engine=external \
            --external-compressor=zstd \
            --manifest-external-decompressor="zstd -d"
    
  2. Restore that backup with a mere Restore command, without having to specify --external-decompressor.

vttablet --throttler-config-via-topo

This flag was introduced in v16 and defaulted to false. In v17 it defaults to true, and there is no need to supply it.

Note that this flag overrides --enable-lag-throttler and --throttle-threshold, which now give warnings, and will be removed in v18.

New stats

####...

Read more

Vitess v16.0.2

09 May 15:15
6076fed
Compare
Choose a tag to compare

Release of Vitess v16.0.2

Known Issues

Schema-initialization stuck on semi-sync ACKs while upgrading to v16.0.2

During upgrades from <= v15.x.x to v16.0.2, as part of PromoteReplica call, the schema-init realizes that there are schema diffs to apply and ends up writing to the database if semi-sync is enabled, all of these writes get blocked indefinitely.
Eventually, PromoteReplica fails, and this fails the entire PRS call.

A fix for this issue was merged on release-16.0 in PR#13441, read the corresponding bug report to learn more.

This issue is fixed in v16.0.3 and later patch releases.

Broken downgrade from v17.x.x when super_read_only turned on by default

In v17.x.x super_read_only is turned on by default meaning that downgrading from v17 to v16.0.2 breaks due to init_db.sql needing write access.

This issue is fixed in >= v16.0.3 thanks to PR #13525

Major Changes

Upgrade to go1.20.3

Vitess v16.0.2 now runs on go1.20.3.
Below is a summary of this Go patch release. You can learn more here.

go1.20.3 (released 2023-04-04) includes security fixes to the go/parser, html/template, mime/multipart, net/http, and net/textproto packages, as well as bug fixes to the compiler, the linker, the runtime, and the time package. See the Go 1.20.3 milestone on our issue tracker for details.

EffectiveCallerId in Vtgate gRPC calls

A new flag grpc-use-static-authentication-callerid is added to gate the behavior introduced in #12050.
Earlier, we used to automatically set immediateCallerID to user from static authentication context that overrode the EffectiveCallerId.

Shard name validation in TopoServer

Prior to v16.0.2, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.

Shard names may no longer contain the forward slash ("/") character, and TopoServer's CreateShard method returns an error if given such a name.


The entire changelog for this release can be found here.

The release includes 24 commits (excluding merges)

Thanks to all our contributors: @GuptaManan100, @ajm188, @frouioui, @github-actions[bot], @harshit-gangal, @mattlord, @systay, @vitess-bot[bot]

Vitess v16.0.1

30 Mar 16:21
d1ba625
Compare
Choose a tag to compare

Release of Vitess v16.0.1

Known Issues

Schema-initialization stuck on semi-sync ACKs while upgrading to v16.0.1

During upgrades from <= v15.x.x to v16.0.1, as part of PromoteReplica call, the schema-init realizes that there are schema diffs to apply and ends up writing to the database if semi-sync is enabled, all of these writes get blocked indefinitely.
Eventually, PromoteReplica fails, and this fails the entire PRS call.

A fix for this issue was merged on release-16.0 in PR#13441, read the corresponding bug report to learn more.

This issue is fixed in v16.0.3 and later patch releases.

Broken downgrade from v17.x.x when super_read_only turned on by default

In v17.x.x super_read_only is turned on by default meaning that downgrading from v17 to v16.0.1 breaks due to init_db.sql needing write access.

This issue is fixed in >= v16.0.3 thanks to PR #13525

Major Changes

Upgrade to go1.20.2

Vitess v16.0.1 now runs on go1.20.2.
Below is a summary of this Go patch release. You can learn more here.

go1.20.2 (released 2023-03-07) includes a security fix to the crypto/elliptic package, as well as bug fixes to the compiler, the covdata command, the linker, the runtime, and the crypto/ecdh, crypto/rsa, crypto/x509, os, and syscall packages.

Keyspace name validation in TopoServer

Prior to v16.0.1, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.

Keyspace names may no longer contain the forward slash ("/") character, and TopoServer's GetKeyspace and CreateKeyspace methods return an error if given such a name.


The entire changelog for this release can be found here.

The release includes 39 commits (excluding merges)

Thanks to all our contributors: @GuptaManan100, @ajm188, @frouioui, @github-actions[bot], @mattlord, @rohit-nayak-ps, @rsajwani, @shlomi-noach, @systay, @vitess-bot[bot]

Vitess v15.0.3

30 Mar 16:34
652b0da
Compare
Choose a tag to compare

Release of Vitess v15.0.3

The entire changelog for this release can be found here.

The release includes 52 commits (excluding merges)

Thanks to all our contributors: @GuptaManan100, @ajm188, @dbussink, @deepthi, @frouioui, @harshit-gangal, @mattlord, @rsajwani, @shlomi-noach, @systay, @vitess-bot[bot], @vmg