1.42.11

Merged PRs

dolt

• 8261: liuliu/graph-spaces-fix
• 8259: liuliu/graph-remove-extra-s
• 8258: Bug fix: Testing for invalid global configuration dir permissions earlier, to prevent a panic
  Customer issue: #8250
• 8252: go/store/nbs: Fix table_index for table files with so many chunks that certain index slice operations overflow a uint32.
• 8251: liuliu/color-string-fix
• 8248: support \checkout, \merge, \show
  Until now there was an awkward behavior in dolt sql shell where the \checkout and \merge commands didn't really play nice when they could have. Specifically, if you used dolt sql and it was connected to a remote host, then running \checkout would give you an error telling you to stop the server. That is no longer the case. \checkout and \merge will work well in the dolt sql shell when connected to a remote host now.
  Also added the \show command
  And made the expect tests more correct and expanded.
• 8247: Fix displaying AddressMap non-leaf nodes in noms show
  This fixes an index out of bounds panic when trying to display AddressMap non-leaf nodes in noms show.
  I didn't add a regression test, because it turns out that all noms show tests only work against the old deprecated format, so adding more tests to the test suite wouldn't actually do anything. It's also mostly unused: we basically only use it as part splunk.pl to visualize chunks for debugging.
  Instead of fixing the test suite, it would be a better use of my time to fully deprecate noms show and switch splunk.pl to use dolt show instead. Especially now that #8143 added support for visualizing prolly tree chunks in dolt show, bringing it to feature parity with noms show.
• 8245: Feature: support for --empty=[drop|keep] in dolt_rebase()
  Adds support for the --empty option to dolt_rebase(). This option controls how commits that become empty are handled. For example, if a branch is rebased and all the changes in one commit on that branch have already been applied to the upstream branch, then when that commit is reapplied, it will end up being empty. The two initially supported values are drop and keep. (Git also supports a stop value, which lets the user manually intervene.)
  Also adds support for the --allow-empty flag for dolt_cherry_pick(). This flag controls whether Dolt will cherry-pick empty commits (i.e. commits that start off as empty, not commits that become empty after they are applied).
  These behaviors are slightly confusing for two reasons: 1) Git distinguishes between a commit that starts off empty and a commit that becomes empty while applying its changes, and 2) rebase and cherry-pick have slightly different default values for these two options. The differences are summarized below.

  	Commits that start empty	Commits that become empty
  rebase	default: keep, can be overridden with --no-keep-empty (--keep-empty is also supported)	default: drop, can be overridden with --empty=keep. For interactive rebases, the default changes to stop, which is not supported by Dolt yet.
  cherry-pick	default: fail, can be overridden with --allow-empty	default: stop, can be overridden with --empty=keep or --empty=drop. Dolt does not support stop yet, so Dolt's default is to fail.
  Related issue: #7830

• 8239: Rewrite the dolt show implementation.
  dolt show had an issue where it would not correctly display the SerialMessage for commits if provided with a hash. This came about as part of a refactor to make dolt show not depend on the env.DoltEnv object, when only exists on locally running servers, and not when connected to a remote server. Unfortunately, it looks like that refactor didn't actually remove the dependency either, as DoltEnv was still used in every possible invocation of dolt show
  To get it working, I essentially rewrote the implementation of dolt show in such a way that it now actually only uses DoltEnv when it can't get the necessary information from a running server: Basically, if we need to display SerialMessages or resolve branch names, we still rely on a locally running server. This can likely be improved in the future. But calls like dolt show #hash should now work against remote servers.
• 8226: Liuliu/log one line graph
  An example of the graph of us-jails:
  
• 8222: [kvexec] customized operator for count aggregation
  Operators that count the number of rows in a relation don't have to deserialize those KV's from storage (ex: select count(y) from xy where x > 1). There are some circumstances where we have to check for field nullability, but otherwise we can just count the KV's returned by the source iterator.
• 8129: Add optimized diffing and three-way merge of indexed JSON Documents.
  This PR adds some additional tests, but I plan on adding more tests around large documents before merging. Still, the implementation is ready for review.
  This adds a new JSON diffing algorithm designed for IndexedJSONDocument. Because three way merge only operates on values read from a Dolt table, which are always returned as a IndexedJSONDocuemt, this should mean that the original implementation is no longer used.

go-mysql-server

• 2629: normalize column defaults
  This PR adds a new analyzer rule to normalize literal column default values.
  This rule ensures that the default value is consistent for the column type (float defaults over int columns are rounded properly).
  It does this by evaluating the column default, and placing that into a NewLiteral of the proper type.
  Additionally, this ensures that dolt serialization receives consistent values (normalized floats and proper types).
  fixes: #8190
• 2627: Fix error when comparing incompatible types in IndexLookups
  When building lookups for IndexedTableAccess, we always convert the key type to the columns type.
  This is problematic when the key can't be converted to the column type without error.
  The expressions used in Filters properly handle this conversion, so we should default to that.
  Example:

  tmp/main*> create table t (i int primary key);
  tmp/main*> select * from t where i = json_array();
  error: '[]interface {}' is not a valid value type for 'int'

  This doesn't errror in MySQL. Also without a primary key or secondary index, the query succeeds in dolt.
• 2625: Bug fix: the timestamp function should convert to a datetime type
  MySQL's timestamp function, despite its name, actually returns a datetime type and not a timestamp type.
  MySQL example:

  mysql -uroot --protocol TCP -e "select timestamp('1000-01-01 00:00:00');" --column-type-info
  Field   1:  `timestamp('1000-01-01 00:00:00')`
  Catalog:    `def`
  Database:   ``
  Table:      ``
  Org_table:  ``
  Type:       DATETIME
  Collation:  binary (63)
  Length:     19
  Max_length: 19
  Decimals:   0
  Flags:      BINARY
  +----------------------------------+
  | timestamp('1000-01-01 00:00:00') |
  +----------------------------------+
  | 1000-01-01 00:00:00              |
  +----------------------------------+

  Note: We still need to add support for the second, optional parameter to timestamp().
  Customer issue: #8236

Closed Issues

• 4367: Add support for --graph option in dolt log
• 8190: Table schema stores unnormalized expression for default values, which leads to unexpected behaviors.
• 8236: Issue with datetime(6) column using timestamp(6) range

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	2.07	1.16	0.6
groupby_scan	12.98	16.71	1.3
index_join	1.34	2.66	2.0
index_join_scan	1.27	2.14	1.7
index_scan	34.33	55.82	1.6
oltp_point_select	0.18	0.3	1.7
oltp_read_only	3.43	5.88	1.7
select_random_points	0.33	0.65	2.0
select_random_ranges	0.39	0.81	2.1
table_scan	34.95	55.82	1.6
types_table_scan	75.82	144.97	1.9
reads_mean_multiplier			1.7

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	7.98	5.88	0.7
oltp_insert	3.75	2.97	0.8
oltp_read_write	8.43	11.87	1.4
oltp_update_index	3.82	2.97	0.8
oltp_update_non_index	3.89	2.91	0.7
oltp_write_only	5.37	6.09	1.1
types_delete_insert	7.7	6.43	0.8
writes_mean_multiplier			0.9

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	99.52	39.06	2.5
tpcc_tps_multiplier			2.5

Overall Mean Multiple	1.70