CNDB-13192: Bump the default cassandra.sai.latest.version from DC to EC so we support BM25 by default.

[ekaterinadimitrova2]

What is the issue

...
Bump the default cassandra.sai.latest.version from DC to EC so we support BM25 by default.
Update jvector_version to 4.

What does this PR fix and why was it fixed

...
Bump the default cassandra.sai.latest.version from DC to EC so we support BM25 by default.
Update jvector_version to 4.

[github-actions]

Checklist before you submit for review

• Make sure there is a PR in the CNDB project updating the Converged Cassandra version
• Use NoSpamLogger for log lines that may appear frequently in the logs
• Verify test results on Butler
• Test coverage for new/modified code is > 80%
• Proper code formatting
• Proper title for each commit staring with the project-issue number, like CNDB-1234
• Each commit has a meaningful description
• Each commit is not very long and contains related changes
• Renames, moves and reformatting are in distinct commits
• All new files should contain the DataStax copyright header instead of the Apache License one

[Copilot]

Pull Request Overview

This PR bumps the default SAI version configuration to support BM25 by default, fixes Version.LATEST to point to ED, and improves test coverage by moving and adding tests.

• Update default SAI version property in CassandraRelevantProperties.java.
• Change Version.LATEST from EC to ED in Version.java.
• Update and refactor tests across BM25Test, FeaturesVersionSupportTest, SAITester, and add FeaturesVersionSupportEDTest.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/unit/org/apache/cassandra/index/sai/cql/FeaturesVersionSupportTest.java	Added new testIndexMetaForNumRows and supporting helper methods for SAI index metadata validation.
test/unit/org/apache/cassandra/index/sai/cql/BM25Test.java	Removed duplicate SAI index metadata test code.
test/unit/org/apache/cassandra/index/sai/SAITester.java	Moved common collection data utilities to SAITester.
test/distributed/org/apache/cassandra/distributed/test/sai/features/FeaturesVersionSupportEDTest.java	Added distributed test for ED version support.
src/java/org/apache/cassandra/index/sai/disk/format/Version.java	Updated Version.LATEST to point to ED.
src/java/org/apache/cassandra/config/CassandraRelevantProperties.java	Updated default property value for SAI version from "dc" to "ec".

[Copilot]

Consider replacing the use of the Java assert statement with an explicit JUnit assertion (such as assertTrue) to ensure the check is enforced even when JVM assertions are disabled.

Suggested change

	assert memIndex instanceof TrieMemtableIndex;
	Assert.assertTrue(memIndex instanceof TrieMemtableIndex);

[Copilot]

If multiple memtables exist, the current implementation overwrites the rowCount per iteration instead of accumulating values. Consider summing the row counts from all memtables to correctly reflect the total number of rows.

Suggested change

	rowCount = Arrays.stream(((TrieMemtableIndex) memIndex).getRangeIndexes())
	.map(index -> ((TrieMemoryIndex) index).getDocLengths().size())
	.mapToInt(Integer::intValue).sum();
	rowCount += Arrays.stream(((TrieMemtableIndex) memIndex).getRangeIndexes())
	.map(index -> ((TrieMemoryIndex) index).getDocLengths().size())
	.mapToInt(Integer::intValue).sum();

[ekaterinadimitrova2]

The switch to testing on the latest version ED led to two tests failing:

• testFallbackToAnotherIndex - I haven't had the chance to look in detail at why, I will take a look tomorrow.
• testSegmentsMetadata - some output needs to be changed to account for the new metadata. I will take care tomorrow.

This one is not a regression, as marked by Butler, there is also a ticket for it - testKDTreePostingsQueryMetricsWithSingleIndex - CNDB-13830

The others seem to be failures with previous versions, where we did not touch anything, so they seem unrelated.
testUnsupportedOperator {p0=db} and testTermsIteration {p0=dc}

[ekaterinadimitrova2]

Updated the PR with only the default version update as per the original intention.
I will take care of the ED testing and fixing some bugs that were discovered in a separate ticket.

[ekaterinadimitrova2]

We fixed 3 issues related to EB+ versions that were caught by CI testing in other tickets.
So I rebased the branch and also updated jvector_version to 4.
Let's see what the new CI run will show.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[cassci-bot]

❌ Build ds-cassandra-pr-gate/PR-1737 rejected by Butler

---

4 new test failure(s) in 1 builds
See build details here

---

Found 4 new test failures

Test	Explanation	Branch history	Upstream history
...dexWhileQueryingTest.testFallbackToAnotherIndex	regression	🔴	🔵🔵🔵🔵🔵🔵🔵
...herTest.testEqQueriesAgainstStringIndex {p0=ec}	regression	🔴	🔵🔵🔵🔵🔵🔵🔵
...testTermQueriesAgainstShortPostingLists {p0=ed}	regression	🔴	🔵🔵🔵🔵🔵🔵🔵
o.a.c.u.b.BinLogTest.testTruncationReleasesLogS...	regression	🔴	🔵🔵🔵🔵🔵🔵🔵

---

Found 2 known test failures