Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-17269: Do not publish synthetic solr core (of Coordinator node) to ZK #2438

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

patsonluk
Copy link
Contributor

@patsonluk patsonluk commented May 2, 2024

https://issues.apache.org/jira/browse/SOLR-17269

Description

The synthetic collection/replica introduced in Solr 9 registers such collection to zookeeper. This might cause issues with other tools which monitor solr collections via zk and they might be mistaken the synthetic collection as a real collection.

Solution

We are introducing a new class SyntheticSolrCore which extends SolrCore, SyntheticSolrCore is only used in Coordinator node to support a subset of SolrCore functionalities required by Coordinator operations such as aggregating and writing out response and providing configset info. There will be one instance of SyntheticSolrCore per config set.

Such synthetic core is only registered to the CoreContainer but is no longer registering itself to zookeeper.

Take note that to minimize code changes, there will still be snapshot/index data directory written to the Coordinator node. We could have overridden initSnapshotMetaDataManager and initIndex to be no-op to avoid creating them, however, there is currently other code (and possibly future code too) that might rely on the existence of those fields. It is safer to just keep them as is.

The data folder for the synthetic core would NOT be loaded on QA node start-up as there's no core.properties in such directory. This is because instead of going through CoreContainer#create for such synthetic core, it's using SyntheticSolrCore#createAndRegisterCore, which bypass core.properties creation and zk registration

Tests

The existing TestCoordinatorRole has already provided good coverage of various scenarios. Take note that we have removed the test code that verifies the existence of cores from the synthetic collection since zk no longer keeps track of synthetic collections. Instead we added a check in TestCoordinatorRole#testSimple to ensure that the synthetic collection no longer exists in cluster state.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide

* Experimental QA core as proxy in memory

* Experimental QA core as proxy in memory

* Experimental QA core as proxy in memory

* ./gradlew tidy

* Comment out portion that checks for synthetic collection/core existence

* Fixes watches and test cases

* Refactoring to avoid modifying existing method modifiers

* Cleanup

* Some refactoring to minimize changes to SolrCore

* javadoc

* ./gradlew tidy

* code cleanup

* code cleanup

* Fix resource loader issue if configSetName is null

* ./gradlew tidy

* Fixed issue with incorrect collection name in context
Ensure only one synthetic core created per config set

* ./gradlew tidy

* Correctly use <config set>_core as synthetic core name for clarity
Should open the registered core on first synthetic core creation since it is a getCore op

* ./gradlew tidy

* Fixed issue with incorrect collection name in context
@patsonluk patsonluk marked this pull request as ready for review May 3, 2024 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant