Skip to content

Introduce NativeEngines990KnnVectorsScorer to decouple native SIMD scoring selection from FaissMemoryOptimizedSearcher#3184

Open
VijayanB wants to merge 1 commit intoopensearch-project:mainfrom
VijayanB:nativescorer-to-use-prefetch
Open

Introduce NativeEngines990KnnVectorsScorer to decouple native SIMD scoring selection from FaissMemoryOptimizedSearcher#3184
VijayanB wants to merge 1 commit intoopensearch-project:mainfrom
VijayanB:nativescorer-to-use-prefetch

Conversation

@VijayanB
Copy link
Member

@VijayanB VijayanB commented Mar 18, 2026

Description

Previously, FaissMemoryOptimizedSearcher was responsible for detecting MMap-backed vectors and choosing between NativeRandomVectorScorer and the default Java scorer. This logic is now extracted into a dedicated FlatVectorsScorer decorator that sits in the codec scoring chain, making native SIMD acceleration available to all scoring paths (including HNSW graph traversal with prefetch support).

By placing NativeEngines990KnnVectorsScorer inside PrefetchableFlatVectorScorer in the decorator chain, the native scorer now benefits from prefetch-enabled bulk scoring during HNSW graph traversal. Previously, NativeRandomVectorScorer was instantiated directly in FaissMemoryOptimizedSearcher, bypassing the prefetch layer entirely. With this change, PrefetchableFlatVectorScorer can wrap the NativeRandomVectorScorer returned by NativeEngines990KnnVectorsScorer, issuing prefetch hints for memory-mapped vector data before native SIMD scoring, reducing memory access latency during graph neighbor evaluation.

Key changes:

  • Add NativeEngines990KnnVectorsScorer which wraps a FlatVectorsScorer delegate and transparently returns NativeRandomVectorScorer when the bottom-level FloatVectorValues implements MMapVectorValues and the similarity function is EUCLIDEAN or MAXIMUM_INNER_PRODUCT
  • Simplify FaissMemoryOptimizedSearcher by removing the native scoring branch and determineNativeFunctionType(); it now unconditionally delegates to the FlatVectorsScorer
  • Refactor NativeRandomVectorScorer to extend AbstractRandomVectorScorer, removing manual maxOrd/ordToDoc/getAcceptOrds overrides, enabling it to be wrapped by PrefetchableFlatVectorScorer for prefetch-enabled bulk scoring during HNSW graph traversal
  • Wire the new scorer into NativeEngines990KnnVectorsFormat between the Lucene99 flat vectors scorer and PrefetchableFlatVectorScorer
  • Add unit tests covering all routing branches

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…oring selection from FaissMemoryOptimizedSearcher

Previously, FaissMemoryOptimizedSearcher was responsible for detecting
MMap-backed vectors and choosing between NativeRandomVectorScorer and
the default Java scorer. This logic is now extracted into a dedicated
FlatVectorsScorer decorator that sits in the codec scoring chain,
making native SIMD acceleration available to all scoring paths
(including HNSW graph traversal with prefetch support).

By placing NativeEngines990KnnVectorsScorer inside
PrefetchableFlatVectorScorer in the decorator chain, the native scorer
now benefits from prefetch-enabled bulk scoring during HNSW graph
traversal. Previously, NativeRandomVectorScorer was instantiated
directly in FaissMemoryOptimizedSearcher, bypassing the prefetch layer
entirely. With this change, PrefetchableFlatVectorScorer can wrap the
NativeRandomVectorScorer returned by NativeEngines990KnnVectorsScorer,
issuing prefetch hints for memory-mapped vector data before native SIMD
scoring, reducing memory access latency during graph neighbor evaluation.

Key changes:
- Add NativeEngines990KnnVectorsScorer which wraps a FlatVectorsScorer
  delegate and transparently returns NativeRandomVectorScorer when the
  bottom-level FloatVectorValues implements MMapVectorValues and the
  similarity function is EUCLIDEAN or MAXIMUM_INNER_PRODUCT
- Simplify FaissMemoryOptimizedSearcher by removing the native scoring
  branch and determineNativeFunctionType(); it now unconditionally
  delegates to the FlatVectorsScorer
- Refactor NativeRandomVectorScorer to extend AbstractRandomVectorScorer,
  removing manual maxOrd/ordToDoc/getAcceptOrds overrides, enabling it
  to be wrapped by PrefetchableFlatVectorScorer for prefetch-enabled
  bulk scoring during HNSW graph traversal
- Wire the new scorer into NativeEngines990KnnVectorsFormat between the
  Lucene99 flat vectors scorer and PrefetchableFlatVectorScorer
- Add unit tests covering all routing branches

Signed-off-by: Vijayan Balasubramanian <balasvij@amazon.com>
@codecov
Copy link

codecov bot commented Mar 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.64%. Comparing base (6c4d427) to head (7a2c310).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3184      +/-   ##
============================================
- Coverage     82.66%   82.64%   -0.02%     
+ Complexity     3980     3979       -1     
============================================
  Files           431      432       +1     
  Lines         14783    14778       -5     
  Branches       1882     1882              
============================================
- Hits          12221    12214       -7     
- Misses         1801     1802       +1     
- Partials        761      762       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants