[FEATURE] Make jVector plugin compatible with NeuralSearch plugin

[sam-herman]

Is your feature request related to a problem?
Currently the neural search plugin will break if we remove the KNN plugin.
However, neural search plugin functionality is important to users trying out OpenSearch functionalities such as hybrid search.

What solution would you like?
We need to follow up on the abstraction of the interfaces from the KNN plugin such that they are working seamlessly and create less friction for customers.
An issue to address this was opened and need to be followed up on.
opensearch-project/k-NN#2605

What alternatives have you considered?
NA

Do you have any additional context?
NA

[andrross]

Catch All Triage - 1 2 3 4

[sam-herman]

Assigned to @akash-shankaran for some reason I can't add him to the "assigned" field, I will add him as maintainer and hopefully this will be fixed.

[nimonkaranurag]

Hey @akash-shankaran, can we collaborate on this?

This is blocking us atm. Is there an ETD? thanks.

[akash-shankaran]

@nimonkaranurag - we have a neural-search version that works with JVector and OpenSearch 3.2 today. It is available as a zip / binary internally on IBM. Feel free to reach out on slack, and I can give you the drop.
The source code is present in my branch, if you want to take a look: akash-shankaran/neural-search#1

The community version of this will take some time. There are some core dependency problems in neural-search that need to be addressed. To achieve that, I've proposed moving some interfaces to opensearch-core which will help address this dependency. opensearch-project/OpenSearch#20050