Ability to use rescore with Inner hits

[benwtrent]

Description

For nested queries, and more recently, collapse, its possible to have a top-level rescore that applies to the top-level documents.

However, the inner-hit scores may not reflect the top-level rescore.

It would be beneficial to add rescore to inner_hits so that individual inner hits can be rescored to reflect their top-level score.

The question becomes an API level one.

A: Do we want to simply add a rescore element that is a fully-feature rescore, just like the rescore field in _searcu?

B: Or do we add an option for inner_hits to take advantage of the already existing top-level rescore, but allow them to set override options like window_size?

@jimczi what do you think?

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[jimczi]

My preference goes to option A. Rescorers can be cascaded so a single window_size would be confusing. Option A would also be aligned with the fact that inner_hits build a complete search request. For instance the sort option is already exposed so applying the rescorer of the main search request might not be always compatible.
We also need to be careful since inner_hits is a concept shared with the nested query so we need to decide whether the new section would be applicable there.

[benwtrent]

@jimczi I agree, option A is my preferred one as well.

We also need to be careful since inner_hits is a concept shared with the nested query so we need to decide whether the new section would be applicable there.

I think using it in nested makes sense to me actually. Folks can use rescore & nested just fine, and it could be useful for more extreme cases (even with passage vector search, doing a passage vector rerank per doc with a script or model).