Highlights from exluded field not showing up in Instantsearch hit (question)

[ghost]

Description

More of a question than an issue.

On the typesense end, I am using a scoped api key to remove fullText field from the search results but including the fullText highlight snippets.

This is what typesense returns for the search results, which is exactly what I need:

It excludes the full field value and only shows the snippet for the relevant content.

However this data/info seems to get lost with when it goes through instantsearch to the 'Hit' object. The highlighted snippet is not present in the _highlightResult or _snippetResult:

Is this a limitation of instantsearch, where it doesn't pick up a highlight field if it's not present in the search results? Is there anyway for me to use the highlight snippet without having to replace instantsearch with something custom? Is there a setting that will allow the highlight snippet from the typesense server to pass through to the instantsearch hit?

Thanks!

[jasonbosco]

Could you share the code snippet that shows how you instantiate the adapter, and also the version of typesense-instantsearch-adapter and typesense server you're using?

[ghost]

I am using typesense cloud (v0.24.0) and typesense-instantsearch-adapter 2.6.0.

First I am creating a scoped api key like so:

  let scopedAPIKey = client
    .keys()
    .generateScopedSearchKey(keyWithSearchPermissions, {
      exclude_fields: "fullText",
      highlight_fields: "name,fullText,providerName,description,tags",
      highlight_full_fields: "name,providerName,description,tags",
      limit_hits: 20,
      //      limit_multi_searches: 20,
    });

Then I am using that scopedApiKey to instantiate the typesense adapter:

const researchSearchClient = new TypesenseInstantSearchAdapter({
  server: TYPESENSE_CONFIG,
  additionalSearchParameters: {
    query_by: "name,fullText",
    drop_tokens_threshold: 0,
    sort_by: "publishedAt:desc,_text_match:desc",
  },
}).searchClient;

I also tried including 'fullText' in the highlight_fields as part of the adapter parameters.

[ghost]

going through the adapter code, I think I've tracked down where the highlight gets lost. The adapter creates highlights by iterating over the document fields and finding corresponding entries in the highlight object. but because I have explicitly excluded the value of the 'fullText' field from the document it wont get added to the snippets, even though the snippet is present in the server response.

The solution is probably to do concat the document key/value entries with the highlight key/snippet entries where the key is not in the document.

What do you think?

Below is the code I am referring to:

  _adaptHighlightInObjectValue(objectValue, highlightObjectValue, snippetOrValue) {
    return Object.assign(
      {},
      **/// here it creates snippet results by mapping over the document, but we want to highlight a field
      /// which we intentionally did not include in the document.**
      ...Object.entries(objectValue).map(([attribute, value]) => {
        let adaptedValue;
        if (value == null) {
          adaptedValue = this._adaptHighlightNullValue();
        } else if (Array.isArray(value)) {
          adaptedValue = this._adaptHighlightInArrayValue(
            value,
            highlightObjectValue?.[attribute] ?? [],
            snippetOrValue
          );
        } else if (typeof value === "object") {
          adaptedValue = this._adaptHighlightInObjectValue(
            value,
            highlightObjectValue?.[attribute] ?? {},
            snippetOrValue
          );
        } else {
          adaptedValue = this._adaptHighlightInPrimitiveValue(value, highlightObjectValue?.[attribute], snippetOrValue);
        }

        return {
          [attribute]: adaptedValue,
        };
      })
    );
  }

[ghost]

I tried to fix this with the code in the pull request. While it worked in the tests, it didn't work when I installed the PR as an npm package so I'm clearly missing something. I will resort to using the _rawTypesenseHit field to construct a custom highlight instead.

[jasonbosco]

The solution is probably to do concat the document key/value entries with the highlight key/snippet entries where the key is not in the document.

Yup. I'll re-open this, so we can keep a track of this issue. I plan to address this in the next few weeks.