Are Advanced text queries on nested json fields (eg regex/phrase_prefix) supported?

[choncou]

It isn't clear if regex/phrase_prefix queries should work on nested json field. Basic search against nested json fields as described in the documentation does work, but not some of the advanced search queries (eg. regex/phrase_prefix/fuzzy_term).

Eg. json data stored in data jsonb column, specified as json field when creating the index.
Given a row that can be found with

SELECT * FROM raw_contacts_bm25.search(
	'data.enriched.email:"[email protected]"'
);

I expected to be able to run a regex search query that would return the same record (and potentially others):

SELECT * FROM raw_contacts_bm25.search(
      query => paradedb.regex(
        field => 'data.enriched.email',
        pattern => '.*harris\.example'
      )
    );

But this raises the following error:

Query 1 ERROR at Line 1: : ERROR:  could not parse query: WrongFieldType("data.enriched.email")
CONTEXT:  SQL statement "SELECT * FROM public.raw_contacts WHERE  id @@@ __paradedb_search_config__"
PL/pgSQL function raw_contacts_bm25.search(paradedb.searchqueryinput,integer,integer,text,boolean,text,text,text,integer) line 16 at RETURN QUERY

[neilyio]

@choncou thanks for bringing this up. This isn't currently supported, but if its possible in Tantivy then will be an easy fix on our side.

The nuance is that there's a little processing work to do convert a JSON path into a "term" for queries like FuzzyTerm. It's not clear if Tantivy supports RegexQuery on JSON documents at all, I can't find any tests in their codebase that demonstrate this.

I'm going to open an issue for this. We'll discuss with the Tantivy maintainers and report back as soon as possible.

[philippemnoel]

Let's track on Neil's issue, here: #1553