[CALCITE-6274] Two Elasticsearch index join return empty result #3696
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CALCITE-6274
Two index of Elasticsearch join return empty result even if the data from both indexes can match。
create index test_01:
PUT /test_01/_doc/1 { "doc_id" : 1, "doc_desc" : "doc01" }
create index test_02:
PUT /test_02/_doc/1 { "doc_id" : 1, "doc_score" : 90 }
execute sql:
select * from es.test_01 t1 join es.test_02 t2 on cast(t1._MAP['doc_id'] as bigint) = cast(t2._MAP['doc_id'] as bigint)
the code generate by ElasticsearchToEnumerableConverter like this:
subquery of index test_01:
{ return ((org.apache.calcite.adapter.elasticsearch.ElasticsearchTable.ElasticsearchQueryable) org.apache.calcite.schema.Schemas.queryable(root, root.getRootSchema().getSubSchema("es"), java.lang.Object[].class, "test_01")).find(java.util.Collections.EMPTY_LIST, java.util.Arrays.asList(new org.apache.calcite.util.Pair[] { new org.apache.calcite.util.Pair( "_MAP", java.util.Map.class), new org.apache.calcite.util.Pair( "_1", java.lang.Long.class)}), java.util.Arrays.asList(new org.apache.calcite.util.Pair[] { new org.apache.calcite.util.Pair( "doc_id", org.apache.calcite.rel.RelFieldCollation.Direction.ASCENDING)}), java.util.Collections.EMPTY_LIST, java.util.Arrays.asList(new org.apache.calcite.util.Pair[] {}), com.google.common.collect.ImmutableMap.of("$f1", "doc_id"), null, null); }
project field names:
"_MAP", "_1"
subquery of index test_02:
{ return ((org.apache.calcite.adapter.elasticsearch.ElasticsearchTable.ElasticsearchQueryable) org.apache.calcite.schema.Schemas.queryable(root, root.getRootSchema().getSubSchema("es"), java.lang.Object[].class, "test_02")).find(java.util.Collections.EMPTY_LIST, java.util.Arrays.asList(new org.apache.calcite.util.Pair[] { new org.apache.calcite.util.Pair( "_MAP", java.util.Map.class), new org.apache.calcite.util.Pair( "_1", java.lang.Long.class)}), java.util.Arrays.asList(new org.apache.calcite.util.Pair[] { new org.apache.calcite.util.Pair( "doc_id", org.apache.calcite.rel.RelFieldCollation.Direction.ASCENDING)}), java.util.Collections.EMPTY_LIST, java.util.Arrays.asList(new org.apache.calcite.util.Pair[] {}), com.google.common.collect.ImmutableMap.of("$f1", "doc_id"), null, null); }
project field names:
"_MAP", "_1"
This org.apache.calcite.adapter.elasticsearch.ElasticsearchTable.ElasticsearchQueryable#find function actually execute request, subq-query result projected according to second paramter fields. Field "_1" can not find from subq-query result. "_1" not in mappings {"$f1":"doc_id"}, cause two sub-query join condition value is null, so the result of sql is empty.
This PR fix the problem, the sub-query project field names is:
"_MAP", "$f1"
. join condition field$f1
can be find in mappings. The result of sql match expectations.