Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: subquery to join on not in #17623

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

KKould
Copy link
Member

@KKould KKould commented Mar 18, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This rule comes from the subquery optimization mentioned in the wetune paper: https://ipads.se.sjtu.edu.cn:1312/opensource/wetune/-/blob/main/wtune_data/issues/issues?ref_type=heads#L9

it will try to convert not in subquery to in subquery

SELECT *
FROM posts
         INNER JOIN topics ON topics.id = posts.topic_id
WHERE NOT posts.id IN (SELECT post_id FROM post_search_data WHERE locale = '0');

👇👇👇

SELECT *
FROM posts
         INNER JOIN topics ON topics.id = posts.topic_id
WHERE posts.id IN (SELECT p2.id
                   FROM posts p2
                            LEFT JOIN post_search_data pd ON locale = '0' AND p2.id = pd.post_id
                   WHERE pd.post_id IS NULL)

part of #17607

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Mar 18, 2025
@KKould KKould force-pushed the feat/subquery_to_join_notin_to_in branch from 2c071d5 to cc4d05f Compare March 18, 2025 11:07
@b41sh b41sh self-requested a review March 18, 2025 11:13
@KKould KKould force-pushed the feat/subquery_to_join_notin_to_in branch from ed3d61e to ef85815 Compare March 18, 2025 12:26
@KKould KKould force-pushed the feat/subquery_to_join_notin_to_in branch from ef85815 to 497995a Compare March 18, 2025 15:14
@KKould KKould force-pushed the feat/subquery_to_join_notin_to_in branch from 497995a to 6bf55e1 Compare March 18, 2025 16:12
@KKould KKould marked this pull request as draft March 19, 2025 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant