Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-43046][FOLLOWUP][SS][CONNECT] Remove not used line in deduplicateWithinWatermark #46345

Closed
wants to merge 1 commit into from

Conversation

WweiL
Copy link
Contributor

@WweiL WweiL commented May 2, 2024

What changes were proposed in this pull request?

An extra assignment was added when we first introduce dropDuplicatesWithinWatermark in 4d76511. We don't need this line.

Why are the changes needed?

Code cleanup

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing CI

Was this patch authored or co-authored using generative AI tooling?

No

@WweiL
Copy link
Contributor Author

WweiL commented May 2, 2024

cc @zhengruifeng Hi Ruifeng, I believe this line was added just to align with the behavior of this commit, 92847d9. But I think we don't have a drop_duplicates_within_watermark API. We could either add drop_duplicates_within_watermark, or remove that assignment. May I get your inputs as the original author of that PR? Thank you in advance!

@HyukjinKwon
Copy link
Member

Merged to master.

JacobZheng0927 pushed a commit to JacobZheng0927/spark that referenced this pull request May 11, 2024
…ateWithinWatermark

### What changes were proposed in this pull request?

An extra assignment was added when we first introduce `dropDuplicatesWithinWatermark` in apache@4d76511. We don't need this line.

### Why are the changes needed?

Code cleanup

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Existing CI

### Was this patch authored or co-authored using generative AI tooling?

No

Closes apache#46345 from WweiL/deduplicate-within-wm-connect.

Authored-by: Wei Liu <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants