Skip to content

[enhance](nereids) add eliminate order by key by data trait (#46225) #52090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: branch-3.1
Choose a base branch
from
Open

[enhance](nereids) add eliminate order by key by data trait (#46225) #52090

wants to merge 1 commit into from

Conversation

feiniaofeiafei
Copy link
Contributor

cherry-pick from #46225

@feiniaofeiafei
[enhance](nereids) add eliminate order by key by data trait (apache#4…
715e1d9 
…6225)

case 1: eliminate by uniform
select a,b,c from test where a=1 order by a;
->
select a,b,c from test where a=1;

case 2: eliminate by function dependency
select a from t1 order by a, a+1;
select a from t1 order by a, abs(a) ;
select a from t1 where a=c order by a,c
->
select a from t1 order by a;

case 3: eliminate by duplicate
select a from t1 order by a, a;
->
select a from t1 order by a;

window expression order by key can also be eliminated.
@Thearas
Copy link
Contributor

Thearas commented Jun 20, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39441 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 715e1d912687d8fc17c92e7cfac35b6bfbf2f4c3, data reload: false

------ Round 1 ----------------------------------
q1	17583	6728	6608	6608
q2	2059	166	159	159
q3	10582	1106	1177	1106
q4	10500	700	729	700
q5	7750	2799	2819	2799
q6	211	136	140	136
q7	997	621	612	612
q8	9337	1930	1946	1930
q9	6631	6376	6408	6376
q10	6978	2254	2289	2254
q11	460	267	266	266
q12	391	208	209	208
q13	17769	2988	2971	2971
q14	231	207	208	207
q15	516	455	477	455
q16	480	367	363	363
q17	969	528	541	528
q18	7171	6662	6618	6618
q19	1346	961	985	961
q20	462	198	205	198
q21	3906	3059	2997	2997
q22	1114	995	989	989
Total cold run time: 107443 ms
Total hot run time: 39441 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6614	6549	6568	6549
q2	320	228	233	228
q3	2828	2797	2795	2795
q4	2018	1773	1796	1773
q5	5711	5735	5698	5698
q6	218	128	127	127
q7	2202	1772	1814	1772
q8	3379	3527	3545	3527
q9	8870	8762	8877	8762
q10	3556	3597	3524	3524
q11	580	488	501	488
q12	822	597	589	589
q13	8987	3126	3129	3126
q14	302	265	266	265
q15	505	468	464	464
q16	508	451	421	421
q17	1828	1612	1601	1601
q18	8190	7764	7835	7764
q19	1666	1603	1525	1525
q20	2113	1869	1916	1869
q21	5215	4903	4959	4903
q22	1150	1047	1023	1023
Total cold run time: 67582 ms
Total hot run time: 58793 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196464 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 715e1d912687d8fc17c92e7cfac35b6bfbf2f4c3, data reload: false

query1	1304	905	893	893
query2	6281	2094	2008	2008
query3	10813	4413	4222	4222
query4	61912	28866	23568	23568
query5	5170	456	469	456
query6	426	177	206	177
query7	5509	307	314	307
query8	313	228	231	228
query9	8484	2562	2571	2562
query10	465	262	270	262
query11	17586	15213	15768	15213
query12	155	106	103	103
query13	1501	442	419	419
query14	10358	6684	6561	6561
query15	200	193	185	185
query16	7163	494	501	494
query17	1132	564	575	564
query18	1874	325	314	314
query19	209	164	157	157
query20	118	114	112	112
query21	201	107	104	104
query22	4881	4634	4787	4634
query23	34537	34476	33953	33953
query24	6115	2925	2917	2917
query25	541	420	435	420
query26	653	165	169	165
query27	1917	353	346	346
query28	4141	2130	2127	2127
query29	721	472	443	443
query30	233	165	164	164
query31	1030	848	796	796
query32	69	57	59	57
query33	429	311	306	306
query34	925	520	532	520
query35	861	750	726	726
query36	1085	985	930	930
query37	109	71	72	71
query38	4016	3968	3934	3934
query39	1547	1526	1453	1453
query40	201	105	100	100
query41	47	44	45	44
query42	115	106	100	100
query43	539	494	487	487
query44	1157	794	796	794
query45	181	166	172	166
query46	1170	736	716	716
query47	2038	1951	1952	1951
query48	427	343	337	337
query49	731	398	402	398
query50	865	439	438	438
query51	7430	7270	7371	7270
query52	102	95	97	95
query53	263	180	177	177
query54	571	463	455	455
query55	78	78	81	78
query56	258	262	258	258
query57	1339	1229	1226	1226
query58	216	212	231	212
query59	3351	3099	3195	3099
query60	284	253	259	253
query61	124	106	106	106
query62	768	661	676	661
query63	209	180	174	174
query64	1346	676	619	619
query65	3271	3175	3167	3167
query66	713	284	307	284
query67	15892	15468	15812	15468
query68	4144	563	566	563
query69	424	276	272	272
query70	1077	1121	1066	1066
query71	359	265	255	255
query72	6426	4005	4109	4005
query73	752	341	346	341
query74	10572	9290	9246	9246
query75	3369	2600	2689	2600
query76	2059	966	1136	966
query77	545	272	265	265
query78	10658	9522	9624	9522
query79	1689	608	600	600
query80	1013	425	411	411
query81	498	219	225	219
query82	175	94	84	84
query83	161	141	144	141
query84	283	78	78	78
query85	972	307	319	307
query86	368	285	303	285
query87	4379	4253	4290	4253
query88	4186	2387	2358	2358
query89	400	285	281	281
query90	2007	183	182	182
query91	145	111	107	107
query92	61	52	50	50
query93	2249	544	546	544
query94	774	299	301	299
query95	353	256	254	254
query96	607	274	278	274
query97	3357	3170	3178	3170
query98	205	203	198	198
query99	1557	1272	1264	1264
Total cold run time: 314494 ms
Total hot run time: 196464 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.37 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 715e1d912687d8fc17c92e7cfac35b6bfbf2f4c3, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.06	0.06
query4	1.62	0.10	0.10
query5	0.52	0.50	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.50	0.50
query10	0.56	0.54	0.55
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.59
query14	0.77	0.79	0.81
query15	0.85	0.83	0.83
query16	0.38	0.37	0.40
query17	1.01	1.04	1.04
query18	0.23	0.23	0.22
query19	1.87	1.86	1.84
query20	0.01	0.01	0.01
query21	15.43	0.59	0.60
query22	2.35	2.14	1.87
query23	17.09	0.98	0.88
query24	3.33	1.92	1.06
query25	0.21	0.19	0.06
query26	0.50	0.15	0.14
query27	0.03	0.04	0.04
query28	9.30	0.52	0.51
query29	12.57	3.27	3.20
query30	0.25	0.06	0.06
query31	2.86	0.39	0.38
query32	3.22	0.47	0.45
query33	2.95	3.01	3.01
query34	17.14	4.50	4.46
query35	4.65	4.51	4.45
query36	0.65	0.47	0.47
query37	0.08	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.78 s
Total hot run time: 30.37 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow Awaiting requested review from morrySnow morrySnow is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@feiniaofeiafei @Thearas @doris-robot