Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids)LogicalPlanDeepCopier will lost some info when coping logical relation #34894

Merged
merged 5 commits into from
May 16, 2024

Conversation

starocean999
Copy link
Contributor

  1. add withRelationId to LogicalRelation. LogicalPlanDeepCopier use this method to do deep copy
  2. remove redundant code and add common method visitLogicalRelation for relations to do deep copy by default

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39809 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 660d50d4bde2ab0e90901817b09a1161f198e702, data reload: false

------ Round 1 ----------------------------------
q1	17965	4341	4197	4197
q2	2019	191	187	187
q3	10616	1246	1180	1180
q4	11057	809	846	809
q5	8448	2708	2608	2608
q6	215	131	130	130
q7	1003	607	581	581
q8	9514	2084	2082	2082
q9	9298	6679	6575	6575
q10	9242	3628	3661	3628
q11	456	238	234	234
q12	449	220	220	220
q13	18714	2972	2929	2929
q14	259	219	227	219
q15	507	475	463	463
q16	503	392	382	382
q17	974	667	665	665
q18	8064	7629	7423	7423
q19	3055	1537	1426	1426
q20	656	307	304	304
q21	5027	3284	4008	3284
q22	338	283	295	283
Total cold run time: 118379 ms
Total hot run time: 39809 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4319	4188	4199	4188
q2	376	263	259	259
q3	2956	2747	2698	2698
q4	1850	1601	1574	1574
q5	5304	5270	5271	5270
q6	211	120	122	120
q7	2206	1862	1833	1833
q8	3179	3299	3297	3297
q9	8353	8317	8371	8317
q10	3885	3685	3616	3616
q11	567	497	494	494
q12	730	585	596	585
q13	16389	2957	2947	2947
q14	289	266	255	255
q15	512	457	470	457
q16	451	423	420	420
q17	1785	1480	1491	1480
q18	7605	7686	7449	7449
q19	1649	1569	1603	1569
q20	1976	1781	1769	1769
q21	4838	4946	4878	4878
q22	585	496	501	496
Total cold run time: 70015 ms
Total hot run time: 53971 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186053 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 660d50d4bde2ab0e90901817b09a1161f198e702, data reload: false

query1	918	387	369	369
query2	6460	2342	2261	2261
query3	6658	202	203	202
query4	23138	21127	21040	21040
query5	4122	406	411	406
query6	260	173	167	167
query7	4596	297	285	285
query8	237	184	186	184
query9	8492	2428	2407	2407
query10	436	249	265	249
query11	14726	14114	14147	14114
query12	134	89	89	89
query13	1649	377	376	376
query14	9670	8300	6973	6973
query15	255	167	174	167
query16	8132	260	273	260
query17	1864	570	542	542
query18	2088	278	268	268
query19	218	145	149	145
query20	96	90	86	86
query21	199	124	127	124
query22	5019	4877	4939	4877
query23	34200	33515	33382	33382
query24	11945	2890	2958	2890
query25	632	363	365	363
query26	1777	162	170	162
query27	2980	325	322	322
query28	7592	2092	2051	2051
query29	1010	620	609	609
query30	310	180	170	170
query31	968	745	740	740
query32	86	50	52	50
query33	757	240	241	240
query34	1081	489	476	476
query35	853	693	685	685
query36	1080	905	952	905
query37	279	70	70	70
query38	2931	2749	2751	2749
query39	1587	1615	1556	1556
query40	282	122	122	122
query41	46	43	48	43
query42	102	96	94	94
query43	557	527	562	527
query44	1268	713	746	713
query45	269	249	254	249
query46	1061	757	711	711
query47	1967	1896	1897	1896
query48	389	299	298	298
query49	1208	399	392	392
query50	757	375	388	375
query51	6840	6789	6819	6789
query52	97	93	89	89
query53	351	282	275	275
query54	1004	432	433	432
query55	75	72	73	72
query56	240	223	222	222
query57	1240	1168	1178	1168
query58	221	204	198	198
query59	3507	3050	3048	3048
query60	260	230	238	230
query61	87	117	101	101
query62	670	470	462	462
query63	317	279	278	278
query64	9726	7359	7408	7359
query65	3110	3130	3111	3111
query66	1376	338	339	338
query67	15185	15011	14931	14931
query68	4583	537	537	537
query69	487	298	302	298
query70	1172	1136	1169	1136
query71	420	275	273	273
query72	7034	2538	2381	2381
query73	703	326	322	322
query74	6587	6066	6145	6066
query75	3419	2624	2566	2566
query76	2757	977	924	924
query77	401	265	264	264
query78	10572	10249	10189	10189
query79	2647	494	510	494
query80	1078	429	443	429
query81	520	241	246	241
query82	970	95	93	93
query83	241	164	171	164
query84	237	83	85	83
query85	1325	264	316	264
query86	467	294	281	281
query87	3280	3112	3102	3102
query88	4111	2400	2406	2400
query89	467	384	379	379
query90	1963	186	186	186
query91	124	99	96	96
query92	56	47	49	47
query93	1611	512	490	490
query94	1235	180	178	178
query95	385	303	300	300
query96	584	268	271	268
query97	3167	2991	2964	2964
query98	236	230	220	220
query99	1138	884	913	884
Total cold run time: 285624 ms
Total hot run time: 186053 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.59 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 660d50d4bde2ab0e90901817b09a1161f198e702, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.04
query4	1.66	0.08	0.08
query5	0.49	0.47	0.49
query6	1.11	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.54	0.48	0.48
query10	0.54	0.55	0.53
query11	0.15	0.11	0.11
query12	0.15	0.11	0.12
query13	0.59	0.59	0.59
query14	0.77	0.80	0.77
query15	0.83	0.80	0.81
query16	0.36	0.37	0.36
query17	0.99	0.98	1.01
query18	0.22	0.24	0.25
query19	1.75	1.68	1.67
query20	0.01	0.01	0.01
query21	15.74	0.66	0.64
query22	4.04	6.97	2.14
query23	18.28	1.42	1.29
query24	1.55	0.38	0.20
query25	0.14	0.07	0.08
query26	0.27	0.16	0.16
query27	0.08	0.08	0.08
query28	13.28	1.02	1.00
query29	13.21	3.28	3.27
query30	0.25	0.05	0.05
query31	2.87	0.38	0.38
query32	3.28	0.48	0.47
query33	2.83	2.80	2.89
query34	17.19	4.45	4.43
query35	4.46	4.58	4.53
query36	0.68	0.45	0.49
query37	0.17	0.16	0.15
query38	0.15	0.14	0.14
query39	0.05	0.03	0.03
query40	0.17	0.14	0.13
query41	0.09	0.04	0.04
query42	0.06	0.04	0.04
query43	0.04	0.04	0.03
Total cold run time: 109.45 s
Total hot run time: 30.59 s

@@ -101,6 +101,11 @@ public Plan withGroupExprLogicalPropChildren(Optional<GroupExpression> groupExpr
return new UnboundTVFRelation(relationId, functionName, properties, groupExpression, logicalProperties);
}

@Override
public UnboundTVFRelation withRelationId(RelationId relationId) {
throw new RuntimeException("should not call UnboundTVFRelation's withRelationId method");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UnboundException

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

visitLogicalDeferMaterializeOlapScan should also add to relation id map

Optional.empty(), Optional.empty(),
selectedPartitionIds, partitionPruned, selectedTabletIds,
selectedIndexId, indexSelected, preAggStatus, manuallySpecifiedPartitions,
hints, cacheSlotWithSlotName, tableSample, directMvScan, projectPulledUp);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cacheSlotWithSlotName should be removed

@@ -89,6 +89,11 @@ public Plan withGroupExprLogicalPropChildren(Optional<GroupExpression> groupExpr
return new LogicalOneRowRelation(relationId, projects, groupExpression, logicalProperties);
}

@Override
public LogicalOneRowRelation withRelationId(RelationId relationId) {
return new LogicalOneRowRelation(relationId, projects, Optional.empty(), Optional.empty());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should not impl, throw exception

@@ -81,6 +81,11 @@ public Plan withGroupExprLogicalPropChildren(Optional<GroupExpression> groupExpr
return new LogicalEmptyRelation(relationId, projects, groupExpression, logicalProperties);
}

@Override
public LogicalEmptyRelation withRelationId(RelationId relationId) {
return new LogicalEmptyRelation(relationId, projects, Optional.empty(), Optional.empty());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

throw exception

@starocean999
Copy link
Contributor Author

run buildall

@starocean999
Copy link
Contributor Author

run buildall

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40696 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1f44b1bcd8e89ae17e4ec765e455ef1bbad431dd, data reload: false

------ Round 1 ----------------------------------
q1	17952	5126	4258	4258
q2	2035	205	197	197
q3	10686	1169	1195	1169
q4	11375	805	754	754
q5	8410	2760	2715	2715
q6	220	135	135	135
q7	1020	607	588	588
q8	9494	2114	2080	2080
q9	9513	6702	6558	6558
q10	8988	3716	3723	3716
q11	456	241	238	238
q12	526	222	213	213
q13	17778	2978	2955	2955
q14	260	222	220	220
q15	528	479	490	479
q16	519	395	384	384
q17	964	630	646	630
q18	8130	7456	7388	7388
q19	1655	1547	1517	1517
q20	645	308	295	295
q21	5084	3956	3930	3930
q22	360	277	288	277
Total cold run time: 116598 ms
Total hot run time: 40696 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4318	4195	4156	4156
q2	371	273	261	261
q3	2964	2745	2761	2745
q4	1866	1601	1581	1581
q5	5279	5319	5295	5295
q6	212	124	125	124
q7	2266	1859	1881	1859
q8	3205	3325	3292	3292
q9	8350	8305	8414	8305
q10	3878	3688	3696	3688
q11	584	482	498	482
q12	759	598	591	591
q13	16467	2993	2924	2924
q14	295	268	263	263
q15	526	475	482	475
q16	474	413	421	413
q17	1777	1488	1481	1481
q18	7817	7539	7250	7250
q19	1661	1502	1530	1502
q20	1963	1778	1779	1778
q21	4950	4757	4832	4757
q22	552	497	487	487
Total cold run time: 70534 ms
Total hot run time: 53709 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187627 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1f44b1bcd8e89ae17e4ec765e455ef1bbad431dd, data reload: false

query1	935	387	381	381
query2	6466	2640	2370	2370
query3	6667	221	222	221
query4	23540	21329	21275	21275
query5	4126	424	416	416
query6	262	175	187	175
query7	4589	299	294	294
query8	238	196	185	185
query9	8621	2421	2399	2399
query10	426	249	262	249
query11	14859	14214	14283	14214
query12	133	93	87	87
query13	1646	367	365	365
query14	9854	7696	8493	7696
query15	248	172	169	169
query16	8106	260	256	256
query17	1861	558	548	548
query18	2088	274	267	267
query19	207	153	169	153
query20	95	84	86	84
query21	199	131	128	128
query22	5082	4837	4798	4798
query23	34216	33468	33451	33451
query24	12169	2970	2943	2943
query25	646	403	363	363
query26	1804	160	156	156
query27	3027	318	318	318
query28	7378	2049	2034	2034
query29	1019	649	614	614
query30	301	177	183	177
query31	989	759	738	738
query32	89	55	57	55
query33	742	253	259	253
query34	1061	490	505	490
query35	820	706	678	678
query36	1089	958	925	925
query37	285	70	73	70
query38	2886	2789	2695	2695
query39	1638	1678	1546	1546
query40	270	130	123	123
query41	47	44	44	44
query42	104	98	98	98
query43	636	554	565	554
query44	1293	729	749	729
query45	277	251	260	251
query46	1093	750	742	742
query47	1946	1881	1896	1881
query48	383	307	310	307
query49	1197	402	400	400
query50	772	394	394	394
query51	6708	6705	6621	6621
query52	106	99	89	89
query53	352	288	288	288
query54	898	433	448	433
query55	79	76	75	75
query56	249	233	240	233
query57	1241	1163	1138	1138
query58	223	217	204	204
query59	3492	3213	3259	3213
query60	256	230	239	230
query61	90	89	87	87
query62	672	475	469	469
query63	317	288	285	285
query64	9823	7476	7417	7417
query65	3153	3088	3087	3087
query66	1375	339	345	339
query67	15276	14989	15046	14989
query68	4553	541	547	541
query69	491	314	307	307
query70	1156	1158	1115	1115
query71	405	277	274	274
query72	7307	2562	2379	2379
query73	717	324	322	322
query74	6441	6103	6182	6103
query75	3505	2672	2649	2649
query76	2833	921	969	921
query77	486	279	268	268
query78	10632	10105	10171	10105
query79	2528	525	535	525
query80	1086	456	445	445
query81	530	243	246	243
query82	885	101	97	97
query83	241	168	173	168
query84	232	89	88	88
query85	1600	270	266	266
query86	515	324	307	307
query87	3291	3110	3096	3096
query88	4410	2359	2372	2359
query89	473	378	389	378
query90	1962	198	207	198
query91	137	112	111	111
query92	58	50	53	50
query93	1717	504	497	497
query94	1210	191	193	191
query95	397	302	309	302
query96	594	276	270	270
query97	3140	3008	2971	2971
query98	237	220	222	220
query99	1130	905	923	905
Total cold run time: 287634 ms
Total hot run time: 187627 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.45 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1f44b1bcd8e89ae17e4ec765e455ef1bbad431dd, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.66	0.08	0.09
query5	0.51	0.48	0.50
query6	1.12	0.72	0.72
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.54	0.49	0.48
query10	0.55	0.56	0.56
query11	0.16	0.12	0.11
query12	0.16	0.11	0.12
query13	0.59	0.60	0.60
query14	0.79	0.79	0.78
query15	0.83	0.82	0.81
query16	0.38	0.36	0.36
query17	0.95	1.00	1.00
query18	0.20	0.25	0.23
query19	1.75	1.65	1.66
query20	0.02	0.01	0.02
query21	15.72	0.65	0.65
query22	4.13	7.70	1.94
query23	18.30	1.45	1.28
query24	1.73	0.25	0.26
query25	0.14	0.08	0.07
query26	0.28	0.18	0.17
query27	0.07	0.08	0.08
query28	13.31	1.01	1.02
query29	13.18	3.29	3.26
query30	0.24	0.06	0.06
query31	2.87	0.39	0.38
query32	3.28	0.47	0.48
query33	2.81	2.85	2.79
query34	17.07	4.35	4.41
query35	4.47	4.53	4.48
query36	0.64	0.46	0.45
query37	0.17	0.16	0.15
query38	0.16	0.14	0.14
query39	0.04	0.04	0.03
query40	0.17	0.14	0.14
query41	0.09	0.05	0.05
query42	0.06	0.06	0.06
query43	0.04	0.04	0.04
Total cold run time: 109.59 s
Total hot run time: 30.45 s

@starocean999
Copy link
Contributor Author

run buildall

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels May 15, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 41971 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 69cb821f29a01290a559151b42de24a042dc1cf5, data reload: false

------ Round 1 ----------------------------------
q1	17604	4335	4277	4277
q2	2011	185	179	179
q3	10487	1287	1254	1254
q4	10193	859	835	835
q5	7469	2763	2735	2735
q6	228	133	127	127
q7	1005	548	526	526
q8	9234	2151	2113	2113
q9	9662	6772	6719	6719
q10	9172	3945	3880	3880
q11	432	253	252	252
q12	478	224	232	224
q13	17271	3143	3196	3143
q14	267	223	235	223
q15	515	476	489	476
q16	489	367	374	367
q17	982	689	684	684
q18	8365	7922	7715	7715
q19	7197	1539	1527	1527
q20	674	320	328	320
q21	5218	4110	4312	4110
q22	366	286	285	285
Total cold run time: 119319 ms
Total hot run time: 41971 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4574	4476	4455	4455
q2	375	269	265	265
q3	3180	3032	2773	2773
q4	1874	1597	1665	1597
q5	5546	5474	5498	5474
q6	222	124	123	123
q7	1760	1439	1374	1374
q8	3335	3435	3408	3408
q9	8653	8713	8683	8683
q10	3986	3702	3837	3702
q11	589	494	490	490
q12	816	644	621	621
q13	16777	3078	3159	3078
q14	311	268	273	268
q15	519	481	502	481
q16	479	428	429	428
q17	1824	1530	1479	1479
q18	7956	7532	7598	7532
q19	1739	1569	1607	1569
q20	1999	1774	1737	1737
q21	6730	4817	4864	4817
q22	565	484	505	484
Total cold run time: 73809 ms
Total hot run time: 54838 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186308 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 69cb821f29a01290a559151b42de24a042dc1cf5, data reload: false

query1	918	388	370	370
query2	7643	2438	2327	2327
query3	6639	223	218	218
query4	23682	21439	21266	21266
query5	4125	420	415	415
query6	265	170	200	170
query7	4606	309	285	285
query8	238	199	193	193
query9	8568	2397	2394	2394
query10	439	257	244	244
query11	14962	14132	14182	14132
query12	134	92	91	91
query13	1659	388	369	369
query14	10406	7648	6991	6991
query15	213	166	172	166
query16	7743	261	249	249
query17	1736	547	531	531
query18	1961	272	267	267
query19	197	146	147	146
query20	95	86	82	82
query21	190	131	126	126
query22	5037	4920	4904	4904
query23	34297	33631	33424	33424
query24	6670	2946	2890	2890
query25	539	372	362	362
query26	703	156	149	149
query27	1941	315	323	315
query28	3869	2060	2042	2042
query29	854	605	596	596
query30	261	173	174	173
query31	970	729	756	729
query32	86	53	54	53
query33	505	244	241	241
query34	855	487	471	471
query35	768	669	671	669
query36	1052	911	951	911
query37	104	72	69	69
query38	2896	2762	2795	2762
query39	1605	1569	1549	1549
query40	189	128	123	123
query41	46	43	45	43
query42	102	98	96	96
query43	591	566	516	516
query44	1099	718	743	718
query45	268	247	246	246
query46	1083	712	677	677
query47	1986	1900	1867	1867
query48	379	302	291	291
query49	774	386	399	386
query50	770	378	386	378
query51	6945	6747	6732	6732
query52	110	90	85	85
query53	353	281	282	281
query54	516	443	429	429
query55	83	75	79	75
query56	237	221	214	214
query57	1217	1143	1143	1143
query58	206	195	206	195
query59	3578	3183	3244	3183
query60	253	231	240	231
query61	91	87	88	87
query62	580	487	481	481
query63	304	276	281	276
query64	8357	7453	7322	7322
query65	3130	3077	3096	3077
query66	763	337	345	337
query67	15650	15057	14927	14927
query68	4605	522	528	522
query69	480	304	306	304
query70	1144	1136	1182	1136
query71	382	269	266	266
query72	7317	2528	2390	2390
query73	704	318	315	315
query74	6568	6180	6145	6145
query75	3279	2695	2560	2560
query76	2285	984	980	980
query77	414	261	262	261
query78	10537	10236	10146	10146
query79	1570	513	510	510
query80	988	429	428	428
query81	531	244	241	241
query82	966	96	99	96
query83	244	168	160	160
query84	248	86	82	82
query85	996	278	356	278
query86	414	310	301	301
query87	3307	3100	3101	3100
query88	2984	2340	2352	2340
query89	472	372	378	372
query90	1943	184	188	184
query91	124	101	97	97
query92	58	48	49	48
query93	1284	494	489	489
query94	1046	180	176	176
query95	382	299	295	295
query96	594	273	265	265
query97	3141	2955	2954	2954
query98	245	226	212	212
query99	1204	898	915	898
Total cold run time: 269547 ms
Total hot run time: 186308 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 69cb821f29a01290a559151b42de24a042dc1cf5, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.66	0.07	0.08
query5	0.51	0.50	0.52
query6	1.12	0.72	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.53	0.51	0.49
query10	0.54	0.55	0.53
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.62
query14	0.76	0.77	0.78
query15	0.82	0.80	0.81
query16	0.38	0.37	0.36
query17	1.02	0.99	0.95
query18	0.21	0.25	0.25
query19	1.75	1.71	1.70
query20	0.01	0.02	0.01
query21	15.75	0.67	0.66
query22	4.15	7.49	2.27
query23	18.27	1.46	1.34
query24	1.71	0.29	0.21
query25	0.14	0.09	0.08
query26	0.27	0.17	0.17
query27	0.08	0.08	0.09
query28	13.34	1.02	1.00
query29	13.79	3.27	3.25
query30	0.23	0.05	0.05
query31	2.87	0.38	0.39
query32	3.28	0.47	0.46
query33	2.82	2.86	2.85
query34	17.17	4.39	4.38
query35	4.46	4.52	4.62
query36	0.64	0.46	0.46
query37	0.17	0.16	0.15
query38	0.16	0.14	0.14
query39	0.04	0.03	0.03
query40	0.15	0.14	0.14
query41	0.09	0.05	0.05
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.29 s
Total hot run time: 30.91 s

@doris-robot
Copy link

TPC-DS: Total hot run time: 187104 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 69cb821f29a01290a559151b42de24a042dc1cf5, data reload: false

query1	932	386	370	370
query2	6452	2329	2325	2325
query3	6656	226	220	220
query4	22994	21142	21184	21142
query5	4138	414	428	414
query6	257	174	190	174
query7	4586	308	285	285
query8	237	194	184	184
query9	8735	2362	2357	2357
query10	453	255	266	255
query11	14772	14250	14125	14125
query12	131	92	91	91
query13	1793	401	362	362
query14	10665	8070	8470	8070
query15	228	167	170	167
query16	7950	249	259	249
query17	1871	549	560	549
query18	2014	279	256	256
query19	205	149	147	147
query20	93	83	85	83
query21	199	130	131	130
query22	4993	4858	4885	4858
query23	34223	33440	33431	33431
query24	11005	2799	2922	2799
query25	612	367	369	367
query26	1174	157	153	153
query27	3011	324	331	324
query28	7515	2025	2014	2014
query29	876	633	611	611
query30	309	175	174	174
query31	993	761	749	749
query32	93	53	56	53
query33	769	251	251	251
query34	1052	486	482	482
query35	799	670	669	669
query36	1064	951	899	899
query37	138	70	72	70
query38	2875	2784	2773	2773
query39	1620	1559	1544	1544
query40	199	123	125	123
query41	44	42	42	42
query42	101	96	99	96
query43	558	564	566	564
query44	1210	733	746	733
query45	262	253	256	253
query46	1082	701	701	701
query47	1925	1839	1915	1839
query48	377	301	292	292
query49	1083	387	399	387
query50	792	377	384	377
query51	7024	6848	6755	6755
query52	109	99	121	99
query53	348	287	291	287
query54	853	430	415	415
query55	75	75	72	72
query56	240	221	218	218
query57	1225	1131	1136	1131
query58	217	200	191	191
query59	3428	3049	3168	3049
query60	249	231	235	231
query61	102	88	88	88
query62	681	478	465	465
query63	311	284	288	284
query64	8887	7340	7363	7340
query65	3131	3104	3086	3086
query66	1401	344	334	334
query67	15447	15158	14948	14948
query68	4546	525	524	524
query69	535	305	300	300
query70	1125	1061	1070	1061
query71	401	268	266	266
query72	7919	2574	2363	2363
query73	689	317	320	317
query74	6545	6134	6043	6043
query75	3323	2617	2628	2617
query76	2497	1012	1002	1002
query77	422	269	267	267
query78	10549	10218	10171	10171
query79	1671	518	507	507
query80	1028	431	445	431
query81	534	245	244	244
query82	971	97	98	97
query83	239	166	165	165
query84	241	86	83	83
query85	1338	327	315	315
query86	388	298	311	298
query87	3326	3111	3073	3073
query88	2888	2323	2346	2323
query89	486	384	390	384
query90	1910	191	195	191
query91	139	108	112	108
query92	61	51	53	51
query93	1265	504	502	502
query94	1227	185	190	185
query95	400	303	310	303
query96	570	271	265	265
query97	3151	2988	3028	2988
query98	235	225	217	217
query99	1124	867	909	867
Total cold run time: 281586 ms
Total hot run time: 187104 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.56 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 69cb821f29a01290a559151b42de24a042dc1cf5, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.08	0.09
query5	0.52	0.50	0.52
query6	1.13	0.72	0.71
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.53	0.49	0.50
query10	0.54	0.56	0.53
query11	0.15	0.11	0.11
query12	0.15	0.12	0.13
query13	0.61	0.59	0.60
query14	0.79	0.77	0.78
query15	0.82	0.82	0.80
query16	0.37	0.36	0.36
query17	0.96	1.05	0.98
query18	0.22	0.26	0.25
query19	1.88	1.76	1.77
query20	0.02	0.01	0.01
query21	15.70	0.64	0.64
query22	3.91	7.27	2.03
query23	18.32	1.31	1.28
query24	1.37	0.40	0.19
query25	0.16	0.09	0.07
query26	0.26	0.17	0.17
query27	0.09	0.08	0.08
query28	13.36	1.01	1.00
query29	13.09	3.30	3.27
query30	0.24	0.06	0.06
query31	2.86	0.39	0.39
query32	3.27	0.49	0.47
query33	2.82	2.84	2.84
query34	17.06	4.37	4.44
query35	4.52	4.45	4.46
query36	0.66	0.46	0.46
query37	0.17	0.16	0.15
query38	0.15	0.14	0.15
query39	0.04	0.04	0.03
query40	0.16	0.13	0.13
query41	0.09	0.05	0.05
query42	0.05	0.05	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.17 s
Total hot run time: 30.56 s

@starocean999
Copy link
Contributor Author

run performance

@starocean999 starocean999 merged commit 41c0e85 into apache:master May 16, 2024
26 of 28 checks passed
dataroaring pushed a commit that referenced this pull request May 16, 2024
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 20, 2024
intro by PR apache#34894
This PR attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the outer structure.
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 20, 2024
intro by PR apache#34894
This PR attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the outer structure.
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 20, 2024
intro by PR apache#34894
This PR attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the outer structure.
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 20, 2024
intro by PR apache#34894
This PR attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the outer structure.
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 20, 2024
intro by PR apache#34894
This PR attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the outer structure.
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 20, 2024
intro by PR apache#34894
This PR attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the outer structure.
morrySnow added a commit that referenced this pull request May 21, 2024
…#35076)

intro by PR #34894
PR #34894 attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the scan.
M1saka2003 pushed a commit to M1saka2003/doris that referenced this pull request May 24, 2024
M1saka2003 pushed a commit to M1saka2003/doris that referenced this pull request May 24, 2024
…apache#35076)

intro by PR apache#34894
PR apache#34894 attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the scan.
dataroaring pushed a commit that referenced this pull request May 26, 2024
…#35076)

intro by PR #34894
PR #34894 attempts to address the issue of losing conjuncts
when performing a deep copy of the outer structure.
However, the timing of copying the conjuncts is incorrect,
resulting in the inability to map slots within the conjuncts
to the output of the scan.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.3-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants