Skip to content

[fix](cloud) compaction and schema change potential data race when retrying prepare rowset (#51048) #52075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: branch-3.1
Choose a base branch
from

Conversation

luwei16
Copy link
Contributor

@luwei16 luwei16 commented Jun 20, 2025

related PR #51129

pick master #51048

@luwei16 luwei16 requested a review from morrySnow as a code owner June 20, 2025 09:41
@Thearas
Copy link
Contributor

Thearas commented Jun 20, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@luwei16
Copy link
Contributor Author

luwei16 commented Jun 20, 2025

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 90.00% (54/60) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.21% (1100/1322)
Line Coverage 66.48% (18634/28029)
Region Coverage 66.30% (9230/13921)
Branch Coverage 56.14% (5004/8914)

@doris-robot
Copy link

TPC-H: Total hot run time: 40064 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eae4d1f56f31fd5845e0bed25032c7c9f9117596, data reload: false

------ Round 1 ----------------------------------
q1	17659	6875	6633	6633
q2	2068	170	184	170
q3	10630	1073	1188	1073
q4	10499	737	828	737
q5	7739	2908	2890	2890
q6	216	136	132	132
q7	969	637	631	631
q8	9352	1964	2026	1964
q9	6603	6416	6475	6416
q10	6952	2222	2287	2222
q11	469	263	276	263
q12	393	216	210	210
q13	17785	2972	2986	2972
q14	233	221	207	207
q15	498	479	474	474
q16	500	384	379	379
q17	990	618	555	555
q18	7502	6717	6713	6713
q19	1433	1116	1002	1002
q20	484	202	199	199
q21	3960	3245	3236	3236
q22	1078	1024	986	986
Total cold run time: 108012 ms
Total hot run time: 40064 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6662	6624	6605	6605
q2	321	233	236	233
q3	2869	2780	2939	2780
q4	2040	1802	1852	1802
q5	5766	5761	5761	5761
q6	207	127	127	127
q7	2170	1794	1765	1765
q8	3357	3598	3533	3533
q9	8918	8832	8928	8832
q10	3529	3555	3540	3540
q11	602	490	492	490
q12	825	622	617	617
q13	8219	3170	3173	3170
q14	295	272	278	272
q15	516	478	458	458
q16	508	454	449	449
q17	1843	1624	1617	1617
q18	8273	7743	7776	7743
q19	1696	1666	1579	1579
q20	2121	1859	1823	1823
q21	5073	4986	5037	4986
q22	1145	1057	1032	1032
Total cold run time: 66955 ms
Total hot run time: 59214 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196758 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eae4d1f56f31fd5845e0bed25032c7c9f9117596, data reload: false

query1	1279	918	894	894
query2	6318	1848	1857	1848
query3	10784	4256	4332	4256
query4	61141	29701	23482	23482
query5	5295	461	434	434
query6	401	172	172	172
query7	5515	305	308	305
query8	310	220	218	218
query9	8754	2583	2580	2580
query10	475	271	255	255
query11	17645	15645	15782	15645
query12	155	108	102	102
query13	1462	439	432	432
query14	10013	7523	6738	6738
query15	208	179	184	179
query16	7159	480	471	471
query17	1225	619	568	568
query18	1895	318	308	308
query19	221	170	158	158
query20	136	107	108	107
query21	207	107	105	105
query22	4629	4605	4735	4605
query23	34101	33937	34000	33937
query24	6178	2945	2934	2934
query25	510	389	410	389
query26	651	164	164	164
query27	1710	358	353	353
query28	3972	2163	2125	2125
query29	672	457	444	444
query30	237	159	150	150
query31	984	816	823	816
query32	64	52	56	52
query33	414	296	301	296
query34	914	511	517	511
query35	843	724	727	724
query36	1082	974	960	960
query37	109	68	72	68
query38	4054	3977	3911	3911
query39	1513	1457	1477	1457
query40	197	101	100	100
query41	51	46	48	46
query42	113	102	99	99
query43	516	463	473	463
query44	1183	815	833	815
query45	182	169	172	169
query46	1178	785	742	742
query47	2012	1919	1913	1913
query48	482	390	387	387
query49	721	394	389	389
query50	845	437	424	424
query51	7399	7302	7411	7302
query52	104	94	92	92
query53	261	195	189	189
query54	583	460	459	459
query55	81	85	81	81
query56	282	251	260	251
query57	1309	1249	1233	1233
query58	219	210	212	210
query59	3128	2970	2969	2969
query60	295	257	258	257
query61	111	115	113	113
query62	769	688	696	688
query63	229	189	193	189
query64	1397	654	628	628
query65	3247	3181	3166	3166
query66	707	323	294	294
query67	15963	15630	15617	15617
query68	4148	600	578	578
query69	427	261	265	261
query70	1184	1100	1122	1100
query71	350	255	255	255
query72	6575	4059	4124	4059
query73	752	362	355	355
query74	9998	8871	9260	8871
query75	3329	2670	2637	2637
query76	2010	1047	1088	1047
query77	482	276	267	267
query78	10532	9644	9444	9444
query79	2141	602	601	601
query80	1396	415	432	415
query81	528	221	215	215
query82	1185	88	87	87
query83	271	151	146	146
query84	282	86	81	81
query85	1053	300	288	288
query86	388	282	296	282
query87	4394	4322	4284	4284
query88	3903	2422	2382	2382
query89	415	291	292	291
query90	1997	185	186	185
query91	179	146	145	145
query92	67	50	49	49
query93	2802	558	552	552
query94	817	292	297	292
query95	361	254	259	254
query96	628	286	282	282
query97	3275	3129	3139	3129
query98	219	206	196	196
query99	1582	1302	1290	1290
Total cold run time: 314469 ms
Total hot run time: 196758 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.65 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eae4d1f56f31fd5845e0bed25032c7c9f9117596, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.08	0.08
query5	0.51	0.51	0.51
query6	1.13	0.75	0.76
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.55	0.48	0.50
query10	0.56	0.54	0.55
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.60	0.59	0.60
query14	0.80	0.79	0.85
query15	0.85	0.85	0.84
query16	0.38	0.38	0.37
query17	1.07	1.06	1.07
query18	0.18	0.18	0.18
query19	1.90	1.85	1.80
query20	0.02	0.02	0.01
query21	15.39	0.65	0.65
query22	4.21	6.54	2.68
query23	18.30	1.44	1.34
query24	2.21	0.23	0.23
query25	0.15	0.08	0.09
query26	0.27	0.18	0.18
query27	0.08	0.09	0.08
query28	13.25	0.61	0.56
query29	12.66	3.40	3.35
query30	0.23	0.06	0.05
query31	2.88	0.39	0.39
query32	3.23	0.48	0.47
query33	2.99	3.02	3.03
query34	16.82	4.52	4.50
query35	4.60	4.55	4.58
query36	0.67	0.47	0.49
query37	0.19	0.17	0.17
query38	0.16	0.15	0.16
query39	0.05	0.04	0.04
query40	0.16	0.14	0.12
query41	0.10	0.05	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.63 s
Total hot run time: 31.65 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 0.00% (0/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 41.16% (10902/26490)
Line Coverage 31.96% (93279/291840)
Region Coverage 31.03% (48099/154988)
Branch Coverage 27.51% (24643/89586)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants