Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more test cases on vector size #4240

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Add more test cases on vector size #4240

wants to merge 2 commits into from

Conversation

ray6080
Copy link
Contributor

@ray6080 ray6080 commented Sep 11, 2024

Description

Copy link

Benchmark Result

Master commit hash: 7593ec9813792d26679540426302b609adb7efa1
Branch commit hash: c69b58fc8b2e2b26e42c878bb4c93d7f9c118a36

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 679.42 681.49 -2.06 (-0.30%)
aggregation q28 11962.98 11614.74 348.25 (3.00%)
filter q14 162.56 167.34 -4.78 (-2.86%)
filter q15 159.32 162.62 -3.30 (-2.03%)
filter q16 333.73 343.06 -9.33 (-2.72%)
filter q17 481.00 486.55 -5.55 (-1.14%)
filter q18 1930.70 1934.12 -3.42 (-0.18%)
fixed_size_expr_evaluator q07 581.09 569.27 11.83 (2.08%)
fixed_size_expr_evaluator q08 786.26 782.94 3.32 (0.42%)
fixed_size_expr_evaluator q09 796.10 781.78 14.33 (1.83%)
fixed_size_expr_evaluator q10 277.45 271.40 6.05 (2.23%)
fixed_size_expr_evaluator q11 272.18 267.36 4.81 (1.80%)
fixed_size_expr_evaluator q12 271.46 264.72 6.74 (2.55%)
fixed_size_expr_evaluator q13 1509.90 1510.36 -0.46 (-0.03%)
fixed_size_seq_scan q23 148.21 149.13 -0.92 (-0.62%)
join q31 11.22 6.45 4.77 (74.04%)
ldbc_snb_ic q35 766.17 768.23 -2.05 (-0.27%)
ldbc_snb_ic q36 46.30 45.93 0.36 (0.79%)
ldbc_snb_is q32 8.28 8.43 -0.16 (-1.85%)
ldbc_snb_is q33 15.39 17.91 -2.52 (-14.08%)
ldbc_snb_is q34 7.76 8.80 -1.03 (-11.76%)
multi-rel multi-rel-large-scan 2761.37 2801.42 -40.05 (-1.43%)
multi-rel multi-rel-lookup 56.52 69.61 -13.09 (-18.80%)
multi-rel multi-rel-small-scan 50.98 57.64 -6.66 (-11.55%)
order_by q25 165.26 164.81 0.45 (0.27%)
order_by q26 497.47 490.87 6.60 (1.35%)
order_by q27 1470.12 1444.58 25.54 (1.77%)
scan_after_filter q01 204.30 205.03 -0.73 (-0.35%)
scan_after_filter q02 194.61 194.35 0.26 (0.13%)
shortest_path_ldbc100 q37 3958.90 3462.77 496.13 (14.33%)
shortest_path_ldbc100 q39 112.23 109.46 2.78 (2.54%)
shortest_path_ldbc100 q40 82.43 75.18 7.25 (9.64%)
var_size_expr_evaluator q03 2138.48 2102.26 36.22 (1.72%)
var_size_expr_evaluator q04 2307.49 2267.97 39.51 (1.74%)
var_size_expr_evaluator q05 2536.70 2592.47 -55.77 (-2.15%)
var_size_expr_evaluator q06 1377.92 1422.56 -44.64 (-3.14%)
var_size_seq_scan q19 1512.02 1515.91 -3.89 (-0.26%)
var_size_seq_scan q20 3258.36 3218.01 40.35 (1.25%)
var_size_seq_scan q21 2488.38 2521.98 -33.60 (-1.33%)
var_size_seq_scan q22 136.78 136.95 -0.16 (-0.12%)

Copy link

codecov bot commented Sep 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.65%. Comparing base (7593ec9) to head (70dda3d).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #4240   +/-   ##
=======================================
  Coverage   86.65%   86.65%           
=======================================
  Files        1327     1327           
  Lines       51365    51365           
  Branches     6895     6895           
=======================================
+ Hits        44511    44512    +1     
+ Misses       6682     6681    -1     
  Partials      172      172           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

Benchmark Result

Master commit hash: 7593ec9813792d26679540426302b609adb7efa1
Branch commit hash: aac4ea1d15f32aef8e9d27598e12d680e3820a6f

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 679.07 681.49 -2.42 (-0.35%)
aggregation q28 11950.08 11614.74 335.34 (2.89%)
filter q14 158.49 167.34 -8.84 (-5.29%)
filter q15 157.80 162.62 -4.82 (-2.96%)
filter q16 334.51 343.06 -8.55 (-2.49%)
filter q17 479.58 486.55 -6.97 (-1.43%)
filter q18 1936.69 1934.12 2.56 (0.13%)
fixed_size_expr_evaluator q07 569.30 569.27 0.03 (0.01%)
fixed_size_expr_evaluator q08 779.75 782.94 -3.19 (-0.41%)
fixed_size_expr_evaluator q09 781.84 781.78 0.06 (0.01%)
fixed_size_expr_evaluator q10 271.09 271.40 -0.31 (-0.11%)
fixed_size_expr_evaluator q11 266.43 267.36 -0.93 (-0.35%)
fixed_size_expr_evaluator q12 264.75 264.72 0.03 (0.01%)
fixed_size_expr_evaluator q13 1505.49 1510.36 -4.87 (-0.32%)
fixed_size_seq_scan q23 150.67 149.13 1.54 (1.03%)
join q31 12.38 6.45 5.93 (92.03%)
ldbc_snb_ic q35 719.04 768.23 -49.19 (-6.40%)
ldbc_snb_ic q36 46.55 45.93 0.62 (1.34%)
ldbc_snb_is q32 8.40 8.43 -0.03 (-0.36%)
ldbc_snb_is q33 16.46 17.91 -1.46 (-8.12%)
ldbc_snb_is q34 8.15 8.80 -0.64 (-7.30%)
multi-rel multi-rel-large-scan 2761.02 2801.42 -40.40 (-1.44%)
multi-rel multi-rel-lookup 46.78 69.61 -22.83 (-32.80%)
multi-rel multi-rel-small-scan 54.16 57.64 -3.48 (-6.04%)
order_by q25 163.16 164.81 -1.65 (-1.00%)
order_by q26 493.12 490.87 2.25 (0.46%)
order_by q27 1437.14 1444.58 -7.44 (-0.52%)
scan_after_filter q01 204.47 205.03 -0.56 (-0.27%)
scan_after_filter q02 193.05 194.35 -1.31 (-0.67%)
shortest_path_ldbc100 q37 3410.23 3462.77 -52.54 (-1.52%)
shortest_path_ldbc100 q39 132.04 109.46 22.58 (20.63%)
shortest_path_ldbc100 q40 73.80 75.18 -1.38 (-1.84%)
var_size_expr_evaluator q03 2104.45 2102.26 2.19 (0.10%)
var_size_expr_evaluator q04 2258.92 2267.97 -9.06 (-0.40%)
var_size_expr_evaluator q05 2578.63 2592.47 -13.85 (-0.53%)
var_size_expr_evaluator q06 1420.36 1422.56 -2.19 (-0.15%)
var_size_seq_scan q19 1519.48 1515.91 3.57 (0.24%)
var_size_seq_scan q20 3232.32 3218.01 14.31 (0.44%)
var_size_seq_scan q21 2484.10 2521.98 -37.88 (-1.50%)
var_size_seq_scan q22 137.77 136.95 0.82 (0.60%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant