Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration of perftest CI with standardised benchmarks #6514

Draft
wants to merge 75 commits into
base: master
Choose a base branch
from

Conversation

vrnimje
Copy link
Contributor

@vrnimje vrnimje commented Jun 21, 2024

Proposed Changes

  • Improved the perftest CI with CDash integration
  • Added separate cmake function for adding performance tests with specific options for benchmarking purposes, which also creates a custom target for generating the plots

Any background context you want to provide?

Extension to #6448
Sample run on CDash: https://my.cdash.org/builds/2589647

@jenkins-cscs
Copy link
Collaborator

Can one of the admins verify this patch?

Copy link

codacy-production bot commented Jun 21, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-0.06% 81.75%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (17bbc63) 222489 189245 85.06%
Head commit (6072f1b) 196545 (-25944) 167059 (-22186) 85.00% (-0.06%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#6514) 263 215 81.75%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences


🚀 Don’t miss a bit, follow what’s new on Codacy.

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

@hkaiser hkaiser added type: enhancement category: CI category: perftests Python performance tests regressions labels Jun 23, 2024
@vrnimje
Copy link
Contributor Author

vrnimje commented Jul 11, 2024

Performance test report

small_vector_benchmark

  1. hpx::small_vector, <fxn<void()>, 2>: Performance is better by 0.46 % (KS-stat: 0.9583333333333334)

stream_report

  1. Stream benchmark - Scale, fork_join_executor_default_fork_join_policy_allocator: Performance is better by 12.69 % (KS-stat: 0.9583333333333334)
  2. Stream benchmark - Triad, fork_join_executor_default_fork_join_policy_allocator: Performance is better by 3.25 % (KS-stat: 0.875)
  3. Stream benchmark - Scale, scheduler_executor_default_scheduler_executor_allocator: Performance is better by 18.40 % (KS-stat: 1.0)
  4. Stream benchmark - Add, scheduler_executor_default_scheduler_executor_allocator: Performance is better by 10.84 % (KS-stat: 0.7916666666666666)

foreach_report

  1. for_each, fork_join_executor: Performance is better by 6.67 % (KS-stat: 0.9166666666666666)

transform_reduce_scaling

  1. transform_reduce, par: Performance is better by 57.55 % (KS-stat: 1.0)
  2. transform_reduce_old, par: Performance is better by 69.66 % (KS-stat: 1.0)

benchmark_is_heap_until

  1. hpx::heap_until, seq: Performance is better by 35.65 % (KS-stat: 1.0)
  2. hpx::heap_until, par: Performance is better by 32.25 % (KS-stat: 0.9583333333333334)

benchmark_inplace_merge

  1. hpx::inplace_merge, seq: Performance is better by 41.21 % (KS-stat: 0.9166666666666666)
  2. hpx::inplace_merge, par_unseq: Performance is better by 49.60 % (KS-stat: 1.0)

benchmark_is_heap

  1. hpx::is_heap, seq: Performance is better by 100.00 % (KS-stat: 1.0)

benchmark_remove

  1. hpx::remove, seq: Performance is better by 21.37 % (KS-stat: 1.0)
  2. hpx::remove, par: Performance is better by 30.38 % (KS-stat: 0.9583333333333334)
  3. hpx::remove, par_unseq: Performance is better by 31.96 % (KS-stat: 0.9166666666666666)

benchmark_remove_if

  1. hpx::remove_if, seq: Performance is better by 32.02 % (KS-stat: 1.0)
  2. hpx::remove_if, par: Performance is better by 38.20 % (KS-stat: 1.0)
  3. hpx::remove_if, par_unseq: Performance is better by 37.82 % (KS-stat: 1.0)

minmax_element_performance

  1. hpx::min, par: Performance is better by 19.53 % (KS-stat: 0.9166666666666666)

small_vector_benchmark

  1. hpx::small_vector, <fxn<void()>, 1>: Performance is worse by 9.37 % (KS-stat: 0.7916666666666666)
  2. hpx::small_vector, <fxn<void()>, 8>: Performance is better by 1.68 % (KS-stat: 0.9166666666666666)

stream_report

  1. stream benchmark - Copy, fork_join_executor_default_fork_join_policy_allocator: Performance is better by 6.28 % (KS-stat: 1.0)
  2. Stream benchmark - Scale, fork_join_executor_default_fork_join_policy_allocator: Performance is better by 13.83 % (KS-stat: 1.0)
  3. Stream benchmark - Add, fork_join_executor_default_fork_join_policy_allocator: Performance is better by 4.41 % (KS-stat: 0.7916666666666666)
  4. Stream benchmark - Triad, fork_join_executor_default_fork_join_policy_allocator: Performance is better by 2.72 % (KS-stat: 0.75)
  5. Stream benchmark - Scale, scheduler_executor_default_scheduler_executor_allocator: Performance is better by 18.22 % (KS-stat: 0.9583333333333334)

foreach_report

  1. for_each, fork_join_executor: Performance is better by 6.40 % (KS-stat: 0.9166666666666666)

transform_reduce_scaling

  1. transform_reduce, par: Performance is better by 57.40 % (KS-stat: 1.0)
  2. transform_reduce_old, par: Performance is better by 69.81 % (KS-stat: 1.0)

benchmark_is_heap_until

  1. hpx::heap_until, seq: Performance is better by 36.00 % (KS-stat: 1.0)
  2. hpx::heap_until, par: Performance is better by 32.02 % (KS-stat: 0.9583333333333334)

benchmark_inplace_merge

  1. hpx::inplace_merge, seq: Performance is better by 41.25 % (KS-stat: 0.9583333333333334)
  2. hpx::inplace_merge, par_unseq: Performance is better by 49.76 % (KS-stat: 1.0)

benchmark_is_heap

  1. hpx::is_heap, seq: Performance is better by 100.00 % (KS-stat: 1.0)

benchmark_remove

  1. hpx::remove, seq: Performance is better by 21.60 % (KS-stat: 0.9583333333333334)
  2. hpx::remove, par: Performance is better by 31.05 % (KS-stat: 1.0)
  3. hpx::remove, par_unseq: Performance is better by 33.81 % (KS-stat: 1.0)

benchmark_remove_if

  1. hpx::remove_if, seq: Performance is better by 29.03 % (KS-stat: 1.0)
  2. hpx::remove_if, par: Performance is better by 37.92 % (KS-stat: 1.0)
  3. hpx::remove_if, par_unseq: Performance is better by 37.52 % (KS-stat: 1.0)

small_vector_benchmark

  1. hpx::small_vector, <int, 1>: Performance is worse by 2.90 % (KS-stat: 0.7916666666666666)
  2. hpx::small_vector, <fxn<void()>, 1>: Performance is better by 8.86 % (KS-stat: 0.7916666666666666)

stream_report

  1. stream benchmark - Copy, fork_join_executor_default_fork_join_policy_allocator: Performance is better by 1.72 % (KS-stat: 0.7916666666666666)

minmax_element_performance

  1. hpx::min, par: Performance is worse by 89.74 % (KS-stat: 0.75)

stream_report

  1. stream benchmark - Copy, fork_join_executor_default_fork_join_policy_allocator: Performance is worse by 3.32 % (KS-stat: 1.0)
  2. Stream benchmark - Scale, fork_join_executor_default_fork_join_policy_allocator: Performance is worse by 21.33 % (KS-stat: 0.8333333333333334)
  3. Stream benchmark - Add, fork_join_executor_default_fork_join_policy_allocator: Performance is worse by 16.76 % (KS-stat: 0.7916666666666666)
  4. Stream benchmark - Triad, fork_join_executor_default_fork_join_policy_allocator: Performance is worse by 3.95 % (KS-stat: 0.75)
  5. Stream benchmark - Add, scheduler_executor_default_scheduler_executor_allocator: Performance is better by 15.89 % (KS-stat: 0.9166666666666666)

benchmark_remove_if

  1. hpx::remove_if, seq: Performance is better by 4.40 % (KS-stat: 0.75)

@Pansysk75
Copy link
Member

@vrnimje Good job! ;)

Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
…minor changes to plotting script

* Further simplified the default template for the benchmark

Signed-off-by: Vedant <[email protected]>
…, and made minor changes to perftests ci

Signed-off-by: Vedant <[email protected]>
…elines, and fixed clang- and cmake-formatting issues

Signed-off-by: Vedant <[email protected]>
@StellarBot
Copy link

Performance test report
Test nameExecutorMean Percentage Diff.Coefficient of variation
minmax_element_performance
hpx::minmaxpar1.96 %1.18651
hpx::minpar-0.11 %15.25031
hpx::maxpar0.05 %42.50429
small_vector_benchmark
hpx::small_vector<int, 8>0.52 %0.47845
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor0.20 %14.95532
stream_report
Stream benchmark - Addscheduler_executor_default_scheduler_executor_allocator1.57 %0.38104
transform_reduce_scaling
transform_reducepar-0.05 %15.06714
benchmark_is_heap_until
hpx::heap_untilpar-0.28 %6.53407
benchmark_merge
hpx::mergeseq1.25 %0.04249
hpx::mergepar-5.67 %0.12528
hpx::mergepar_unseq-1.26 %0.45807
benchmark_inplace_merge
hpx::inplace_mergepar10.49 %0.15894
hpx::inplace_mergepar_unseq11.33 %0.12265
benchmark_is_heap
hpx::is_heappar0.41 %7.33730
benchmark_remove_if
hpx::remove_ifseq-0.90 %0.11926
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq5.45 %0.02869
hpx::partial_sort, size: 100000seq-1.46 %0.09224
benchmark_partial_sort_parallel
hpx::partial_sort, size: 100000par9.37 %0.02204
hpx::partial_sort, size: 100000, step: 1000par9.83 %0.05209
benchmark_nth_element_parallel
hpx::nth_element, size: 1000, step: 1par-9.17 %0.98629

…n CDash, and added them to the build artifacts

Signed-off-by: Vedant <[email protected]>
@StellarBot
Copy link

Performance test report
Test nameExecutorMean Percentage Diff.Coefficient of variation
minmax_element_performance
hpx::minmaxpar-1.19 %1.76652
hpx::minpar-0.42 %4.10981
hpx::maxpar-0.03 %21.59520
small_vector_benchmark
hpx::small_vector<fxn<void()>, 1>-0.70 %0.25417
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor-12.63 %0.30442
stream_report
stream benchmark - Copyparallel_executor_default_parallel_policy_allocator-4.30 %0.19204
Stream benchmark - Scaleparallel_executor_default_parallel_policy_allocator-2.52 %0.44144
Stream benchmark - Scalefork_join_executor_default_fork_join_policy_allocator1.32 %0.03324
stream benchmark - Copyscheduler_executor_default_scheduler_executor_allocator-2.47 %0.45755
benchmark_is_heap_until
hpx::heap_untilpar0.00 %545.61520
benchmark_merge
hpx::mergeseq1.42 %0.04377
hpx::mergepar-3.80 %0.16191
hpx::mergepar_unseq-1.84 %0.34002
benchmark_inplace_merge
hpx::inplace_mergeseq-1.15 %0.06430
hpx::inplace_mergepar9.01 %0.23894
hpx::inplace_mergepar_unseq10.28 %0.12878
benchmark_is_heap
hpx::is_heappar-1.09 %2.25798
benchmark_remove_if
hpx::remove_ifseq0.90 %0.12822
hpx::remove_ifpar1.14 %0.07750
hpx::remove_ifpar_unseq1.05 %0.26647
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq11.04 %0.01236
hpx::partial_sort, size: 100000seq-1.33 %0.11409
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par1.46 %0.10163
hpx::partial_sort, size: 100000, step: 1000par11.22 %0.04772
benchmark_nth_element_parallel
hpx::nth_element, size: 100000, step: 5000par-0.83 %1.46625

@StellarBot
Copy link

Performance test report
Test nameExecutorMean Percentage Diff.Coefficient of variation
minmax_element_performance
hpx::minmaxpar1.44 %1.37947
hpx::minpar0.61 %2.33319
hpx::maxpar-1.19 %1.02057
small_vector_benchmark
hpx::small_vector<int, 8>0.53 %0.48545
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor-0.57 %6.01747
stream_report
Stream benchmark - Scaleparallel_executor_default_parallel_policy_allocator0.10 %15.05362
Stream benchmark - Addparallel_executor_default_parallel_policy_allocator0.99 %0.49465
Stream benchmark - Scalescheduler_executor_default_scheduler_executor_allocator-3.21 %0.41208
foreach_report
for_eachscheduler_executor1.68 %0.45645
transform_reduce_scaling
transform_reducepar-0.35 %3.86105
benchmark_is_heap_until
hpx::heap_untilpar0.24 %8.86754
benchmark_merge
hpx::mergeseq1.31 %0.04086
hpx::mergepar-4.19 %0.16176
benchmark_inplace_merge
hpx::inplace_mergepar3.85 %0.38700
hpx::inplace_mergepar_unseq2.51 %0.30816
benchmark_is_heap
hpx::is_heappar0.46 %6.13407
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq2.12 %0.08383
hpx::partial_sort, size: 100000seq-1.61 %0.08373
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par4.87 %0.03424
hpx::partial_sort, size: 100000par6.46 %0.04234
hpx::partial_sort, size: 100000, step: 1000par8.15 %0.06312
benchmark_nth_element_parallel
hpx::nth_element, size: 1000, step: 1par-11.40 %0.95884
hpx::nth_element, size: 100000, step: 5000par-0.85 %1.66926

@StellarBot
Copy link

Performance test report
Test nameExecutorMean Percentage Diff.Coefficient of variation
minmax_element_performance
hpx::minmaxpar-0.02 %38.85135
hpx::minpar-0.45 %3.00622
hpx::maxpar-0.79 %1.32714
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor2.13 %1.30068
stream_report
stream benchmark - Copyparallel_executor_default_parallel_policy_allocator-0.50 %1.70264
Stream benchmark - Addparallel_executor_default_parallel_policy_allocator-1.72 %0.36338
Stream benchmark - Triadparallel_executor_default_parallel_policy_allocator1.15 %0.43536
stream benchmark - Copyscheduler_executor_default_scheduler_executor_allocator-3.75 %0.23468
benchmark_is_heap_until
hpx::heap_untilpar-0.32 %6.67886
hpx::heap_untilpar_unseq0.55 %1.61591
benchmark_merge
hpx::mergeseq1.39 %0.03691
hpx::mergepar-4.63 %0.15019
hpx::mergepar_unseq-2.20 %0.29576
benchmark_inplace_merge
hpx::inplace_mergepar0.78 %2.12736
hpx::inplace_mergepar_unseq-0.17 %7.34590
benchmark_is_heap
hpx::is_heappar1.70 %1.47054
benchmark_remove_if
hpx::remove_ifseq0.89 %0.11261
hpx::remove_ifpar0.98 %0.08889
hpx::remove_ifpar_unseq1.04 %0.26621
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq7.16 %0.02518
hpx::partial_sort, size: 100000seq-0.89 %0.14436
benchmark_partial_sort_parallel
hpx::partial_sort, size: 100000par2.15 %0.12404
hpx::partial_sort, size: 100000, step: 1000par4.85 %0.09645
benchmark_nth_element_parallel
hpx::nth_element, size: 100000, step: 5000par-0.98 %1.18164

@StellarBot
Copy link

Performance test report
Test nameExecutorMean Percentage Diff.Coefficient of variation
minmax_element_performance
hpx::minmaxpar0.76 %2.77272
hpx::minpar0.79 %1.61304
hpx::maxpar-0.17 %5.72621
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor-0.03 %22.37570
stream_report
stream benchmark - Copyparallel_executor_default_parallel_policy_allocator-0.49 %1.92481
Stream benchmark - Scaleparallel_executor_default_parallel_policy_allocator4.08 %0.24543
Stream benchmark - Scalescheduler_executor_default_scheduler_executor_allocator-0.25 %4.56995
Stream benchmark - Addscheduler_executor_default_scheduler_executor_allocator1.58 %0.32848
foreach_report
for_eachscheduler_executor0.34 %2.50820
transform_reduce_scaling
transform_reduce_oldpar1.95 %0.24662
benchmark_is_heap_until
hpx::heap_untilpar-0.40 %4.96255
hpx::heap_untilpar_unseq1.71 %0.42793
benchmark_merge
hpx::mergeseq1.00 %0.05525
hpx::mergepar-6.01 %0.11866
hpx::mergepar_unseq-2.66 %0.22683
benchmark_inplace_merge
hpx::inplace_mergepar-0.25 %5.96318
hpx::inplace_mergepar_unseq3.69 %0.24123
benchmark_is_heap
hpx::is_heappar0.24 %12.59392
hpx::is_heappar_unseq-1.71 %0.44053
benchmark_remove
hpx::removeseq2.08 %0.08116
hpx::removepar1.18 %0.11226
hpx::removepar_unseq0.90 %0.09645
benchmark_remove_if
hpx::remove_ifseq1.00 %0.10714
hpx::remove_ifpar1.29 %0.07869
hpx::remove_ifpar_unseq1.32 %0.18357
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq9.64 %0.01445
hpx::partial_sort, size: 100000seq-1.13 %0.09270
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par1.62 %0.10247
hpx::partial_sort, size: 100000, step: 1000par10.23 %0.05292
benchmark_nth_element_parallel
hpx::nth_element, size: 100000, step: 5000par0.70 %0.39393

Copy link

codacy-production bot commented Aug 15, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-0.06% 86.70%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (17bbc63) 222489 189245 85.06%
Head commit (ff62d46) 196730 (-25759) 167219 (-22026) 85.00% (-0.06%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#6514) 436 378 86.70%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

@StellarBot
Copy link

Performance test report
Test nameExecutorMean Percentage Diff.Coefficient of variation
minmax_element_performance
hpx::minmaxpar4.10 %0.33835
hpx::minpar2.22 %0.47294
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor-2.48 %1.14423
stream_report
Stream benchmark - Scaleparallel_executor_default_parallel_policy_allocator5.08 %0.18510
stream benchmark - Copyscheduler_executor_default_scheduler_executor_allocator-5.06 %0.41326
Stream benchmark - Scalescheduler_executor_default_scheduler_executor_allocator2.40 %0.24650
Stream benchmark - Addscheduler_executor_default_scheduler_executor_allocator1.53 %0.32002
transform_reduce_scaling
transform_reducepar0.72 %1.15210
transform_reduce_oldpar1.19 %0.33932
benchmark_is_heap_until
hpx::heap_untilpar5.66 %0.26522
benchmark_merge
hpx::mergeseq1.49 %0.02982
hpx::mergepar-0.84 %0.47515
hpx::mergepar_unseq-1.12 %0.38487
benchmark_inplace_merge
hpx::inplace_mergepar8.98 %0.16203
hpx::inplace_mergepar_unseq10.69 %0.08418
benchmark_is_heap
hpx::is_heappar5.14 %0.34268
benchmark_remove
hpx::removeseq1.01 %0.11027
benchmark_remove_if
hpx::remove_ifseq0.89 %0.10283
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq4.11 %0.02851
hpx::partial_sort, size: 100000seq-1.11 %0.09047
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par-5.42 %0.02644
hpx::partial_sort, size: 100000par5.34 %0.04335
hpx::partial_sort, size: 100000, step: 1000par7.37 %0.06949

@StellarBot
Copy link

Performance test report
Test nameExecutorMean Percentage Diff.Coefficient of variation
minmax_element_performance
hpx::minpar-1.25 %0.45074
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor9.20 %0.25129
stream_report
Stream benchmark - Addparallel_executor_default_parallel_policy_allocator1.03 %0.30720
Stream benchmark - Triadparallel_executor_default_parallel_policy_allocator0.91 %0.36907
stream benchmark - Copyscheduler_executor_default_scheduler_executor_allocator3.72 %0.78148
Stream benchmark - Triadscheduler_executor_default_scheduler_executor_allocator1.53 %0.22015
benchmark_merge
hpx::mergepar0.55 %0.47600
benchmark_inplace_merge
hpx::inplace_mergepar0.34 %3.88061
hpx::inplace_mergepar_unseq1.05 %1.06177
benchmark_is_heap
hpx::is_heappar0.56 %1.41438
benchmark_remove
hpx::removepar_unseq-0.72 %1.45818
benchmark_remove_if
hpx::remove_ifpar_unseq0.10 %10.37297
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq1.52 %0.07678
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par6.47 %0.04699
hpx::partial_sort, size: 100000par2.04 %0.11232
benchmark_nth_element_parallel
hpx::nth_element, size: 1000, step: 1par-6.31 %0.93441
hpx::nth_element, size: 100000, step: 5000par0.23 %5.23035

@StellarBot
Copy link

Performance test report
Test nameExecutorMean Percentage Diff.Coefficient of variation
minmax_element_performance
hpx::maxpar0.75 %0.71582
small_vector_benchmark
hpx::small_vector<int, 2>-1.09 %0.43155
hpx::small_vector<int, 4>-1.00 %0.21861
hpx::small_vector<int, 16>1.39 %0.00417
hpx::small_vector<fxn<void()>, 1>2.63 %0.03075
hpx::small_vector<fxn<void()>, 2>1.55 %0.05928
hpx::small_vector<fxn<void()>, 4>4.30 %0.00848
hpx::small_vector<fxn<void()>, 8>1.35 %0.04220
hpx::small_vector<fxn<void()>, 16>3.77 %0.00194
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor7.44 %0.25946
stream_report
stream benchmark - Copyparallel_executor_default_parallel_policy_allocator-1.26 %0.45600
Stream benchmark - Addparallel_executor_default_parallel_policy_allocator0.53 %0.20760
Stream benchmark - Triadparallel_executor_default_parallel_policy_allocator0.67 %0.19165
stream benchmark - Copyscheduler_executor_default_scheduler_executor_allocator2.15 %0.16984
Stream benchmark - Addscheduler_executor_default_scheduler_executor_allocator1.14 %0.74768
Stream benchmark - Triadscheduler_executor_default_scheduler_executor_allocator0.77 %0.24945
transform_reduce_scaling
transform_reducepar-0.52 %0.43732
transform_reduce_oldpar-0.67 %0.32381
benchmark_is_heap_until
hpx::heap_untilseq16.66 %0.00071
benchmark_merge
hpx::mergepar1.20 %0.24594
hpx::mergepar_unseq1.23 %0.30648
benchmark_inplace_merge
hpx::inplace_mergeseq0.39 %0.16341
hpx::inplace_mergepar-7.56 %0.08207
hpx::inplace_mergepar_unseq-9.89 %0.05748
benchmark_is_heap
hpx::is_heapseq16.64 %0.00047
hpx::is_heappar-0.02 %3.44765
benchmark_remove
hpx::removepar_unseq-1.22 %0.02323
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq5.18 %0.01505
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par7.42 %0.01615
hpx::partial_sort, size: 100000par3.74 %0.06045
hpx::partial_sort, size: 100000, step: 1000par-2.06 %0.02687

@StellarBot
Copy link

Performance test report
Test nameExecutorPercentage Diff.Classification
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor1.57 %?
stream_report
Stream benchmark - Triadparallel_executor_default_parallel_policy_allocator2.32 %-
stream benchmark - Copyscheduler_executor_default_scheduler_executor_allocator5.63 %--
Stream benchmark - Scalescheduler_executor_default_scheduler_executor_allocator1.54 %-
Stream benchmark - Addscheduler_executor_default_scheduler_executor_allocator1.31 %-
Stream benchmark - Triadscheduler_executor_default_scheduler_executor_allocator1.89 %-
benchmark_is_heap_until
hpx::heap_untilseq16.65 %---
benchmark_inplace_merge
hpx::inplace_mergepar0.77 %?
hpx::inplace_mergepar_unseq-1.58 %+
benchmark_is_heap
hpx::is_heapseq16.66 %---
hpx::is_heappar0.21 %?
benchmark_partial_sort_parallel
hpx::partial_sort, size: 100000par3.07 %-
hpx::partial_sort, size: 100000, step: 1000par-3.09 %+
benchmark_nth_element_parallel
hpx::nth_element, size: 1000, step: 1par-15.38 %+++
hpx::nth_element, size: 100000, step: 5000par-7.18 %++

@StellarBot
Copy link

Performance test report
Test nameExecutorPercentage Diff.Classification
minmax_element_performance
hpx::minmaxpar-0.74 %?
hpx::maxpar-1.84 %+
small_vector_benchmark
hpx::small_vectorint, 12.99 %-
hpx::small_vectorint, 2-1.51 %+
hpx::small_vectorint, 42.69 %-
hpx::small_vectorint, 815.83 %---
hpx::small_vectorint, 1635.71 %---
hpx::small_vectorfxn:void(), 17.22 %--
hpx::small_vectorfxn:void(), 26.00 %--
hpx::small_vectorfxn:void(), 45.38 %--
hpx::small_vectorfxn:void(), 88.88 %--
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor1.51 %?
stream_report
Stream benchmark - Addparallel_executor_default_parallel_policy_allocator1.55 %-
Stream benchmark - Triadparallel_executor_default_parallel_policy_allocator2.98 %-
stream benchmark - Copyscheduler_executor_default_scheduler_executor_allocator6.43 %--
Stream benchmark - Scalescheduler_executor_default_scheduler_executor_allocator1.30 %?
Stream benchmark - Addscheduler_executor_default_scheduler_executor_allocator2.28 %-
Stream benchmark - Triadscheduler_executor_default_scheduler_executor_allocator2.07 %-
foreach_report
for_eachparallel_executor-0.02 %(+)
benchmark_is_heap_until
hpx::heap_untilseq16.65 %---
benchmark_merge
hpx::mergepar-3.96 %+
benchmark_inplace_merge
hpx::inplace_mergepar1.18 %?
hpx::inplace_mergepar_unseq-1.16 %+
benchmark_is_heap
hpx::is_heapseq16.65 %---
hpx::is_heappar-3.49 %++
benchmark_partial_sort_parallel
hpx::partial_sort, size: 100000par2.79 %-
hpx::partial_sort, size: 100000, step: 1000par2.29 %-
benchmark_nth_element_parallel
hpx::nth_element, size: 1000, step: 1par-17.33 %+++
hpx::nth_element, size: 100000, step: 5000par-8.50 %++

…ternal benchmarking iterations to 1000

Signed-off-by: Vedant <[email protected]>
@StellarBot
Copy link

Performance test report
Test nameExecutorPercentage Diff.Classification
minmax_element_performance
hpx::minmaxpar1.64 %-
small_vector_benchmark
hpx::small_vectorint, 12.53 %-
hpx::small_vectorint, 2-3.50 %+
hpx::small_vectorint, 815.97 %---
hpx::small_vectorint, 1634.28 %---
hpx::small_vectorfxn:void(), 16.99 %--
hpx::small_vectorfxn:void(), 23.54 %-
hpx::small_vectorfxn:void(), 45.39 %--
hpx::small_vectorfxn:void(), 88.94 %--
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor-1.82 %+
stream_report
stream benchmark - Copyparallel_executor_default_parallel_policy_allocator2.18 %-
Stream benchmark - Addparallel_executor_default_parallel_policy_allocator2.77 %-
Stream benchmark - Triadparallel_executor_default_parallel_policy_allocator4.53 %--
stream benchmark - Copyscheduler_executor_default_scheduler_executor_allocator6.86 %--
Stream benchmark - Scalescheduler_executor_default_scheduler_executor_allocator2.38 %-
Stream benchmark - Addscheduler_executor_default_scheduler_executor_allocator1.77 %-
Stream benchmark - Triadscheduler_executor_default_scheduler_executor_allocator3.74 %--
benchmark_is_heap_until
hpx::heap_untilseq16.71 %---
benchmark_merge
hpx::mergepar_unseq-2.83 %+
benchmark_inplace_merge
hpx::inplace_mergepar0.93 %?
hpx::inplace_mergepar_unseq-1.24 %+
benchmark_is_heap
hpx::is_heapseq16.71 %---
hpx::is_heappar0.48 %?
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq10.67 %---
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par3.01 %-
hpx::partial_sort, size: 100000par1.75 %-
hpx::partial_sort, size: 100000, step: 1000par5.22 %--
benchmark_nth_element_parallel
hpx::nth_element, size: 1000, step: 1par-16.44 %+++
hpx::nth_element, size: 100000, step: 5000par-7.93 %++

@StellarBot
Copy link

Performance test report
Test nameExecutorPercentage Diff.Classification
small_vector_benchmark
hpx::small_vectorfxn:void(), 1-2.20 %+
hpx::small_vectorfxn:void(), 4-4.90 %+
hpx::small_vectorfxn:void(), 8-4.02 %+
hpx::small_vectorfxn:void(), 16-3.08 %+
stream_report
Stream benchmark - Triadparallel_executor_default_parallel_policy_allocator-1.86 %+
benchmark_is_heap_until
hpx::heap_untilpar11.95 %---
benchmark_inplace_merge
hpx::inplace_mergepar-9.69 %++
hpx::inplace_mergepar_unseq-12.82 %+++
benchmark_is_heap
hpx::is_heappar-18.75 %+++
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq-8.20 %++
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par7.43 %--
hpx::partial_sort, size: 100000par2.82 %-
hpx::partial_sort, size: 100000, step: 1000par-11.46 %+++
benchmark_nth_element_parallel
hpx::nth_element, size: 100000, step: 5000par2.86 %-

@StellarBot
Copy link

Performance test report
Test nameExecutorPercentage Diff.Classification
small_vector_benchmark
hpx::small_vectorfxn:void(), 1-2.24 %+
hpx::small_vectorfxn:void(), 4-4.82 %+
hpx::small_vectorfxn:void(), 8-3.89 %+
hpx::small_vectorfxn:void(), 16-3.05 %+
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor-1.56 %+
benchmark_is_heap_until
hpx::heap_untilpar11.67 %---
benchmark_merge
hpx::mergepar_unseq2.39 %-
benchmark_is_heap
hpx::is_heappar-18.59 %+++
benchmark_partial_sort
hpx::partial_sort, size: 1000, step: 1seq-4.32 %+
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par3.59 %-
hpx::partial_sort, size: 100000par4.32 %-
hpx::partial_sort, size: 100000, step: 1000par-4.45 %+
benchmark_nth_element_parallel
hpx::nth_element, size: 100000, step: 5000par2.03 %-

Copy link

codacy-production bot commented Aug 24, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-0.05% 91.82%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (17bbc63) 222489 189245 85.06%
Head commit (c69f9a3) 196783 (-25706) 167291 (-21954) 85.01% (-0.05%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#6514) 501 460 91.82%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

…tifacts, and updated baselines to a stable set

Signed-off-by: Vedant <[email protected]>
@StellarBot
Copy link

Performance test report
Test nameExecutorPercentage Diff.Classification
minmax_element_performance
hpx::minmaxpar-3.11 %+
small_vector_benchmark
hpx::small_vectorint, 162.17 %-
hpx::small_vectorfxn:void(), 23.84 %-
hpx::small_vectorfxn:void(), 43.93 %-
hpx::small_vectorfxn:void(), 83.03 %-
future_overhead_report
future overhead - create_thread_hierarchical - latchno-executor-6.14 %++
stream_report
Stream benchmark - Addscheduler_executor_default_scheduler_executor_allocator-1.73 %+
Stream benchmark - Triadscheduler_executor_default_scheduler_executor_allocator1.83 %-
benchmark_partial_sort_parallel
hpx::partial_sort, size: 1000, step: 1par-2.57 %+
hpx::partial_sort, size: 100000par-2.11 %+
hpx::partial_sort, size: 100000, step: 1000par5.64 %--

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants