Skip to content

BenchmarkToolkit is a Python library designed to simplify and streamline benchmarking tasks. This library provides a high-level abstraction over various benchmarking functions, allowing developers to easily perform performance testing and analysis with minimal setup.

Notifications You must be signed in to change notification settings

flatironinstitute/PyBenchFramework

Folders and files

NameName
Last commit message
Last commit date

Latest commit

6abe0a4 · Feb 14, 2025
Oct 22, 2024
Feb 12, 2025
Feb 14, 2025
Nov 3, 2024
Aug 16, 2024
Sep 8, 2024
Sep 8, 2024
Sep 8, 2024
Sep 8, 2024
Jan 27, 2025

Repository files navigation

PyBenchFramework

#throw this change out This application is being used with slurm but it can run on its own.

User must set the environment variable "PyBench_root_dir" to run run.py. Try to use mdtest_env to start the environment for MDTEST or env_start for FIO. Don't use both at the same time.

The app needs a root directory to read templates and store intermediate config files as well as log files and the like.

Options can be included as arguments to 'run.py' or as fields in a YAML config file. The config file for an FIO job looks like so:

slurm_job_number: 
block_size: "4M,64K,4K"
directory: /mnt/cephtest-fi5k/test-rep3-ssd/skrit/fio
time: 120
io_type: write
platform_type: nvme_rep3_kernel
split_hosts_file: 0
job_number: '48,16,8,4,2,1'
node_count: 20,16,8,4,2,1
hosts_file:
no_scrub: 0
unit_restart: 1
template_path: /mnt/home/skrit/Documents/testing_clones/clone1/PyBenchFramework/examples/template/template.fio

The config file for an mdtest job looks like so:

mpi_ranks: 40,30,20,10,5 
directory: /mnt/cephtestk/test-ec63/skrit/mdtest
files_per_rank: 20,10,5
test_repetition: 3
slurm_job_number:
io_type: metadata
platform_type: kernel_EC63 
offset: 1
write_data: '3901'
read_data: '3901'
node_count: 10,5,1

One thing to note is that the main loop takes mpi ranks and multiplies it to the node count. For example, when mpi_ranks is 5 and node_count is 1, mpirun is given '-n 5' for a total of 5 mpi ranks. When mpi_ranks is 5 and node_count is 10, mpirun is given '-n 50' (5 * 10) which is distributed across 10 nodes (--map-by node -N 10).

'slurm_job_number' should, at least for now, be an inline argument. Keep in mind inline arguments are likely separated by - rather than _ --help should show all arguments as they should be inline.

Under the 'submit_scripts' folder, you'll see several slurm scripts. 'mdtest.sh' is a good template for submitting mdtest jobs and the scripts starting with 'ssd...' or 'hdd...' are good templates for FIO jobs.

About

BenchmarkToolkit is a Python library designed to simplify and streamline benchmarking tasks. This library provides a high-level abstraction over various benchmarking functions, allowing developers to easily perform performance testing and analysis with minimal setup.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published