Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New directives syntax #819

Draft
wants to merge 34 commits into
base: main
Choose a base branch
from
Draft

New directives syntax #819

wants to merge 34 commits into from

Commits on Nov 1, 2023

  1. refactor: Create partition and partition config classes

    @b-butler
    b-butler committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    9e220e0 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. fix: argument ordering of _Partition

    @b-butler
    b-butler committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    48d3c03 View commit details
    Browse the repository at this point in the history

  2. refactor: Switch shared_partition for node_types 

    Supports 3 node types:
- shared: 1 node or less
- mixed: any number of tasks
- wholenode: whole node increments
    @b-butler
    b-butler committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    a18bf6a View commit details
    Browse the repository at this point in the history

  3. refactor: Raise more template errors in Python

    @b-butler
    b-butler committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    45f47f8 View commit details
    Browse the repository at this point in the history

  4. doc/refactor: Document new classes 

    Also move threshold to _PartitionConfig class and gpus_per_node default
to 0.
    @b-butler
    b-butler committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    ba63e0f View commit details
    Browse the repository at this point in the history

  5. test: add force=True to submit (handle ngpu directives)

    @b-butler
    b-butler committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    14f4076 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. test: Test new errors in _Partition

    @b-butler
    b-butler committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    c3ccf3d View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2023

  1. refactor (WIP): Switch over directives to new structure

    @b-butler
    b-butler committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    aaa0e7f View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. refactor: Split _JobOperation into _JobOperation and _RunOperation

    @b-butler
    b-butler committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    5be1991 View commit details
    Browse the repository at this point in the history

  2. test: Remove fork directive tests.

    @b-butler
    b-butler committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    20bf55a View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. fix: Errors in run operations from no _RunOperation.id

    @b-butler
    b-butler committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    92a6ec6 View commit details
    Browse the repository at this point in the history

  2. test: Fix test of run_options

    @b-butler
    b-butler committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    0244e00 View commit details
    Browse the repository at this point in the history

  3. feat: Make _Directives.evaluate returns self.

    @b-butler
    b-butler committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    2782aaf View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into refactor/environment-config

    @b-butler
    b-butler committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    fed4c6e View commit details
    Browse the repository at this point in the history

  5. Merge branch 'refactor/environment-config' into refactor/new-directives

    @b-butler
    b-butler committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    2c9ba3a View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2023

  1. Merge branch 'main' into refactor/new-directives

    @b-butler
    b-butler committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    dcd9250 View commit details
    Browse the repository at this point in the history

  2. feat (WIP): "Working" submission logic for slurm.

    @b-butler
    b-butler committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    e40dc2b View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2024

  1. Merge branch 'main' into refactor/new-directives

    @b-butler
    b-butler committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    7ad0419 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. Merge branch 'main' into refactor/new-directives

    @b-butler
    b-butler committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    ad56f23 View commit details
    Browse the repository at this point in the history

Commits on Feb 29, 2024

  1. feat: Add None tolerance max, sum, argmax functions

    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    5f3a845 View commit details
    Browse the repository at this point in the history

  2. refactor: _Directives.evaluate now computes total cpus, gpus, and memory 

    This is useful for various group/bundling resource aggregation.
    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    36b303e View commit details
    Browse the repository at this point in the history

  3. refactor: Use total cpu, gpu, and memory internal directives 

    We use these directives internally to help with scheduling. Even if used
in templates, using the per-process first allows for more control over
resource request and computing the totals later is cheap and simple.
    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    76f300c View commit details
    Browse the repository at this point in the history

  4. refactor: MPI/cmd prefix logic 

    Uses new directives and -per-task options.
    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    2123989 View commit details
    Browse the repository at this point in the history

  5. refactor: Remove Summit's calc_num_nodes

    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    ca805eb View commit details
    Browse the repository at this point in the history

  6. refactor: Update template filters

    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    4701573 View commit details
    Browse the repository at this point in the history

  7. refactor: Attempt to update drexel configuration

    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    3b03c5a View commit details
    Browse the repository at this point in the history

  8. refactor: Update remaining templates to new directives

    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    ea51b43 View commit details
    Browse the repository at this point in the history

  9. fix: some documention and miscellaneous code

    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    1156b4f View commit details
    Browse the repository at this point in the history

  10. test: Update tests to new directives

    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    1aec7f5 View commit details
    Browse the repository at this point in the history

  11. feat: Allow mixed MPI/nonMPI operation submission.

    @b-butler
    b-butler committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    b523c7a View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. Merge branch 'main' into refactor/new-directives

    @b-butler
    b-butler committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    70b2778 View commit details
    Browse the repository at this point in the history

  2. (WIP)fix: bugs surrounding directives processing

    @b-butler
    b-butler committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    8f007b5 View commit details
    Browse the repository at this point in the history

  3. test: Correct template test project launcher values

    @b-butler
    b-butler committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    c86c4d4 View commit details
    Browse the repository at this point in the history

  4. test: Update template tests configuration generation

    @b-butler
    b-butler committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    c17fd6f View commit details
    Browse the repository at this point in the history