Skip to content

Conversation

ehigham
Copy link
Member

@ehigham ehigham commented Jul 23, 2025

Change Description

Refactored the branching factor parameter to be controlled through feature flags instead of being passed directly to the HailContext constructor. This change:

  1. Removes the branching factor parameter from HailContext constructor and methods
  2. Adds a new feature flag "branch_factor" with a default value of 50
  3. Creates a new agg package with utility functions to retrieve the branch factor from context
  4. Updates all code that previously accessed HailContext.get.branchingFactor to use agg.branchFactor(ctx) instead
  5. Modifies Python backends to pass branching factor through flags rather than constructor parameters

This approach provides a more consistent way to configure the branching factor through environment variables or programmatic flags, aligning with how other Hail parameters are managed.

Security Assessment

This change cannot impact the Hail Batch instance as deployed by Broad Institute in GCP

Copy link
Member Author

ehigham commented Jul 23, 2025

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from bb507e7 to 0bf6462 Compare July 23, 2025 03:41
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from 3adc65e to 11403ed Compare July 23, 2025 03:41
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from 0bf6462 to 969d3a0 Compare July 23, 2025 03:49
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch 2 times, most recently from 2e67e7b to 417e8a3 Compare July 23, 2025 15:32
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from 969d3a0 to 3c9e3ce Compare July 23, 2025 15:32
@ehigham ehigham changed the title [query] remove branchingFactor from HailContext` [query] remove branchingFactor from HailContext Jul 24, 2025
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from 417e8a3 to e739136 Compare July 30, 2025 20:07
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch 2 times, most recently from 5e95ee0 to 3d6c841 Compare July 31, 2025 16:12
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch 2 times, most recently from 2d38548 to c3bb8a1 Compare July 31, 2025 16:38
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch 2 times, most recently from 9873870 to afe05c8 Compare July 31, 2025 17:41
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from c3bb8a1 to 24a200f Compare July 31, 2025 17:41
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from afe05c8 to d370b72 Compare July 31, 2025 17:41
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch 2 times, most recently from ae470ba to c164f2f Compare August 1, 2025 14:58
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from d370b72 to 5ce40c9 Compare August 1, 2025 14:58
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from c164f2f to 4538bfd Compare August 1, 2025 22:11
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch 2 times, most recently from 17d1b88 to c6bdbe9 Compare August 1, 2025 22:16
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch 2 times, most recently from 9d81d62 to 4f0cd2b Compare August 2, 2025 04:21
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch 2 times, most recently from 3eee0f5 to 7cca08e Compare August 2, 2025 04:59
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from 4f0cd2b to c070764 Compare August 2, 2025 04:59
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from 7cca08e to ae46567 Compare August 2, 2025 05:02
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from c070764 to 8834837 Compare August 2, 2025 05:02
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from ae46567 to 008bd40 Compare August 4, 2025 17:07
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from 8834837 to 1eedaf6 Compare August 4, 2025 17:07
@ehigham ehigham changed the base branch from ehigham/optimiser-flags to graphite-base/14963 August 5, 2025 21:01
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from 1eedaf6 to a4de72a Compare August 6, 2025 03:24
@ehigham ehigham force-pushed the graphite-base/14963 branch from 008bd40 to 0d67776 Compare August 6, 2025 03:24
@ehigham ehigham changed the base branch from graphite-base/14963 to ehigham/optimiser-flags August 6, 2025 03:24
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from a4de72a to 86a588b Compare August 6, 2025 04:20
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from 0d67776 to 30456d7 Compare August 6, 2025 04:20
@ehigham ehigham force-pushed the ehigham/branching-factor-flags branch from 86a588b to a6deb82 Compare August 6, 2025 04:23
@ehigham ehigham force-pushed the ehigham/optimiser-flags branch from 30456d7 to 5977e83 Compare August 6, 2025 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant