Subphase API Proposal

Now that we have the PR #1255 for subphase labels, here's the proposal for the user-facing API for subphases.

There are two elements to the design. First, is the interaction with vt::CollectionChainSet; second, the pure epoch variant:

CollectionChainSet

nextStep

I propose we add another call for nextStep that also creates a subphase:

  void CollectionChainSet::nextStep(
    std::string const& label, std::function<PendingSend(Index)> step_action
  );

This new call will create a subphase with the label that already exists and will group actions in step_action from that rooted step into a rooted subphase:

  void CollectionChainSet::nextStepSubphase(
    std::string const& label, std::function<PendingSend(Index)> step_action
  );

nextStepCollective

I propose we add a similar call for nextStepCollective that creates a subphase:

  void CollectionChainSet::nextStepCollective(
    std::string const& label, std::function<PendingSend(Index)> step_action
  );

This new call with also create a subphase with the label similar to the previous call:

  void CollectionChainSet::nextStepCollectiveSubphase(
    std::string const& label, std::function<PendingSend(Index)> step_action
  );

Higher-level calls

The calls will be stateful calls on the chain set that will start/stop a new subphase across the whole collection. They will end up creating an epoch under the hood to group all the work enqueued by them and associating it with the subphase. This brings up potential nesting issues wrt to subphases...

  void CollectionChainSet::startCollectiveSubphase(std::string const& label);
  void CollectionChainSet::endCollectiveSubphase(std::string const& label);