Skip to content

API/TL/UCP: Add per-team endpoint traffic class (QoS) - #1321

Draft
ybenvidia wants to merge 36 commits into
openucx:masterfrom
ybenvidia:collectives-prioritization
Draft

API/TL/UCP: Add per-team endpoint traffic class (QoS)#1321
ybenvidia wants to merge 36 commits into
openucx:masterfrom
ybenvidia:collectives-prioritization

Conversation

@ybenvidia

Copy link
Copy Markdown
Contributor

What

Allow a UCC team to be created with an endpoint traffic class so all UCX
endpoints opened for that team use a chosen DSCP/priority.

Why

Enables per-team (≈ per communicator / process-group) QoS: run a
higher-priority collective on a team tagged with a higher-priority traffic
class, and let the NIC/fabric arbitrate between flows accordingly.

Changes

  • Public API: new ucc_team_params.ep_traffic_class field, enabled with
    UCC_TEAM_PARAM_FIELD_EP_TRAFFIC_CLASS.
  • Plumbing: ucc_team_create_posttl_ucp team
    (UCC_CLASS_INIT_FUNC(ucc_tl_ucp_team_t)) → ucc_tl_ucp_connect_ep, which
    sets ucp_ep_params.ep_traffic_class + UCP_EP_PARAM_FIELD_EP_TRAFFIC_CLASS.
  • Sentinel UCC_TL_UCP_NO_TCLASS for "not set"; service/internal teams keep
    the default.
  • ucc_perftest: new UCC_PT_EP_TRAFFIC_CLASS env to set the team's traffic
    class for benchmarking.

Dependency

Requires a UCX with the per-endpoint traffic class API
(UCP_EP_PARAM_FIELD_EP_TRAFFIC_CLASS / ucp_ep_params.ep_traffic_class) —
see the companion UCX PR.

Compatibility

Opt-in: teams created without the field mask bit behave exactly as before.

Testing

  • ucc_perftest -c allreduce with UCC_PT_EP_TRAFFIC_CLASS=<dscp> on 2 nodes:
    the tagged team's endpoints open QPs with the configured DSCP, verified down
    to the RoCEv2 packets on the wire.

@ybenvidia
ybenvidia marked this pull request as draft July 2, 2026 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant