MVP support for serving workloads running as LeaderWorkerSet #3232
Open
Labels
kind/feature
Categorizes issue or PR as related to a new feature.
What would you like to be added:
MVP support for LeaderWorkerSet in Kueue. It does not need to be ideal, but we want to have some support to unblock users and collect users' feedback.
The idea is to base the support on StatefulSets, so the integration would also use Pod Groups, similarly as for regular StatefulSets. Each LeaderWorkerGroup creates a new Pod Group. I a single pod group we will have:
The size of the group will be taken from LeaderWorkerSet.Spec.LeaderWorkerTemplate.Size and increased by 1 (to include the leader).
This is a follow up to #2717.
Why is this needed:
We want to support serving primitives in Kueue as there is an increasing demand among users to run clusters mixing AI training and inference who want to manage the expensive GPU resources.
LeaderWorkerSet is a new serving API which is gaining popularity as a primitive to host AI/ML inference.
The text was updated successfully, but these errors were encountered: