Skip to content

feat: support rolling update in the cluster API #8577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Feb 26, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
f15db24
feat: support InstanceUpdateStrategy
free6om Dec 4, 2024
95b47a1
rename to RollingUpdate
free6om Dec 4, 2024
02208d3
move&refactor all update related fields into a single struct
free6om Dec 5, 2024
374702c
update stratey constraint
free6om Dec 5, 2024
4f3546d
flattern update strategy in cmpd
free6om Dec 6, 2024
0a07c96
adjust fields name in controllers according to apis
free6om Dec 11, 2024
4692b7b
v1alpha1 <-> v1 conversion
free6om Dec 12, 2024
19b19ba
Merge branch 'main' into support/instance-update-strategy
free6om Dec 12, 2024
c47b102
fix staticcheck error
free6om Dec 12, 2024
d2c0b28
Merge branch 'main' into support/instance-update-strategy
free6om Dec 19, 2024
7ffb807
remove InstanceUpdatePolicy from cmpd
free6om Dec 23, 2024
48d9c85
support OnDelete update strategy
free6om Dec 23, 2024
bcb1635
fix broken ut
free6om Dec 23, 2024
72599d6
Merge branch 'main' into support/instance-update-strategy
free6om Jan 15, 2025
eca9492
Merge branch 'main' into support/instance-update-strategy
leon-inf Feb 19, 2025
3b3034b
revert changes to component and sharding definition
leon-inf Feb 19, 2025
362d4ea
revert pod update policy
leon-inf Feb 19, 2025
d4fa971
Merge branch 'main' into support/instance-update-strategy
leon-inf Feb 20, 2025
ef22954
rename update strategy
leon-inf Feb 20, 2025
7cb8f1f
remove update concurrency from cluster/comp API
leon-inf Feb 20, 2025
dc44672
rename and refine
leon-inf Feb 20, 2025
d051655
Merge branch 'main' into support/instance-update-strategy
leon-inf Feb 20, 2025
d1f1743
rename update strategy in its
leon-inf Feb 20, 2025
0c6a2f8
tidyup
leon-inf Feb 20, 2025
62877c1
remove update concurrency and use member update strategy
leon-inf Feb 21, 2025
4b72127
reuse apps definition
leon-inf Feb 21, 2025
cb737ab
fix test
leon-inf Feb 21, 2025
319836f
Merge branch 'main' into support/instance-update-strategy
leon-inf Feb 25, 2025
173fa77
tidyup
leon-inf Feb 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions apis/apps/v1/cluster_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -438,6 +438,11 @@ type ClusterComponentSpec struct {
// +optional
PodUpdatePolicy *PodUpdatePolicyType `json:"podUpdatePolicy,omitempty"`

// Provides fine-grained control over the spec update process of all instances.
//
// +optional
InstanceUpdateStrategy *InstanceUpdateStrategy `json:"instanceUpdateStrategy,omitempty"`

// Allows for the customization of configuration values for each instance within a Component.
// An instance represent a single replica (Pod and associated K8s resources like PVCs, Services, and ConfigMaps).
// While instances typically share a common configuration as defined in the ClusterComponentSpec,
Expand Down
5 changes: 5 additions & 0 deletions apis/apps/v1/component_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,11 @@ type ComponentSpec struct {
// +optional
PodUpdatePolicy *PodUpdatePolicyType `json:"podUpdatePolicy,omitempty"`

// Provides fine-grained control over the spec update process of all instances.
//
// +optional
InstanceUpdateStrategy *InstanceUpdateStrategy `json:"instanceUpdateStrategy,omitempty"`

// Specifies the scheduling policy for the Component.
//
// +optional
Expand Down
54 changes: 54 additions & 0 deletions apis/apps/v1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package v1

import (
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/util/intstr"
)

const (
Expand Down Expand Up @@ -526,6 +527,59 @@ const (
PreferInPlacePodUpdatePolicyType PodUpdatePolicyType = "PreferInPlace"
)

// InstanceUpdateStrategy defines fine-grained control over the spec update process of all instances.
type InstanceUpdateStrategy struct {
// Indicates the type of the update strategy.
// Default is RollingUpdate.
//
// +optional
Type InstanceUpdateStrategyType `json:"type,omitempty"`

// Specifies how the rolling update should be applied.
//
// +optional
RollingUpdate *RollingUpdate `json:"rollingUpdate,omitempty"`
}

// InstanceUpdateStrategyType is a string enumeration type that enumerates
// all possible update strategies for the KubeBlocks controllers.
//
// +enum
// +kubebuilder:validation:Enum={RollingUpdate,OnDelete}
type InstanceUpdateStrategyType string

const (
// RollingUpdateStrategyType indicates that update will be
// applied to all Instances with respect to the workload
// ordering constraints.
RollingUpdateStrategyType InstanceUpdateStrategyType = "RollingUpdate"
// OnDeleteStrategyType indicates that ordered rolling restarts are disabled. Instances are recreated
// when they are manually deleted.
OnDeleteStrategyType InstanceUpdateStrategyType = "OnDelete"
)

// RollingUpdate specifies how the rolling update should be applied.
type RollingUpdate struct {
// Indicates the number of instances that should be updated during a rolling update.
// The remaining instances will remain untouched. This is helpful in defining how many instances
// should participate in the update process.
// Value can be an absolute number (ex: 5) or a percentage of desired instances (ex: 10%).
// Absolute number is calculated from percentage by rounding up.
// The default value is ComponentSpec.Replicas (i.e., update all instances).
//
// +optional
Replicas *intstr.IntOrString `json:"replicas,omitempty"`

// The maximum number of instances that can be unavailable during the update.
// Value can be an absolute number (ex: 5) or a percentage of desired instances (ex: 10%).
// Absolute number is calculated from percentage by rounding up. This can not be 0.
// Defaults to 1. The field applies to all instances. That means if there is any unavailable pod,
// it will be counted towards MaxUnavailable.
//
// +optional
MaxUnavailable *intstr.IntOrString `json:"maxUnavailable,omitempty"`
}

type SchedulingPolicy struct {
// If specified, the Pod will be dispatched by specified scheduler.
// If not specified, the Pod will be dispatched by default scheduler.
Expand Down
55 changes: 55 additions & 0 deletions apis/apps/v1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

68 changes: 34 additions & 34 deletions apis/workloads/v1/instanceset_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -184,13 +184,20 @@ type InstanceSetSpec struct {
// +optional
PodUpdatePolicy PodUpdatePolicyType `json:"podUpdatePolicy,omitempty"`

// Indicates the StatefulSetUpdateStrategy that will be
// employed to update Pods in the InstanceSet when a revision is made to
// Template.
// UpdateStrategy.Type will be set to appsv1.OnDeleteStatefulSetStrategyType if MemberUpdateStrategy is not nil
// Provides fine-grained control over the spec update process of all instances.
//
// Note: This field will be removed in future version.
UpdateStrategy appsv1.StatefulSetUpdateStrategy `json:"updateStrategy,omitempty"`
// +optional
InstanceUpdateStrategy *InstanceUpdateStrategy `json:"instanceUpdateStrategy,omitempty"`

// Members(Pods) update strategy.
//
// - serial: update Members one by one that guarantee minimum component unavailable time.
// - parallel: force parallel
// - bestEffortParallel: update Members in parallel that guarantee minimum component un-writable time.
//
// +kubebuilder:validation:Enum={Serial,Parallel,BestEffortParallel}
// +optional
MemberUpdateStrategy *MemberUpdateStrategy `json:"memberUpdateStrategy,omitempty"`

// A list of roles defined in the system. Instanceset obtains role through pods' role label `kubeblocks.io/role`.
//
Expand All @@ -207,16 +214,6 @@ type InstanceSetSpec struct {
// +optional
TemplateVars map[string]string `json:"templateVars,omitempty"`

// Members(Pods) update strategy.
//
// - serial: update Members one by one that guarantee minimum component unavailable time.
// - bestEffortParallel: update Members in parallel that guarantee minimum component un-writable time.
// - parallel: force parallel
//
// +kubebuilder:validation:Enum={Serial,BestEffortParallel,Parallel}
// +optional
MemberUpdateStrategy *MemberUpdateStrategy `json:"memberUpdateStrategy,omitempty"`

// Indicates that the InstanceSet is paused, meaning the reconciliation of this InstanceSet object will be paused.
// +optional
Paused bool `json:"paused,omitempty"`
Expand Down Expand Up @@ -308,16 +305,29 @@ type InstanceSetStatus struct {
// +kubebuilder:object:generate=false
type InstanceTemplate = kbappsv1.InstanceTemplate

type PodUpdatePolicyType string
// PodUpdatePolicyType indicates how pods should be updated
//
// +kubebuilder:object:generate=false
type PodUpdatePolicyType = kbappsv1.PodUpdatePolicyType

const (
// StrictInPlacePodUpdatePolicyType indicates that only allows in-place upgrades.
// Any attempt to modify other fields will be rejected.
StrictInPlacePodUpdatePolicyType PodUpdatePolicyType = "StrictInPlace"
// InstanceUpdateStrategy defines fine-grained control over the spec update process of all instances.
//
// +kubebuilder:object:generate=false
type InstanceUpdateStrategy = kbappsv1.InstanceUpdateStrategy

// PreferInPlacePodUpdatePolicyType indicates that we will first attempt an in-place upgrade of the Pod.
// If that fails, it will fall back to the ReCreate, where pod will be recreated.
PreferInPlacePodUpdatePolicyType PodUpdatePolicyType = "PreferInPlace"
// RollingUpdate specifies how the rolling update should be applied.
//
// +kubebuilder:object:generate=false
type RollingUpdate = kbappsv1.RollingUpdate

// MemberUpdateStrategy defines Cluster Component update strategy.
// +enum
type MemberUpdateStrategy string

const (
SerialUpdateStrategy MemberUpdateStrategy = "Serial"
ParallelUpdateStrategy MemberUpdateStrategy = "Parallel"
BestEffortParallelUpdateStrategy MemberUpdateStrategy = "BestEffortParallel"
)

// ReplicaRole represents a role that can be assigned to a component instance, defining its behavior and responsibilities.
Expand Down Expand Up @@ -404,16 +414,6 @@ type MembershipReconfiguration struct {
Switchover *kbappsv1.Action `json:"switchover,omitempty"`
}

// MemberUpdateStrategy defines Cluster Component update strategy.
// +enum
type MemberUpdateStrategy string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for removing MemberUpdateStrategy here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And in the current design, how can we ensure that during the upgrade and deletion process, we first operate on the slave instance and then on the master instance?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the Addon has defined roles, it will still be updated according to the roles, the behavior has not changed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the Addon has defined roles, it will still be updated according to the roles, the behavior has not changed.

So this is a built-in logic, there is no switch that can control it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep.


const (
SerialUpdateStrategy MemberUpdateStrategy = "Serial"
BestEffortParallelUpdateStrategy MemberUpdateStrategy = "BestEffortParallel"
ParallelUpdateStrategy MemberUpdateStrategy = "Parallel"
)

type MemberStatus struct {
// Represents the name of the pod.
//
Expand Down
16 changes: 10 additions & 6 deletions apis/workloads/v1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading