Skip to content

Commit 7e4b3db

Browse files
jcanocanjcanocan
committed
design-proposal: Feature configurables
This design document states how features that require to have a mechanism to change it's state, e.g., enabled/disabled should be implemented in KubeVirt. Signed-off-by: Javier Cano Cano <[email protected]>
1 parent db2ea07 commit 7e4b3db

File tree

1 file changed

+155
-0
lines changed

1 file changed

+155
-0
lines changed

‎design-proposals/configurable-features.md

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Overview
2+
3+
With the introduction
4+
of [KubeVirt Feature Lifecycle](https://github.com/kubevirt/community/blob/main/design-proposals/feature-lifecycle.md)
5+
policy, features reaching General Availability (GA) need to drop their use of feature gates. This applies also to
6+
configurable features that we may want to disable.
7+
8+
## Motivation
9+
10+
Users or developers may want certain features to be in a given state, for example to make the best use out of given
11+
resources or for compliance reasons features may expose sensitive information from the host to the virtual machines (VM)
12+
or add additional containers to the launcher pod, which are not required by the user. The behavior of other features
13+
might be changed by editing configurables, e.g. the maximum of CPU sockets allowed for each VM can be configured.
14+
15+
Before the introduction
16+
of [KubeVirt Feature Lifecycle](https://github.com/kubevirt/community/blob/main/design-proposals/feature-lifecycle.md)
17+
policy, many feature gates remained after feature's graduation to GA with the sole purpose of acting as a switch for the
18+
feature. Generally speaking, this is a bad practice, because feature gates should be reserved for controlling a feature
19+
until it reaches maturity. i.e., GA. Therefore, in the case that a developer wants to provide the ability to tune/change
20+
the state of the feature, configurables exposed in the KubeVirt CR should be provided. This should be
21+
accomplished while achieving [eventually consistency](https://en.wikipedia.org/wiki/Eventual_consistency). This forces
22+
us to avoid the feature state control checking on webhooks and moving the feature state control closer to the
23+
responsible code. Moreover, it has to be decided how the system should behave if a virtual machine instance (VMi) is
24+
requiring a feature in a state different from what was configured in the KubeVirt CR, or what should happen if the
25+
configuration of a feature in use is changed. (see matrix below).
26+
27+
## Goals
28+
29+
- Establish how the features enablement switch should work.
30+
- Describe how the system should react in these scenarios:
31+
- A feature in KubeVirt is set to state A and a VMi requests the feature to be in state B.
32+
- A feature in KubeVirt is set to state A, there are running VMis using the feature in state A, and the feature is
33+
changed in KubeVirt to state B.
34+
- A feature in KubeVirt is set to state A, and pending VMis want to use it.
35+
- A feature in KubeVirt is set to state A, and running VMis using the feature in state B wants to live migrate.
36+
- Graduate as many features as possible from features gates to configurables.
37+
38+
## Non Goals
39+
40+
- Describe how features protected with features gates should work.
41+
42+
## Definition of Users
43+
44+
Development contributors.
45+
46+
Cluster administrators.
47+
48+
## User Stories
49+
50+
As a developer, I want to make a given feature configurable.
51+
52+
As a cluster administrator, I want to be able to change the cluster wide state of a feature by editing configurables.
53+
54+
As VM owner, I want to use a given feature.
55+
56+
## Repos
57+
58+
Kubevirt/Kubevirt
59+
60+
# Design
61+
62+
If a developer wants to make a feature configurable, he needs to do so by adding new fields to the KubeVirt CR
63+
under `spec.configuration`.
64+
65+
66+
> **NOTE:** The inclusion of these new KubeVirt API fields should be carefully considered and justified. The feature
67+
> configurables should be avoided as much as possible.
68+
69+
70+
This is current list of GA'd features present in KubeVirt/KubeVirt which are still using feature gates and are shown as
71+
configurables in [HCO](https://github.com/kubevirt/hyperconverged-cluster-operator/blob/main/controllers/operands/kubevirt.go#L166-L174):
72+
73+
- DownwardMetrics
74+
- Root (not sure about this one)
75+
- DisableMDEVConfiguration
76+
- PersistentReservation
77+
- AutoResourceLimitsGate
78+
- AlignCPUs
79+
80+
This is the current list of GA'd features present in KubeVirt/KubeVirt which are still using feature gates and are always
81+
enabled by [HCO](https://github.com/kubevirt/hyperconverged-cluster-operator/blob/main/controllers/operands/kubevirt.go#L125-L142):
82+
83+
- CPUManager
84+
- Snapshot
85+
- HotplugVolumes
86+
- GPU
87+
- HostDevices
88+
- NUMA
89+
- VMExport
90+
- DisableCustomSELinuxPolicy
91+
- KubevirtSeccompProfile
92+
- HotplugNICs
93+
- VMPersistentState
94+
- NetworkBindingPlugins
95+
- VMLiveUpdateFeatures
96+
97+
Please note that only feature gates included in KubeVirt/KubeVirt are listed here.
98+
99+
## API Examples
100+
The proposal configuration field, for a given feature in the KubeVirt CR, may look like:
101+
102+
```yaml
103+
apiVersion: kubevirt.io/v1
104+
kind: KubeVirt
105+
[...]
106+
spec:
107+
certificateRotateStrategy: {}
108+
configuration:
109+
feature-A: {}
110+
[...]
111+
```
112+
The VM object may or may not include a configuration field inside the relevant spec.
113+
114+
## Interactions with the VMis requests
115+
116+
In case that, the VM exposes a configuration field to request the feature as well as the KubeVirt CRD, the system may
117+
encounter some inconsistent states that should be handled in the following way:
118+
119+
- If the feature is set to state A in the KubeVirt CR and the VMi is requesting the feature in state B, the VMis must
120+
stay in Pending state. The VM status should be updated, showing a status message, highlighting the reason(s) for the
121+
Pending state.
122+
- Feature status checks should only be performed during the scheduling process, not at runtime. Therefore, the feature
123+
status changes in the KubeVirt CR should not affect running VMis. Moreover, the VMi should still be able to live
124+
migrate, preserving its original feature state.
125+
- Optionally, It could enable the possibility to reject the KubeVirt CR change request if running VMis are using the
126+
feature in a given state. However, by the default the request should be accepted.
127+
128+
## Scalability
129+
130+
The feature state swapping should not affect in a meaningful way the cluster resource usage.
131+
132+
## Update/Rollback Compatibility
133+
134+
The feature enablement should not affect forward or backward compatibility once the feature GA. Before GA, it should
135+
honor [feature stages](https://github.com/kubevirt/community/blob/main/design-proposals/feature-lifecycle.md#releases)
136+
guidelines.
137+
138+
## Functional Testing Approach
139+
140+
The unit and functional testing frameworks should cover the relevant scenarios for each feature.
141+
142+
# Implementation Phases
143+
144+
The feature status check should be placed in the VMi reconciliation loop. In this way, the feature status evaluation is
145+
close to the VMi scheduling process, as well as allowing KubeVirt to reconcile itself if it is out of sync temporally.
146+
147+
Regarding already existing features transitioning from feature gates as a way to set the feature status to configurable
148+
fields, this change is acceptable, but it should be marked as a breaking change and documented. Moreover, all feature
149+
gates should be evaluated to determine if they need to be dropped and transitioned to configurables.
150+
151+
## About implementing the checking logic in the VM controller
152+
153+
The checking in the VM controller could be added to let the user know if a VM has requested a feature in a state which
154+
is different from what it is specified in the KubeVirt CR. The VM will update the VM status, showing a status message
155+
highlighting the misconfiguration.

0 commit comments

Comments
 (0)