OCPBUGS-28647: tuned: operand: add support for deferred updates #1019

ffromani · 2024-04-04T10:14:01Z

Users are able to make tuning changes through NTO/TuneD by updating the CRs on a running system.
When the tuning change is done, the NTO sends a SIGHUP to the tuned process, which results in the tuned process rolling back ALL the tuning and then reapplying it.
In some key use cases, this disrupts the workload behavior.

We add an annotation that cause the update not to be applied immediately, but to be deferred till the next tuned restart. The NTO operator will continue merging the tuned objects into the special rendered object as it does today.
The deferred annotation will be "sticky". If even just one of the current tuned object (e.g. just the perfprofile one) have the annotation, the rendered tuned will.

However, if there is a deferred updated pending but the tuned objects are edited in such a way the final rendered update will be immediate (= no deferred annotation), then the immediate update will be applied overwriting the pending deferred update, which will then be lost.

The deferred updates can stack, meaning a deferred update can be applied on top of a previous deferred update; the latter will just overwrite the former as it never was received. An immediate update can overwrite a deferred update, and clear it.

Like the immediate update, the NTO operand will take care of applying the deferred updates, as such:

The NTO operand will gain another chance to process tuned objects at startup, right before the main loops start.

If a tuned object is detected and it has NOT the deferred annotation:

the regular reconcile code will work as today
the startup reconcile code will not engage

If a tuned object is detected and it DOES HAVE the deferred annotation:

the regular reconcile code will set a filesystem flag, will move the status DEGRADED and will ignore the object.
at the first followup operand restart, the startup reconcile code will notice the filesystem flag, apply the rendered tuned objects (granted it DOES have the deferred flag, otherwise it will log and skip), clear the filesystem flag. The DEGRADED state will be cleared.

IOW, the startup reconcile code will engage IFF the rendered tuned has the deferred annotation and the filesystem flag does exist. Likewise, the regular reconcile loop will only set the filesystem flag if detects a rendered tuned with the deferred annotation.

The default is obviously kept to immediate updates, for backward compatibility.

openshift-ci-robot · 2024-04-04T10:14:06Z

@ffromani: This pull request references Jira Issue OCPBUGS-28647, which is invalid:

expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Users are able to make tuning changes through NTO/TuneD by updating the CRs on a running system.
When the tuning change is done, the NTO sends a SIGHUP to the tuned process, which results in the tuned process rolling back ALL the tuning and then reapplying it.
In some key use cases, this disrupts the workload behavior.

We add an annotation that cause the update not to be applied immediately, but to be deferred till the next tuned restart. The NTO operator will continue merging the tuned objects into the special rendered object as it does today.
The deferred annotation will be "sticky". If even just one of the current tuned object (e.g. just the perfprofile one) have the annotation, the rendered tuned will.

However, if there is a deferred updated pending but the tuned objects are edited in such a way the final rendered update will be immediate (= no deferred annotation), then the immediate update will be applied overwriting the pending deferred update, which will then be lost.

The deferred updates can stack, meaning a deferred update can be applied on top of a previous deferred update; the latter will just overwrite the former as it never was received. An immediate update can overwrite a deferred update, and clear it.

Like the immediate update, the NTO operand will take care of applying the deferred updates, as such:

The NTO operand will gain another chance to process tuned objects at startup, right before the main loops start.

If a tuned object is detected and it has NOT the deferred annotation:

the regular reconcile code will work as today

the startup reconcile code will not engage

If a tuned object is detected and it DOES HAVE the deferred annotation:

the regular reconcile code will set a filesystem flag, will move the status DEGRADED and will ignore the object.

at the first followup operand restart, the startup reconcile code will notice the filesystem flag, apply the rendered tuned objects (granted it DOES have the deferred flag, otherwise it will log and skip), clear the filesystem flag. The DEGRADED state will be cleared.

IOW, the startup reconcile code will engage IFF the rendered tuned has the deferred annotation and the filesystem flag does exist. Likewise, the regular reconcile loop will only set the filesystem flag if detects a rendered tuned with the deferred annotation.

The default is obviously kept to immediate updates, for backward compatibility.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

ffromani · 2024-04-04T10:15:04Z

/jira refresh

openshift-ci-robot · 2024-04-04T10:15:10Z

@ffromani: This pull request references Jira Issue OCPBUGS-28647, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.16.0) matches configured target version for branch (4.16.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @gsr-shanks

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2024-04-04T10:15:14Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ffromani

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ffromani]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ffromani · 2024-05-09T06:40:49Z

/retest

ffromani · 2024-05-09T09:02:00Z

 ERRO[2024-05-09T07:59:44Z] 
  * could not run steps: step e2e-gcp-pao failed: "e2e-gcp-pao" pre steps failed: "e2e-gcp-pao" pod "e2e-gcp-pao-ipi-install-install" failed: could not watch pod: the pod ci-op-t8wxg3mq/e2e-gcp-pao-ipi-install-install failed after 1h0m10s (failed containers: test): ContainerFailed one or more containers exited

stil looks as infra issue to me.

ffromani · 2024-05-10T09:30:11Z

the PR is now reviewable

add support to report the deferred status in conditions. Signed-off-by: Francesco Romani <[email protected]>

TunedProfiles -> tunedProfiles trivial rename. The function was not used elsewhere anyway. Signed-off-by: Francesco Romani <[email protected]>

rework profilesExtract and helper functions to make more testable, enable round-tripping, and add helpers which will be used in the deferred update functionality. Signed-off-by: Francesco Romani <[email protected]>

Simplify the interface packing the many return values in a single struct, to enable further extensibility. No intended changes in behavior. Signed-off-by: Francesco Romani <[email protected]>

Gather the node name once and use it in the lifecycle of the controller - is not supposed to change anyway. Bubble up the client creation, so we don't need to keep around the kubeconfig. Signed-off-by: Francesco Romani <[email protected]>

Align to left for readability with no intended changes in behavior. Signed-off-by: Francesco Romani <[email protected]>

Read extracted profiles on startup and reconstruct the node state, to be used ro implement the deferred updates feature. Please note all the operations are non-destructive read-only so pretty safe. Signed-off-by: Francesco Romani <[email protected]>

The deferred updates feature need to do specific actions when the tuned daemon is reloaded (and only then). Add hook point with specific code for that. Signed-off-by: Francesco Romani <[email protected]>

Rework how tuned controller updates status to make room for the deferred update feature. Signed-off-by: Francesco Romani <[email protected]>

Now that all the pieces are in place, we can set and use the deferred update flag to implement the desired behavior. Signed-off-by: Francesco Romani <[email protected]>

clarify the logs when the tuned controller reacts to a write event. Signed-off-by: Francesco Romani <[email protected]>

Trigger an event when the recommend file is written, and always do that even if does not change. This is needed to react properly when a deferred update is un-deferred by editing the tuned object. Without a trigger in this state the k8s object status wouldn't be correctly updated. Signed-off-by: Francesco Romani <[email protected]>

Signed-off-by: Francesco Romani <[email protected]>

MarSik · 2024-05-22T16:03:08Z

@ffromani Will this apply the profile even when you just kill the tuned pod on a node? Or is there logic to wait for clean reboot (say using a /var/run/ file or its absence)?

ffromani · 2024-05-22T17:34:28Z

@ffromani Will this apply the profile even when you just kill the tuned pod on a node? Or is there logic to wait for clean reboot (say using a /var/run/ file or its absence)?

we will apply the profile on each tuned restart, not on node reboot

jmencak · 2024-05-22T20:21:36Z

/retest

jmencak · 2024-05-23T05:56:51Z

/retest

ffromani · 2024-05-23T05:58:50Z

level=error msg=These cluster operators were not stable: [node-tuning]

I wonder if my changes are causing this CI failure. Testing on my cluster revelead no issue whatsoever though.
I'll run a bisect with a test PR

jmencak · 2024-05-23T06:06:25Z

level=error msg=These cluster operators were not stable: [node-tuning]

I wonder if my changes are causing this CI failure. Testing on my cluster revelead no issue whatsoever though. I'll run a bisect with a test PR

Good find. Yeah, I suspect updates or flips with ClusterOperator/node-tuning. I've noticed in the past that simple PRs to this repo passed, whereas this bigger change was failing multiple tests.

ffromani · 2024-05-23T06:20:45Z

level=error msg=These cluster operators were not stable: [node-tuning]

I wonder if my changes are causing this CI failure. Testing on my cluster revelead no issue whatsoever though. I'll run a bisect with a test PR

#1065

openshift-ci · 2024-05-23T07:27:44Z

@ffromani: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-operator	`5672523`	link	true	`/test e2e-aws-operator`
ci/prow/e2e-aws-ovn	`5672523`	link	true	`/test e2e-aws-ovn`
ci/prow/e2e-gcp-pao	`5672523`	link	true	`/test e2e-gcp-pao`
ci/prow/e2e-gcp-pao-workloadhints	`5672523`	link	true	`/test e2e-gcp-pao-workloadhints`
ci/prow/e2e-gcp-pao-updating-profile	`5672523`	link	true	`/test e2e-gcp-pao-updating-profile`
ci/prow/e2e-hypershift	`5672523`	link	true	`/test e2e-hypershift`
ci/prow/e2e-aws-ovn-techpreview	`5672523`	link	true	`/test e2e-aws-ovn-techpreview`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

ffromani · 2024-05-23T07:29:26Z

/hold

let's stop retesting until #1065 sheds some light here

jmencak · 2024-05-23T07:43:53Z

@ffromani Will this apply the profile even when you just kill the tuned pod on a node? Or is there logic to wait for clean reboot (say using a /var/run/ file or its absence)?

we will apply the profile on each tuned restart, not on node reboot

From my testing, yes, if you delete the TuneD pod, the current code as is will apply the new profile even though it has the deferred annotation set. It will, however refuse to update a current profile when you create a new Tuned CR with deferred annotation set.

So far, I've noticed one issue. If I reboot the node and the system has a profile with deferred annotation set, I still see:

$ oc get co/node-tuning
NAME          VERSION                              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
node-tuning   4.16.0-0.nightly-2024-05-21-221942   True        True          False      16m     Waiting for 1/1 Profiles to be applied

even though the profile has been successfully applied.

ffromani · 2024-05-23T07:47:31Z

So far, I've noticed one issue. If I reboot the node and the system has a profile with deferred annotation set, I still see:
$ oc get co/node-tuning
NAME          VERSION                              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
node-tuning   4.16.0-0.nightly-2024-05-21-221942   True        True          False      16m     Waiting for 1/1 Profiles to be applied
even though the profile has been successfully applied.

thanks, this is new. I'll check once I get rid of the CI woes

jmencak · 2024-05-23T08:01:02Z

thanks, this is new. I'll check once I get rid of the CI woes

I wonder if that could be related. CI doesn't like things like CO objects progressing, etc. Another thing is whether we want to block the update until the next reboot, i.e. what @MarSik hinted at. Should the pod deletion apply the deferred update?

ffromani · 2024-05-23T08:25:12Z

thanks, this is new. I'll check once I get rid of the CI woes

I wonder if that could be related. CI doesn't like things like CO objects progressing, etc. Another thing is whether we want to block the update until the next reboot, i.e. what @MarSik hinted at. Should the pod deletion apply the deferred update?

This needs to be verified (by yours truly) but the failure seems into the early stage of CI setup, well before the e2e tests run. And we create profiles with the deferred annotation only in the newly added e2e tests.

jmencak

This is great work. I like the changes. I found only a few nits so far, but I haven't finished the full review, it is a lot of code. Sharing a part of it.

jmencak · 2024-05-23T09:16:52Z

pkg/tuned/controller.go

 return profileData == string(content)
 }

 // ProfilesExtract extracts TuneD daemon profiles to tunedProfilesDirCustom directory.
 // Returns:
 // - True if the data in the to-be-extracted recommended profile or the profiles being
 // included from the current recommended profile have changed.
+// - If the data changed, the fingerprint of the new profile, or "" otherwise


SuperNit: Let's be consistent.

s/or "" otherwise/or "" otherwise./

Similarly elsewhere, for example in the Daemon/Change struct. I try to add punctuation when a line starts a comment and feels like a sentence/paragraph. I typically don't add it for very short comments that don't feel like a sentence. These are typically on the same line with a code.

jmencak · 2024-05-23T09:56:01Z

pkg/tuned/controller.go

- recommendedProfileDeps[recommendedProfile] = true
- }
- extracted := map[string]bool{} // TuneD profile names present in TuneD CR and successfully extracted to tunedProfilesDirCustom
+ if len(recommendedProfile) == 0 {


This is a "heritage" from my previous code, but I believe that this check should be moved to tuned_parser.go . Then, we can probably remove this function completely and just move the assignment

// Add the recommended profile itself. recommendedProfileDeps[recommendedProfile] = true

If you agree, then, there'll be little point to have profilesExtractPathRecommends() too and we can have the whole code in ProfilesExtract() as it was before and there will be no need to document/comment/explain the new functions.

jmencak · 2024-05-23T11:31:46Z

pkg/tuned/controller.go

+ return change, profilesFP, nil
+}
+
+func profilesFingerprint(profiles []tunedv1.TunedProfile, recommendedProfile string) string {


I believe this function deserves a comment. Something along the lines of:

// profilesFingerprint returns a hash of `recommendedProfile` name joined with the data sections of all TuneD profiles in the `profiles` slice.

jmencak · 2024-05-23T11:33:20Z

pkg/tuned/controller.go

+func profilesFingerprint(profiles []tunedv1.TunedProfile, recommendedProfile string) string {
+ h := sha256.New()
+ h.Write([]byte(recommendedProfile))
+ for _, prof := range profiles {


Do we expect any issues with profile ordering? Will they always come ordered the same way, e.g. profile a prior to profile b and not vice versa the next time?

jmencak · 2024-05-23T11:43:32Z

pkg/tuned/controller.go

+ return change, profilesFP, extracted, recommendedProfileDeps, nil
+}
+
+func profilesRepackPath(rfPath, profilesRootDir string) ([]tunedv1.TunedProfile, string, error) {


Why do we need rfPath, cannot we just use tunedRecommendFile ? Similarly profilesRootDir.

Is the name Perl inspired by pack/unpack functions? I believe this function deserves a comment and possibly even function renaming. Comment something along the lines of:

profiles(Re)[Pp]ackFromPath reconstructs the slice of TunedProfile objects out of the TuneD configuration files from `tunedProfilesDirCustom`.

jmencak · 2024-05-23T12:02:58Z

pkg/tuned/controller.go

+ return profiles, recommendedProfile, nil
+}
+
+func ProfilesRepack() ([]tunedv1.TunedProfile, string, error) {


Do we need to export this? Also, do we need such a "shallow" function?

jmencak · 2024-05-23T13:26:47Z

pkg/tuned/controller.go

- klog.Error(err.Error())
- return false // retry later
- }
+func (c *Controller) changeSyncerPostReload(change Change) bool {


I believe we need some comments to describe the general purpose of this function. Possibly even adjust its name to changeSyncerPostDaemonReload ?

jmencak · 2024-05-23T13:27:32Z

pkg/tuned/controller.go

 }

+ klog.Infof("changeSyncerPostReload(): current effective profile fingerprint %q -> %q", c.daemon.profileFingerprintEffective, profileFP)


Should we log this every time even though the fingerprint hasn't changed? I.e. does it bring any value or are we just poluting logs?

jmencak · 2024-05-23T13:43:24Z

pkg/tuned/controller.go

+ klog.Infof("changeSyncer(%s)", change.String()) // TODO v=2
+ defer klog.Infof("changeSyncer(%s) done", change.String()) // TODO v=2
+ // Sync internal status after a TuneD reload
+ _ = c.changeSyncerPostReload(change)


Do we need the return value for anything?

jmencak

Adding a few more comments. In my view, the main things that need to be resolved:

CI issues
As agreed over Slack, the pod restart should not apply a new profile when deferred annotation is set.
co/node-tuning status

After that, I'll do another round of review and testing.

jmencak · 2024-05-27T12:26:01Z

pkg/tuned/status.go

@@ -88,7 +88,7 @@ func InitializeStatusConditions() []tunedv1.ProfileStatusCondition {
 // 'status' contains all the information necessary for creating a new slice of
 // conditions apart from LastTransitionTime, which is set based on checking the
 // old conditions.
-func computeStatusConditions(status Bits, stderr string, conditions []tunedv1.ProfileStatusCondition) []tunedv1.ProfileStatusCondition {
+func computeStatusConditions(status Bits, message string, conditions []tunedv1.ProfileStatusCondition) []tunedv1.ProfileStatusCondition {


An additional parameter was added to the function. Let's explain what it does by updating the comments above.

jmencak · 2024-05-27T12:34:02Z

pkg/tuned/controller.go

+ }
+
+ var message string
+ deferred := (change.deferred || util.HasDeferredUpdateAnnotation(profile.Annotations))


Do we need both or is util.HasDeferredUpdateAnnotation(profile.Annotations) enough?

jmencak · 2024-05-27T18:38:48Z

pkg/tuned/controller.go

- klog.V(1).Infof("write event on: %s", fsEvent.Name)
- // Notify the event processor that the TuneD daemon calculated new kernel command-line parameters.
+ klog.Infof("write event on: %s", fsEvent.Name)
+ // Notify the event processor that profiles where unpacked on the node or


This is a comment which is slightly misleading. We're watching writes to bootcmdline and recommend file, not unpacking profiles on the node. They are unpacked, but that's not why the write event was triggered.

jmencak · 2024-05-27T18:42:21Z

pkg/tuned/controller.go

 }

+ klog.Infof("changeSyncerPostReload(): current effective profile fingerprint %q -> %q", c.daemon.profileFingerprintEffective, profileFP)
+ c.daemon.profileFingerprintEffective = profileFP
+ c.daemon.status &= ^scDeferred // force clear even if it was never set. No biggie.


Can we drop the drop the familiar "No biggie."?

jmencak · 2024-05-27T18:55:22Z

pkg/tuned/status.go

+ } else if (status & scDeferred) != 0 {
+ tunedDegradedCondition.Status = corev1.ConditionTrue
+ tunedDegradedCondition.Reason = "TunedDeferredUpdate"
+ tunedDegradedCondition.Message = "Profile will be applied at the next daemon restart" + deferredMessage


We should probably be consistent here

s/next daemon restart/next daemon restart: /

jmencak · 2024-05-27T20:00:21Z

test/e2e/deferred/basic.go

@@ -0,0 +1,407 @@
+package e2e


Do the test in e2e/deferred/ actually run? I don't see the Makefile adjusted so that they run automatically.

openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 4, 2024

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 4, 2024

openshift-ci bot requested review from MarSik and yanirq April 4, 2024 10:15

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 4, 2024

openshift-ci bot requested a review from gsr-shanks April 4, 2024 10:15

ffromani changed the title ~~WIP: OCPBUGS-28647: tuned: operand: add support for deferred updates~~ WIP: POC: OCPBUGS-28647: tuned: operand: add support for deferred updates Apr 4, 2024

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 4, 2024

ffromani force-pushed the tuned-deferred-updates branch from ab08d68 to 7a63c46 Compare April 4, 2024 12:15

ffromani mentioned this pull request Apr 16, 2024

OCPBUGS-32469: Remove tuned/rendered object #1036

Merged

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 20, 2024

ffromani force-pushed the tuned-deferred-updates branch from 7a63c46 to 30cdab0 Compare April 30, 2024 11:58

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 30, 2024

ffromani force-pushed the tuned-deferred-updates branch from 30cdab0 to 5376eac Compare April 30, 2024 14:08

ffromani force-pushed the tuned-deferred-updates branch 4 times, most recently from f0b7ece to 2314db4 Compare May 8, 2024 17:19

ffromani force-pushed the tuned-deferred-updates branch 2 times, most recently from 04635db to f0873a9 Compare May 10, 2024 09:29

ffromani changed the title ~~WIP: POC: OCPBUGS-28647: tuned: operand: add support for deferred updates~~ OCPBUGS-28647: tuned: operand: add support for deferred updates May 10, 2024

ffromani added 13 commits May 22, 2024 17:15

status: report deferred status

ff2d6a3

add support to report the deferred status in conditions. Signed-off-by: Francesco Romani <[email protected]>

operator: unexport TunedProfiles

ae3ed9e

TunedProfiles -> tunedProfiles trivial rename. The function was not used elsewhere anyway. Signed-off-by: Francesco Romani <[email protected]>

tuned: controller: profiles extract/repack

6867299

rework profilesExtract and helper functions to make more testable, enable round-tripping, and add helpers which will be used in the deferred update functionality. Signed-off-by: Francesco Romani <[email protected]>

operator: profilecalculator: pack return values in a struct

bbf5f0f

Simplify the interface packing the many return values in a single struct, to enable further extensibility. No intended changes in behavior. Signed-off-by: Francesco Romani <[email protected]>

tuned: controller: simplify flow

054095d

Gather the node name once and use it in the lifecycle of the controller - is not supposed to change anyway. Bubble up the client creation, so we don't need to keep around the kubeconfig. Signed-off-by: Francesco Romani <[email protected]>

tuned: changeSyncerProfileStatus align to left

d9afec9

Align to left for readability with no intended changes in behavior. Signed-off-by: Francesco Romani <[email protected]>

tuned: profilesRepack on startup

f0336c4

Read extracted profiles on startup and reconstruct the node state, to be used ro implement the deferred updates feature. Please note all the operations are non-destructive read-only so pretty safe. Signed-off-by: Francesco Romani <[email protected]>

tuned: add changeSyncerPostReload

77bc910

The deferred updates feature need to do specific actions when the tuned daemon is reloaded (and only then). Add hook point with specific code for that. Signed-off-by: Francesco Romani <[email protected]>

tuned: refactor status update

b374cab

Rework how tuned controller updates status to make room for the deferred update feature. Signed-off-by: Francesco Romani <[email protected]>

tuned: set the deferred flag

1e164ad

Now that all the pieces are in place, we can set and use the deferred update flag to implement the desired behavior. Signed-off-by: Francesco Romani <[email protected]>

tuned: review logging

27ce434

clarify the logs when the tuned controller reacts to a write event. Signed-off-by: Francesco Romani <[email protected]>

assets: update tuned snapshot

5672523

Signed-off-by: Francesco Romani <[email protected]>

ffromani force-pushed the tuned-deferred-updates branch from 747e901 to 5672523 Compare May 22, 2024 15:21

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 23, 2024

jmencak reviewed May 23, 2024

View reviewed changes

jmencak reviewed May 27, 2024

View reviewed changes

		}

		klog.Infof("changeSyncerPostReload(): current effective profile fingerprint %q -> %q", c.daemon.profileFingerprintEffective, profileFP)

OCPBUGS-28647: tuned: operand: add support for deferred updates #1019

Are you sure you want to change the base?

OCPBUGS-28647: tuned: operand: add support for deferred updates #1019

Conversation

ffromani commented Apr 4, 2024

openshift-ci-robot commented Apr 4, 2024

ffromani commented Apr 4, 2024

openshift-ci-robot commented Apr 4, 2024

openshift-ci bot commented Apr 4, 2024

ffromani commented May 9, 2024

ffromani commented May 9, 2024

ffromani commented May 10, 2024

MarSik commented May 22, 2024

ffromani commented May 22, 2024

jmencak commented May 22, 2024

jmencak commented May 23, 2024

ffromani commented May 23, 2024

jmencak commented May 23, 2024

ffromani commented May 23, 2024

openshift-ci bot commented May 23, 2024

ffromani commented May 23, 2024

jmencak commented May 23, 2024

ffromani commented May 23, 2024

jmencak commented May 23, 2024

ffromani commented May 23, 2024

jmencak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmencak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment