OCPBUGS-60656: OCPBUGS-60883: Fix PinnedImageSet test instability & update flow for tests using custom MCPs #30136
Conversation
openshift-ci
bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
openshift-ci
bot
added
the
approved
|
/payload-job periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-virtualmedia-1of2 |
|
@isabella-janssen: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/edd38480-7d60-11f0-8469-01cddc550bca-0 |
|
/payload-aggregate periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-virtualmedia-1of2 6 |
|
@isabella-janssen: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/0050bf10-7dc0-11f0-8ae8-fa73f2d591b7-0 |
isabella-janssen
changed the title
openshift-ci-robot
added
jira/severity-important
|
@isabella-janssen: This pull request references Jira Issue OCPBUGS-60591, which is valid. The bug has been moved to the POST state. 3 validation(s) were run on this bug
Requesting review from QA contact: The bug has been updated to refer to the pull request using the external bug tracker. In response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@isabella-janssen: This pull request references Jira Issue OCPBUGS-60591, which is valid. 3 validation(s) were run on this bug
Requesting review from QA contact: In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/payload-job periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-serial periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview-serial periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node-serial periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-ipv4 periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-virtualmedia-1of2 periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-serial-ovn-ipv6 |
|
@isabella-janssen: trigger 6 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b4b2c220-82b9-11f0-9e68-434bb60f536a-0 |
|
/payload-job periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-serial periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview-serial periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node-serial periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-ipv4 periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-virtualmedia-1of2 periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-serial-ovn-ipv6 |
|
@isabella-janssen: trigger 6 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a6c09f00-834c-11f0-8512-19b9cb7e79c6-0 |
isabella-janssen
changed the title
openshift-ci-robot
added
jira/invalid-bug
|
@isabella-janssen: This pull request references Jira Issue OCPBUGS-60656, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
openshift-ci-robot
added
jira/valid-bug
|
@isabella-janssen: This pull request references Jira Issue OCPBUGS-60656, which is valid. The bug has been moved to the POST state. 3 validation(s) were run on this bug
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@isabella-janssen: This pull request references Jira Issue OCPBUGS-60656, which is valid. 3 validation(s) were run on this bug
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
isabella-janssen
force-pushed
the
ocpbugs-60591
branch
from
c9ae593 to
4184a85
Compare
isabella-janssen
force-pushed
the
ocpbugs-60591
branch
2 times, most recently
from
40f738e to
c18f1c2
Compare
|
/payload-job periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-serial periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview-serial periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node-serial periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-ipv4 periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-virtualmedia-1of2 periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-serial-ovn-ipv6 |
|
@isabella-janssen: trigger 6 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/8a4135a0-837f-11f0-8aac-7e01574679ba-0 |
isabella-janssen
changed the title
openshift-ci
bot
removed
the
do-not-merge/work-in-progress
isabella-janssen
force-pushed
the
ocpbugs-60591
branch
from
c18f1c2 to
6e3de92
Compare
|
/payload-job periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-serial periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-techpreview-serial periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node-serial periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-ipv4 periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-ovn-serial-virtualmedia-1of2 periodic-ci-openshift-release-master-nightly-4.20-e2e-metal-ipi-serial-ovn-ipv6 |
|
@isabella-janssen: trigger 6 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/9a070e20-84eb-11f0-9b76-4159ddbe3187-0 |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: isabella-janssen, pablintino The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@isabella-janssen: This pull request references Jira Issue OCPBUGS-60656, which is valid. 3 validation(s) were run on this bug
The bug has been updated to refer to the pull request using the external bug tracker. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Job Failure Risk Analysis for sha: 6e3de92
|
|
/label acknowledge-critical-fixes-only This bug fix is necessary for stabilizing the MCO's component readiness. Additionally, these fixes have been validated in payload rehearsals, so this change should be safe to merge. |
openshift-ci
bot
added
the
acknowledge-critical-fixes-only
|
/retest-required |
1 similar comment
|
@isabella-janssen: all tests passed! Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
openshift-merge-bot
bot
merged commit f90f90a
into
openshift:main
|
@isabella-janssen: Jira Issue OCPBUGS-60656: All pull requests linked via external trackers have merged: Jira Issue OCPBUGS-60656 has been moved to the MODIFIED state. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/cherrypick release-4.19 |
|
@isabella-janssen: new pull request created: #30205 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
This addresses two bugs to improve the performance of the PinnedImageSet tests.
OCPBUGS-60656:
This bug relates to instability seen in the
Invalid PIS leads to degraded MCN in a standard Pooltest on thevirtualmediatest variant. To stabilize this test, the verification of the PIS degrade was updated to include a retrying wait to validate MCN conditions are correctly set.OCPBUGS-60883:
This bug relates to flaws associated with the three PIS tests making use of custom MCPs,
All Nodes in a custom Pool should have the PinnedImages even after Garbage Collection,All Nodes in a Custom Pool should have the PinnedImages in PIS, andInvalid PIS leads to degraded MCN in a custom Pool. To address the issue in these tests, a wait was added after the worker node labeling to ensure the node successfully becomes part of the custom MCP before proceeding with the test.General:
In general, some more comments and enhanced error messages were added to help with developer experience in maintaining and monitoring these tests.
Verifying changes:
I was not able to reproduce the issue for OCPBUGS-60656 locally. However, the changes in this PR should make the failing test more stable in general. The following payload tests were run with the changes and the PinnedImages tests were successful & took a more reasonable amount of time to complete than previously: