CNTRLPLANE-945: improve logging, add startup probe to Keycloak Deployment

[everettraven]

We noticed in a few job runs that our tests were failing due to Keycloak Deployment availability checks timing out.

This PR:

• Updates our logging behavior to tie into the Ginkgo logr.Logger implementation so we can avoid using standard library print statements for additional debugging logic
• Adds a startup probe for the Keycloak Deployment. It looks like our availability issues stemmed from the liveness/readiness probes hitting connection refused errors because Keycloak hadn't successfully started yet. This lead to Kubelet restarting the Keycloak Pod indefinitely. Chose a ~200 second startup window for Keycloak before it gets restarted and this seems to have resolved our issues.

[openshift-ci-robot]

@everettraven: This pull request references CNTRLPLANE-945 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[everettraven]

/hold

[everettraven]

/payload-aggregate periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview 10

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/383244d0-8356-11f0-96bf-b354a4277ec6-0

[everettraven]

/payload-aggregate periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-configure-techpreview 10

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-configure-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/60b24f90-8356-11f0-9950-efea8bdabbd2-0

[everettraven]

/payload-job periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview

[everettraven]

/payload-job periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-configure-techpreview

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/c0a057a0-8363-11f0-86d8-6d93f003bb3d-0

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-configure-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/c7e96c90-8363-11f0-96f4-ffe5ba70a924-0

[everettraven]

/payload-job periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-configure-techpreview

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-configure-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/8e52f9e0-836a-11f0-85f3-e31393781287-0

[everettraven]

/payload-job periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/39887500-8411-11f0-8611-e6ba04d74a76-0

[everettraven]

/payload-aggregate periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview 5

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/48fdd2f0-8411-11f0-996e-0929bfc43bd8-0

[everettraven]

/payload-aggregate periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview 10

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-azure-external-oidc-rollback-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/db8fde10-8434-11f0-9c69-64695eb201e2-0

[everettraven]

/payload-aggregate periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-configure-techpreview 10

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-configure-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/e9dc2550-8434-11f0-8078-a62d61f4c0fe-0

[openshift-trt]

Job Failure Risk Analysis for sha: 7c333fe

Job Name	Failure Risk
pull-ci-openshift-origin-main-e2e-aws-disruptive	IncompleteTests Tests for this run (106) are below the historical average (948): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[liouk]

/retest-required

[liouk]

/lgtm

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-uid-extra-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/f4e46a00-8585-11f0-815a-6a9dda982413-0

[everettraven]

/payload-aggregate periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-uid-extra-techpreview 10

[openshift-ci]

@everettraven: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-cluster-authentication-operator-release-4.20-periodics-e2e-gcp-external-oidc-uid-extra-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/1cdf7df0-85e1-11f0-895a-dc5e89906d8c-0

[everettraven]

Had an interesting flake I couldn't reproduce over the last 20 runs. Verified that these changes work fine.

/hold cancel

[liouk]

/lgtm

[liouk]

/label acknowledge-critical-fixes-only

[liouk]

/lgtm

[liouk]

/label acknowledge-critical-fixes-only

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: everettraven, liouk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• test/extended/authentication/OWNERS [everettraven,liouk]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[liouk]

/retest-required

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD f90f90a and 2 for PR HEAD 8c81b4e in total

[openshift-trt]

Job Failure Risk Analysis for sha: 8c81b4e

Job Name	Failure Risk
pull-ci-openshift-origin-main-e2e-openstack-ovn	IncompleteTests Tests for this run (21) are below the historical average (2754): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[openshift-trt]

Job Failure Risk Analysis for sha: 8c81b4e

Job Name	Failure Risk
pull-ci-openshift-origin-main-e2e-openstack-ovn	IncompleteTests Tests for this run (21) are below the historical average (2754): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[kevinrizza]

/retest-required

[openshift-trt]

Job Failure Risk Analysis for sha: 8c81b4e

Job Name	Failure Risk
pull-ci-openshift-origin-main-e2e-openstack-ovn	IncompleteTests Tests for this run (21) are below the historical average (2632): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[liouk]

/test e2e-metal-ipi-ovn-ipv6

[openshift-trt]

Job Failure Risk Analysis for sha: 8c81b4e

Job Name	Failure Risk
pull-ci-openshift-origin-main-e2e-openstack-ovn	IncompleteTests Tests for this run (21) are below the historical average (2558): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New Test Risks for sha: 8c81b4e

Job Name	New Test Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-fips	High - "Find all of the input images from ocp/4.21 and tag them into the stable stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-fips	High - "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-fips	High - "[Jira:storage-version-migrator][sig-api-machinery] sanity test should always pass [Suite:openshift/cluster-kube-storage-version-migrator-operator/conformance/parallel]" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift	High - "Find all of the input images from ocp/4.21 and tag them into the stable stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift	High - "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift	High - "[Jira:storage-version-migrator][sig-api-machinery] sanity test should always pass [Suite:openshift/cluster-kube-storage-version-migrator-operator/conformance/parallel]" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift-serial	High - "Find all of the input images from ocp/4.21 and tag them into the stable stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift-serial	High - "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-serial-1of2	High - "Find all of the input images from ocp/4.21 and tag them into the stable stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-serial-1of2	High - "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-serial-2of2	High - "Find all of the input images from ocp/4.21 and tag them into the stable stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-aws-ovn-serial-2of2	High - "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn	High - "Find all of the input images from ocp/4.21 and tag them into the stable stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn	High - "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn	High - "[Jira:storage-version-migrator][sig-api-machinery] sanity test should always pass [Suite:openshift/cluster-kube-storage-version-migrator-operator/conformance/parallel]" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-vsphere-ovn	High - "Find all of the input images from ocp/4.21 and tag them into the stable stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-vsphere-ovn	High - "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-vsphere-ovn	High - "[Jira:storage-version-migrator][sig-api-machinery] sanity test should always pass [Suite:openshift/cluster-kube-storage-version-migrator-operator/conformance/parallel]" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-vsphere-ovn-upi	High - "Find all of the input images from ocp/4.21 and tag them into the stable stream" is a new test that was not present in all runs against the current commit.
pull-ci-openshift-origin-main-e2e-vsphere-ovn-upi	High - "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" is a new test that was not present in all runs against the current commit.
	(...showing 20 of 21 rows)

New tests seen in this PR at sha: 8c81b4e

• "Find all of the input images from ocp/4.21 and tag them into the stable stream" [Total: 19, Pass: 19, Fail: 0, Flake: 0]
• "Find all of the input images from ocp/4.21 and tag them into the stable-initial stream" [Total: 19, Pass: 19, Fail: 0, Flake: 0]
• "[Jira:storage-version-migrator][sig-api-machinery] sanity test should always pass [Suite:openshift/cluster-kube-storage-version-migrator-operator/conformance/parallel]" [Total: 11, Pass: 11, Fail: 0, Flake: 0]

[everettraven]

/verified by @everettraven

[openshift-ci-robot]

@everettraven: This PR has been marked as verified by @everettraven.

In response to this:

/verified by @everettraven

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[everettraven]

/test e2e-gcp-ovn-upgrade

[openshift-ci]

@everettraven: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-trt]

Job Failure Risk Analysis for sha: 8c81b4e

Job Name	Failure Risk
pull-ci-openshift-origin-main-e2e-openstack-ovn	IncompleteTests Tests for this run (21) are below the historical average (2558): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[everettraven]

/cherry-pick release-4.20

[openshift-cherrypick-robot]

@everettraven: new pull request created: #30204

In response to this:

/cherry-pick release-4.20

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.