Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add commatrix tests #28634

Closed
wants to merge 1 commit into from
Closed

Conversation

liornoy
Copy link

@liornoy liornoy commented Mar 4, 2024

Coverage tests,

Please only consider the second commit: "Refactor commatrix tests"
The first one is kept for reference and will be squashed before merge.

Also - the node-comm-lib will eventually be moved to a more suitable place
e.g. openshift/library-go

@openshift-ci openshift-ci bot requested review from dcbw and jparrill March 4, 2024 12:52
@openshift-ci openshift-ci bot added the vendor-update Touching vendor dir or related files label Mar 4, 2024
@liornoy liornoy force-pushed the add-commatrix-tests branch 2 times, most recently from fae4daa to c7f1dd5 Compare March 4, 2024 14:18
g "github.com/onsi/ginkgo/v2"
o "github.com/onsi/gomega"

"github.com/liornoy/node-comm-lib/commatrix"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any discussion made WRT a suitable host for the code ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're considering moving it to openshift/library-go.
this is pointing to my own repo temporarily just to see first that the library does what we want it to do.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or even under openshift/oc

@openshift-trt-bot
Copy link

Job Failure Risk Analysis for sha: c7f1dd5

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-openstack-ovn IncompleteTests
Tests for this run (98) are below the historical average (1573): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-sdn IncompleteTests
Tests for this run (99) are below the historical average (1547): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6 IncompleteTests
Tests for this run (99) are below the historical average (1211): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-upgrade IncompleteTests
Tests for this run (101) are below the historical average (814): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-rt-upgrade IncompleteTests
Tests for this run (101) are below the historical average (762): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-builds IncompleteTests
Tests for this run (100) are below the historical average (734): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade IncompleteTests
Tests for this run (104) are below the historical average (678): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade IncompleteTests
Tests for this run (100) are below the historical average (2017): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial IncompleteTests
Tests for this run (99) are below the historical average (635): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (99) are below the historical average (1607): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-serial IncompleteTests
Tests for this run (99) are below the historical average (762): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-fips IncompleteTests
Tests for this run (99) are below the historical average (1854): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (99) are below the historical average (1682): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-agnostic-ovn-cmd IncompleteTests
Tests for this run (100) are below the historical average (729): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@liornoy
Copy link
Author

liornoy commented Mar 10, 2024

/retest

@openshift-trt-bot
Copy link

Job Failure Risk Analysis for sha: e8143e7

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-openstack-ovn IncompleteTests
Tests for this run (11) are below the historical average (1352): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-sdn IncompleteTests
Tests for this run (10) are below the historical average (1466): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6 IncompleteTests
Tests for this run (10) are below the historical average (1185): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-upgrade IncompleteTests
Tests for this run (14) are below the historical average (659): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-rt-upgrade IncompleteTests
Tests for this run (14) are below the historical average (572): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-builds IncompleteTests
Tests for this run (13) are below the historical average (656): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn IncompleteTests
Tests for this run (13) are below the historical average (1539): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-csi IncompleteTests
Tests for this run (13) are below the historical average (648): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade IncompleteTests
Tests for this run (14) are below the historical average (609): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade IncompleteTests
Tests for this run (13) are below the historical average (1746): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial IncompleteTests
Tests for this run (12) are below the historical average (616): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (12) are below the historical average (1427): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-serial IncompleteTests
Tests for this run (12) are below the historical average (681): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-fips IncompleteTests
Tests for this run (12) are below the historical average (1660): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (12) are below the historical average (1432): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-csi IncompleteTests
Tests for this run (12) are below the historical average (586): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-agnostic-ovn-cmd IncompleteTests
Tests for this run (13) are below the historical average (632): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@liornoy liornoy force-pushed the add-commatrix-tests branch 2 times, most recently from 6f97a9e to 782a7a8 Compare March 11, 2024 12:12
@openshift-trt-bot
Copy link

Job Failure Risk Analysis for sha: 782a7a8

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-openstack-ovn IncompleteTests
Tests for this run (11) are below the historical average (1276): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-sdn IncompleteTests
Tests for this run (10) are below the historical average (1385): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6 IncompleteTests
Tests for this run (10) are below the historical average (1133): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-upgrade IncompleteTests
Tests for this run (14) are below the historical average (623): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-rt-upgrade IncompleteTests
Tests for this run (14) are below the historical average (539): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn-builds IncompleteTests
Tests for this run (13) are below the historical average (580): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-ovn IncompleteTests
Tests for this run (13) are below the historical average (1468): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-gcp-csi IncompleteTests
Tests for this run (13) are below the historical average (609): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade IncompleteTests
Tests for this run (14) are below the historical average (574): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade IncompleteTests
Tests for this run (13) are below the historical average (1653): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial IncompleteTests
Tests for this run (12) are below the historical average (583): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (12) are below the historical average (1345): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-serial IncompleteTests
Tests for this run (12) are below the historical average (645): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-fips IncompleteTests
Tests for this run (12) are below the historical average (1580): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (12) are below the historical average (1349): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-csi IncompleteTests
Tests for this run (12) are below the historical average (551): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-agnostic-ovn-cmd IncompleteTests
Tests for this run (13) are below the historical average (595): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt-bot
Copy link

Job Failure Risk Analysis for sha: f62655c

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-gcp-ovn-rt-upgrade IncompleteTests
Tests for this run (27) are below the historical average (426): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-agnostic-ovn-cmd IncompleteTests
Tests for this run (26) are below the historical average (521): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@liornoy liornoy force-pushed the add-commatrix-tests branch 2 times, most recently from 1bb2289 to 8a5422a Compare March 14, 2024 17:48
@openshift-trt-bot
Copy link

Job Failure Risk Analysis for sha: 8a5422a

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-openstack-ovn IncompleteTests
Tests for this run (14) are below the historical average (1175): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (15) are below the historical average (1194): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt-bot
Copy link

Job Failure Risk Analysis for sha: a45a87b

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade High
[sig-apps] job-upgrade
This test has passed 100.00% of 27 runs on jobs ['periodic-ci-openshift-release-master-ci-4.16-e2e-aws-ovn-upgrade'] in the last 14 days.


nodesComDeatils := []types.ComDetails{}
for _, n := range nodesList.Items {
cds, err := ssutil.CreateComDetailsFromNode(oc, &n, tcpfile, udpfile)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be more clear on what this is doing let's consider changing to ss.ToComDetails or ss.ConvertToComDetails

cs, err := clientutil.New(kubeconfig)
o.Expect(err).ToNot(o.HaveOccurred())

g.By("fetching open ports on the cluster")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's be more clear here, this is fetching open ports on the nodes with ss

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 11, 2024
@openshift-ci openshift-ci bot added the e2e-images-update Related to images used by e2e tests label Apr 11, 2024
Add a test for the 'comMatrix' (Communication Matrix) library.
This library generates a description of the ports to which the
cluster listens for ingress traffic.

The test currently validates that the Communication Matrix
generated by the library covers all the open ports on the nodes.
It does so by comparing the output of the 'ss' command on the nodes.

Note: the test also manully creates endpointslices that
represent host services that don't have one natively.
Some of them are static and known ports. See more here:
https://github.com/openshift/enhancements/blob/master/dev-guide/host-port-registry.md
Others are host services with random ports that we create
the endpointslices from the given "ss" output for that
specific service (rpc, and crio, ovnkube).

The test create the relevant artifacts under the "commatrix"
dir. the artifacts that can be found are:
1. The communicatiom matrix.
2. The ss entries we compare against.
3. Diff of 1, and 2.
4. Diff of 2, and 1.
5. Raw tcp 'ss' output on nodes.
6. Raw udp 'ss' output on nodes.

Signed-off-by: Lior Noy <[email protected]>
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 11, 2024
Copy link
Contributor

openshift-ci bot commented Apr 11, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: liornoy
Once this PR has been reviewed and has the lgtm label, please assign dennisperiquet for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Contributor

openshift-ci bot commented Apr 12, 2024

@liornoy: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-single-node-serial ea5c916 link false /test e2e-aws-ovn-single-node-serial
ci/prow/e2e-metal-ipi-sdn ea5c916 link false /test e2e-metal-ipi-sdn
ci/prow/e2e-gcp-ovn-rt-upgrade ea5c916 link false /test e2e-gcp-ovn-rt-upgrade
ci/prow/e2e-aws-ovn-single-node-upgrade ea5c916 link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/e2e-metal-ipi-ovn-ipv6 ea5c916 link true /test e2e-metal-ipi-ovn-ipv6
ci/prow/verify-deps ea5c916 link true /test verify-deps
ci/prow/e2e-gcp-ovn-builds ea5c916 link true /test e2e-gcp-ovn-builds
ci/prow/e2e-gcp-ovn-image-ecosystem 2ddcc2b link true /test e2e-gcp-ovn-image-ecosystem
ci/prow/e2e-aws-ovn-single-node ea5c916 link false /test e2e-aws-ovn-single-node
ci/prow/e2e-aws-ovn-image-registry 2ddcc2b link true /test e2e-aws-ovn-image-registry

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-trt-bot
Copy link

Job Failure Risk Analysis for sha: ea5c916

Job Name Failure Risk
pull-ci-openshift-origin-master-e2e-metal-ipi-sdn IncompleteTests
Tests for this run (23) are below the historical average (1461): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6 IncompleteTests
Tests for this run (23) are below the historical average (702): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

if err != nil {
return nil, fmt.Errorf("failed writing to file: %s", err)
}
_, err = tcpfile.Write([]byte(fmt.Sprintf("node: %s\n%s", node.Name, strings.Join(ssOutFilteredUDP, "\n"))))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both UDP and TCP are printed to TCP file

@liornoy
Copy link
Author

liornoy commented May 5, 2024

closing this in favor of the updated test #28706

@liornoy liornoy closed this May 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
e2e-images-update Related to images used by e2e tests vendor-update Touching vendor dir or related files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants