✨ Rosa Config implementaiton #5499

PanSpagetka · 2025-05-21T13:37:12Z

Based on proposal #5451
Adding RosaRoleConfig API with implementation. that should create Account/Operator roles and OIDC config/provider necessary to create ROSA cluster.

We need to move RosaMachinePoolAutoScaling definition to controlplane, because otherwise there would be circular dependency.

What type of PR is this?
/kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Checklist:

Release note:

k8s-ci-robot · 2025-05-21T13:37:15Z

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-05-21T13:37:18Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign neolit123 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2025-05-21T13:37:22Z

Hi @PanSpagetka. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

serngawy · 2025-06-17T19:36:21Z

Dockerfile

@@ -28,12 +28,17 @@ WORKDIR /workspace
 # Copy the Go Modules manifests
 COPY go.mod go.mod
 COPY go.sum go.sum
+COPY ./rosa /workspace/rosa


why are we adding this ? what is the rosa file

serngawy · 2025-06-17T19:36:38Z

Dockerfile

 # Cache deps before building and copying source so that we don't need to re-download as much
 # and so that source changes don't invalidate our downloaded layer
 RUN  --mount=type=cache,target=/root/.local/share/golang \
     --mount=type=cache,target=/go/pkg/mod \
     go mod download

+# RUN go mod download


no need for this line

serngawy · 2025-06-17T19:37:35Z

PROJECT

can you just add the RosaRoleConfig item without changing the order of other items

serngawy · 2025-06-17T19:38:28Z

config/default/tmp.yaml

why do we need this file ?

serngawy · 2025-06-17T19:38:54Z

config/crd/kustomization.yaml

@@ -38,6 +39,7 @@ patchesStrategicMerge:
 - patches/webhook_in_awsmanagedcontrolplanes.yaml
 - patches/webhook_in_eksconfigs.yaml
 - patches/webhook_in_eksconfigtemplates.yaml
+#- patches/webhook_in_rosaroleconfigs.yaml


I believe you need to uncomment this line

serngawy · 2025-06-17T19:46:49Z

exp/controllers/rosaroleconfig_controller.go

+		}
+	}
+
+	if scope.RosaRoleConfig.Status.OIDCID == "" {


I believe you should check/set the RosaRoleConfig condition first , then get the oidc using OCM client if it is not exist then create it.
Same applied for account-roles and operator-roles

serngawy · 2025-06-17T19:47:52Z

exp/controllers/rosaroleconfig_controller.go

+		}
+	}
+
+	err = r.deleteOperatorRoles(ocmClient, awsClient, scope.RosaRoleConfig.Spec.AccountRoleConfig.Prefix)


Do we need to delete the operator roles before the oidc-provider ?

Nope, a changed it so it matches reverse creation order.

serngawy · 2025-06-17T19:49:45Z

exp/controllers/rosaroleconfig_controller.go

+	return ocmClient.DeleteOidcConfig(oidcConfigID)
+}
+
+type reporter struct {


please move this to another file , better to be under pkg/.../rosa

serngawy · 2025-07-09T09:35:27Z

@PanSpagetka would you fix the PR title so PR verify job can pass

serngawy · 2025-07-09T09:35:37Z

/ok-to-test

k8s-ci-robot · 2025-07-21T08:07:28Z

@PanSpagetka: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-provider-aws-apidiff-main	`3f9bd3a`	link	false	`/test pull-cluster-api-provider-aws-apidiff-main`
pull-cluster-api-provider-aws-e2e-blocking	`3f9bd3a`	link	true	`/test pull-cluster-api-provider-aws-e2e-blocking`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

serngawy · 2025-07-21T08:15:56Z

controlplane/rosa/api/v1beta2/rosacontrolplane_webhook.go

@@ -179,6 +183,24 @@ func (r *ROSAControlPlane) validateExternalAuthProviders() *field.Error {
 	return nil
 }

+func (r *ROSAControlPlane) validateRosaRoleConfig() *field.Error {


Not sure if the logic here is valid, hasDirectRoleFields will be true even if one condition is true and others are false. Ex; r.Spec.OIDCID can be true but all others fields can be false that case hasDirectRoleFields will be true.

serngawy · 2025-07-21T08:24:23Z

controlplane/rosa/api/v1beta2/rosacontrolplane_webhook.go

+		r.Spec.RolesRef.NetworkARN != "" || r.Spec.RolesRef.KubeCloudControllerARN != "" || r.Spec.RolesRef.NodePoolManagementARN != "" ||
+		r.Spec.RolesRef.ControlPlaneOperatorARN != "" || r.Spec.RolesRef.KMSProviderARN != ""
+
+	if hasRosaRoleConfigRef && hasDirectRoleFields {


Lets make the logic as below;
if RosaRoleConfigRef is set, we use the roleConfigRef to get all roles (ignore all other role fields even if set) just log warning.
if RosaRoleConfigRef not set and all other Role fields are set, we use the roles fields
if RosaRoleConfigRef not set and some Roles fields are missing we raise error
if RosaRoleConfigRef not set and all Roles fields are missing we raise error

serngawy · 2025-07-21T08:33:18Z

controlplane/rosa/controllers/rosacontrolplane_controller.go

+
+		conditions.MarkTrue(rosaScope.ControlPlane, rosacontrolplanev1.ROSARoleConfigReadyCondition)
+
+		// Update spec fields from RosaRoleConfig


This logic is not correct, If the user is already setting those fields value, we are updating those values. I believe we should define internal roleConfig in rosacontrolPlane scope and then check which roles to use based on availability

serngawy · 2025-07-21T08:35:53Z

exp/api/v1beta2/rosaroleconfig_types.go

+	OperatorRoleConfig OperatorRoleConfig            `json:"operatorRoleConfig"`
+	OIDCConfig         OIDCConfig                    `json:"oidcConfig"`
+	IdentityRef        *infrav1.AWSIdentityReference `json:"identityRef,omitempty"`
+	Region             string                        `json:"region,omitempty"`


Missing description and kbuilder tags.

serngawy · 2025-07-21T08:39:38Z

exp/api/v1beta2/rosaroleconfig_types.go

+	Region                string                                    `json:"region,omitempty"`
+	Prefix                string                                    `json:"prefix"`


Missing description and kbuilder tags.

serngawy · 2025-07-21T08:41:15Z

exp/api/v1beta2/rosaroleconfig_types.go

+	//  User-defined prefix for generated AWS operator policies.
+	// +kubebuilder:validation:MaxLength:=4
+	// +kubebuilder:validation:Required
+	Prefix string `json:"prefix"`


Prefix must be immutable cause changing the prefix will create new role

serngawy · 2025-07-21T08:41:46Z

exp/api/v1beta2/rosaroleconfig_types.go

+	// User-defined prefix for all generated AWS resources
+	// +kubebuilder:validation:MaxLength:=4
+	// +kubebuilder:validation:Required
+	Prefix string `json:"prefix"`


Prefix must be immutable cause changing the prefix will create new roles

mzazrivec · 2025-07-21T17:16:31Z

exp/api/v1beta2/rosaroleconfig_webhook.go

+)
+
+// SetupWebhookWithManager will setup the webhooks for the ROSARoleConfig.
+func (r *ROSARoleConfig) SetupWebhookWithManager(mgr ctrl.Manager) error {


This routine is never called from main.go, which is why I'm getting:

$ kubectl apply -f rosa-roleconfig-01.yaml Error from server (InternalError): error when creating "rosa-roleconfig-01.yaml": Internal error occurred: failed calling webhook "default.rosaroleconfig.infrastructure.cluster.x-k8s.io": failed to call webhook: the server could not find the requested resource

With this change in place the above error goes away:

$ git diff diff --git a/main.go b/main.go index c65ff7356..6348fd0de 100644 --- a/main.go +++ b/main.go @@ -285,6 +285,11 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "ROSAMachinePool") os.Exit(1) } + + if err := (&expinfrav1.ROSARoleConfig{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "ROSARoleConfig") + os.Exit(1) + } } if err = (&expcontrollers.ROSARoleConfigReconciler{

Although I don't quite understand the point of this webhook, since it's currently not doing anything.

k8s-ci-robot added do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 21, 2025

k8s-ci-robot requested review from Ankitasw and serngawy May 21, 2025 13:37

k8s-ci-robot added needs-priority size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 21, 2025

PanSpagetka marked this pull request as draft May 21, 2025 13:37

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 21, 2025

PanSpagetka force-pushed the rosa-roles-implementations branch from 11dac0b to 6056618 Compare May 28, 2025 13:43

serngawy mentioned this pull request May 29, 2025

✨ Add RosaRoles config #5473

Closed

5 tasks

PanSpagetka force-pushed the rosa-roles-implementations branch 6 times, most recently from 1587db4 to 9121ec2 Compare June 9, 2025 13:40

PanSpagetka force-pushed the rosa-roles-implementations branch 3 times, most recently from 07b73a3 to 097252f Compare June 16, 2025 11:47

serngawy suggested changes Jun 17, 2025

View reviewed changes

PanSpagetka force-pushed the rosa-roles-implementations branch from 097252f to 0c9fa93 Compare June 18, 2025 11:40

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 19, 2025

PanSpagetka force-pushed the rosa-roles-implementations branch 3 times, most recently from b3aded3 to 23fa4cb Compare June 24, 2025 11:29

PanSpagetka force-pushed the rosa-roles-implementations branch from 23fa4cb to 2515be2 Compare July 2, 2025 08:09

PanSpagetka force-pushed the rosa-roles-implementations branch from 2515be2 to c0d0d2a Compare July 7, 2025 10:28

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 7, 2025

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 9, 2025

PanSpagetka force-pushed the rosa-roles-implementations branch 4 times, most recently from ac54449 to f1fdb1c Compare July 15, 2025 09:00

PanSpagetka changed the title ~~Rosa Config implementaiton~~ ✨ Rosa Config implementaiton Jul 15, 2025

PanSpagetka force-pushed the rosa-roles-implementations branch 2 times, most recently from 29f9584 to 844a962 Compare July 15, 2025 10:14

PanSpagetka marked this pull request as ready for review July 15, 2025 10:55

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 15, 2025

k8s-ci-robot requested review from faiq and serngawy July 15, 2025 10:55

Add RosaRoleConfig API and CRD.

7395772

PanSpagetka force-pushed the rosa-roles-implementations branch from 844a962 to 7395772 Compare July 15, 2025 10:57

Use RoleConfigRef in Rosa Control Plane

3f9bd3a

PanSpagetka force-pushed the rosa-roles-implementations branch from 5997fac to 3f9bd3a Compare July 21, 2025 07:46

serngawy suggested changes Jul 21, 2025

View reviewed changes

mzazrivec reviewed Jul 21, 2025

View reviewed changes


		conditions.MarkTrue(rosaScope.ControlPlane, rosacontrolplanev1.ROSARoleConfigReadyCondition)

		// Update spec fields from RosaRoleConfig

		Region string `json:"region,omitempty"`
		Prefix string `json:"prefix"`

✨ Rosa Config implementaiton #5499

Are you sure you want to change the base?

✨ Rosa Config implementaiton #5499

Conversation

PanSpagetka commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented May 21, 2025

Uh oh!

k8s-ci-robot commented May 21, 2025

Uh oh!

k8s-ci-robot commented May 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serngawy Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serngawy commented Jul 9, 2025

Uh oh!

serngawy commented Jul 9, 2025

Uh oh!

k8s-ci-robot commented Jul 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serngawy Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serngawy Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

PanSpagetka commented May 21, 2025 •

edited

Loading

serngawy Jun 17, 2025 •

edited

Loading

serngawy Jul 21, 2025 •

edited

Loading

serngawy Jul 21, 2025 •

edited

Loading