Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SecurityGroup groupName setting from UserIdGroupPair spec not working as expected #2058

Closed
adriananeci opened this issue Apr 22, 2024 · 2 comments
Labels
service/ec2 Indicates issues or PRs that are related to ec2-controller.

Comments

@adriananeci
Copy link

Describe the bug

We are creating/grouping multiple SecurityGroups ACK objects into a single helm chart, along with other ACK objects (VPC, Subnets, etc). Some of these SGs need to be referenced in other SGs that are part of the same chart. Based on https://aws-controllers-k8s.github.io/community/reference/ec2/v1alpha1/securitygroup/ there are few options via userIDGroupPairs when a SG needs to be referenced in another SG: groupID or groupName. Given the SG ID is not known beforehand the only option left in this case is groupName. During our testing we found out that this option doesn't work as expected and we started getting errors like:

- lastTransitionTime: "2024-04-19T20:41:14Z"
  status: "True"
  type: ACK.ReferencesResolved
- message: "InvalidParameterCombination: An IP permission for a VPC security group may only include a group ID, not a group name.\n\tstatus code: 400, request id: de0f7840-4f3d-4b8d-8e38-d7107e9a318f"
  status: "True"
  type: ACK.Recoverable
- lastTransitionTime: "2024-04-19T20:41:15Z"
  message: Unable to determine if desired resource state matches latest observed state
  reason: "InvalidParameterCombination: An IP permission for a VPC security group may only include a group ID, not a group name.\n\tstatus code: 400, request id: de0f7840-4f3d-4b8d-8e38-d7107e9a318f"
  status: Unknown
  type: ACK.ResourceSynced

or

- lastTransitionTime: "2024-04-19T20:41:15Z"
  status: "True"
  type: ACK.ReferencesResolved
- message: "VPCIdNotSpecified: No default VPC for this user\n\tstatus code: 400, request id: 1ae9d2e2-a5dc-439f-b323-909a12bfea29"
  status: "True"
  type: ACK.Recoverable
- lastTransitionTime: "2024-04-19T20:41:16Z"
  message: Unable to determine if desired resource state matches latest observed state
  reason: "VPCIdNotSpecified: No default VPC for this user\n\tstatus code: 400, request id: 1ae9d2e2-a5dc-439f-b323-909a12bfea29"
  status: Unknown
  type: ACK.ResourceSynced

Steps to reproduce
Create a security group object like:

❯ k get securitygroup ci-sbx -oyaml | k neat
apiVersion: ec2.services.k8s.aws/v1alpha1
kind: SecurityGroup
metadata:
  annotations:
    services.k8s.aws/region: us-west-2
  name: ci-sbx
spec:
  description: Applied to all instances
  egressRules:
  - fromPort: 1
    ipProtocol: tcp
    toPort: 65535
    userIDGroupPairs:
    - description: Allow all nodes to communicate
      groupName: ci-sbx-2
      userID: "<accountID>"
  ingressRules:
  - fromPort: -1
    ipProtocol: icmp
    toPort: -1
    userIDGroupPairs:
    - description: Allow all nodes to communicate ICMP
      groupName: ci-sbx-2
      userID: "<accountID>"
  name: ci-sbx
  tags:
  - key: Name
    value: ci-sbx-k8s
  - key: AckName
    value: ci-sbx
  vpcRef:
    from:
      name: ci-sbx

Check the status of the newly created object:

❯ kubectl describe securitygroup ci-sbx

Expected outcome

Security group should be created without issues.

If VPC ID is not specified inside the userIDGroupPairs, use the one referenced via the vpcRef setting.

Other options might include adding support for securitygroupref or vpcRef settings inside the userIDGroupPairs .

Environment

  • Kubernetes version: 1.28
  • Using EKS (yes/no), if so version?: yes, 1.28
  • AWS service targeted (S3, RDS, etc.): security groups
@a-hilaly a-hilaly added the service/ec2 Indicates issues or PRs that are related to ec2-controller. label Apr 25, 2024
ack-prow bot pushed a commit to aws-controllers-k8s/ec2-controller that referenced this issue Jun 3, 2024
Closes aws-controllers-k8s/community#2068,
aws-controllers-k8s/community#2061, and
aws-controllers-k8s/community#2058

The EC2 API for setting ingress/egress rules has many special restrictions,
making its behavior hard to predict. For example, `GroupName` should only be
used with default VPCs. When using non default VPCs users should use `GroupID`
instead

To address this problem, we are introducing a defaulting mechanism to help the
controller infer and use the correct `GroupID` when a user doesnt provide one.

You might wonder why all the trouble, and why not just use ACK resource references?
Well.. this is necessary because ACK resource references cannot do self
references, making fully declarative egress/ingress rule definition impossible in some
cases.

Changes:
- Mark `UserIDGroupPairs.GroupName` as non-required (at the CRD level)
- Default `UserIDGroupPairs.GroupID` to the parent security group ID
- Default `UserIDGroupPairs.VPCID` to the VPC of the parent security group
- Add more e2e tests for `UserIDGroupPairs`

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
@a-hilaly
Copy link
Member

a-hilaly commented Jun 3, 2024

@ack-prow ack-prow bot closed this as completed Jun 3, 2024
Copy link

ack-prow bot commented Jun 3, 2024

@a-hilaly: Closing this issue.

In response to this:

Should be fixed in https://github.com/aws-controllers-k8s/ec2-controller/releases/tag/v1.2.9
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

nnbu pushed a commit to nnbu/ack-ec2-controller that referenced this issue Sep 18, 2024
…s-controllers-k8s#194)

Closes aws-controllers-k8s/community#2068,
aws-controllers-k8s/community#2061, and
aws-controllers-k8s/community#2058

The EC2 API for setting ingress/egress rules has many special restrictions,
making its behavior hard to predict. For example, `GroupName` should only be
used with default VPCs. When using non default VPCs users should use `GroupID`
instead

To address this problem, we are introducing a defaulting mechanism to help the
controller infer and use the correct `GroupID` when a user doesnt provide one.

You might wonder why all the trouble, and why not just use ACK resource references?
Well.. this is necessary because ACK resource references cannot do self
references, making fully declarative egress/ingress rule definition impossible in some
cases.

Changes:
- Mark `UserIDGroupPairs.GroupName` as non-required (at the CRD level)
- Default `UserIDGroupPairs.GroupID` to the parent security group ID
- Default `UserIDGroupPairs.VPCID` to the VPC of the parent security group
- Add more e2e tests for `UserIDGroupPairs`

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
service/ec2 Indicates issues or PRs that are related to ec2-controller.
Projects
None yet
Development

No branches or pull requests

2 participants