Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RDS controller cannot recover from "Storage Full" #2214

Open
johnjeffers opened this issue Nov 18, 2024 · 0 comments
Open

RDS controller cannot recover from "Storage Full" #2214

johnjeffers opened this issue Nov 18, 2024 · 0 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. service/rds Indicates issues or PRs that are related to rds-controller.

Comments

@johnjeffers
Copy link

johnjeffers commented Nov 18, 2024 •
edited
Loading

Describe the bug
When an RDS instance gets into a storage-full status, the only way to recover is to add more storage. However, after modifying the DBInstance resource to increase allocatedStorage, the ACK controller for RDS will not start the resize because it sees that the instance is not in an available state.

    - message: >-
        DB Instance in 'storage-full' state, cannot be modified until 'available'.

The ACK RDS controller should bypass the available check when the state is storage-full if a storage resize operation is pending.

  • EKS 1.31
  • ACK RDS controller 1.4.6
@michaelhtm michaelhtm added service/rds Indicates issues or PRs that are related to rds-controller. kind/bug Categorizes issue or PR as related to a bug. labels Nov 19, 2024
@michaelhtm michaelhtm mentioned this issue Nov 25, 2024
Merged
ack-prow bot pushed a commit to aws-controllers-k8s/rds-controller that referenced this issue Nov 25, 2024
@michaelhtm
Allow storage update operation to unavailable db instance (#200)
cfe16d6 
Issue [#2214](aws-controllers-k8s/community#2214)

Description of changes:
With this change we allow the controller to make a storage update 
call when the DB instance is unavailable due to a `storage-full` status.

This case is not easy to test, since the minimum storage to allocate is 20GiB.
Options for testing:
* Create an existing DB instance that is already unavailable due to `storage-full`
  status and update storage (this would then require us to fill up the remaining space)

more ideas are welcome...


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug. service/rds Indicates issues or PRs that are related to rds-controller.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@johnjeffers @michaelhtm