-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MM-52240: Stop/Start DB Cluster after load test #690
Conversation
We add the ability to stop/start the DB cluster after a load-test is complete. There is a dedicated command that the users can run any time. Additionally, there is also an automated way to stop the DB after a load test finishes, which is exposed via a flag. Using the flag makes the load test runs synchronously, so it is advised to be used only by advanced users running the command preferably inside an EC2 instance in a screen session. https://mattermost.atlassian.net/browse/MM-52240
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes look great! I've left some comments below, but mostly questions and nits. Thank you for this!
A more general question: do we always create a DB cluster, even if we have a single DB instance? The changes here assume there's always a cluster deployed.
return err | ||
} | ||
|
||
return t.Sync() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm ok with the solution here, but I guess there must be a way to do this via Terraform so we can avoid the AWS command + Terraform sync 🤔
// Sleeping for 5 minutes gives 12 lines an hour. | ||
// For an avg unbounded test of 4-5 hours, it gives around 50 lines, | ||
// which should be acceptable. | ||
time.Sleep(5 * time.Minute) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree it's acceptable 👍
Yep, we always create a cluster. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for being so nit-picky with the wording, it's solely for my future understanding of the logs 😂
Co-authored-by: Alejandro García Montoro <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! 🚀
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just a question
break | ||
} | ||
fmt.Println("Sleeping... ") | ||
time.Sleep(30 * time.Second) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't 30 second too much for the ping interval?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really. These take a long time to stop or start. In the order of ~5 minutes.
We add the ability to stop/start the DB cluster after a load-test is complete.
There is a dedicated command that the users can run any time. Additionally, there is also an automated way to stop the DB after a load test finishes, which is exposed via a flag.
Using the flag makes the load test runs synchronously, so it is advised to be used only by advanced users running the command preferably inside an EC2 instance in a screen session.
https://mattermost.atlassian.net/browse/MM-52240