Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: ability to stop/start environments #354

Open
nathantypanski opened this issue Mar 14, 2017 · 5 comments
Open

Feature request: ability to stop/start environments #354

nathantypanski opened this issue Mar 14, 2017 · 5 comments

Comments

@nathantypanski
Copy link
Contributor

I'm evaluating Teletraan as a replacement for Capistrano. One important feature in our Capistrano deployment system is the ability to run our init scripts to stop/start all hosts in what Teletraan calls an "environment".

It looks like I can roughly emulate this with the following workflow, when hosts are enumerated explicitly in the "Capacity" page:

  1. Go to the environment -> all hosts page.
  2. Terminate the desired hosts.
  3. Do whatever workflow tasks are required while the hosts are down.
  4. Go to the environment -> all hosts page.
  5. Perform the RESET action on the previously stopped hosts.

This is pretty clunky, however: it downloads a new build and goes through the complete lifecycle for a host (from PRE_DOWNLOAD to POST_RESTART). Really, all we need to run is RESTARTING and POST_RESTART to resume a host from the STOPPED state. It would be nice if there were a way to do this directly.

@lilida
Copy link
Contributor

lilida commented Mar 14, 2017

Have you tried the restart button on the environment page? It should restart services on all hosts

@nathantypanski
Copy link
Contributor Author

Yes, I've tried the restart button - that does not allow me to take an environment offline for a controlled period of time, which is the main thing I'm trying to do here. e.g., bring down an environment while I run migrations. Does that make sense?

@lilida
Copy link
Contributor

lilida commented Mar 15, 2017

I see. So the real ask here is a STOP script support as existing PRE_RESTART or RESTART?

@nathantypanski
Copy link
Contributor Author

Sure. Akin to clicking the RESTART button, you click a STOP button that runs a STOP script. Then, maybe reusing the RESUME button starts the environment again.

nathantypanski added a commit to nathantypanski/teletraan that referenced this issue Mar 16, 2017
Previously, the environment overview page only showed hosts in the deploy
status that were not STOPPING or STOPPED. The only way to look at them is to
look in "Total" hosts.

First of all, it doesn't make sense to me to to have an agent state that shows
up in the TOTAL but isn't available in the board.

This adds an extra configuration parameter to display STOPPING and STOPPED
hosts. It is disabled by default, in case you have some reason you do not
display those states.

Related: pinterest#354
@nathantypanski
Copy link
Contributor Author

One thing I noticed: the service will actually try to use STOPPING and STOPPED scripts for these states already when they are encountered. In my deploy-agent.log, I noticed:

INFO:deployd.common.executor:script: /tmp/simplehttpserver/teletraan/STOPPING does not exist.

To test this out, I created a STOPPING script, and sure enough, it worked! But it doesn't perform interpolation of environment variables like the normal script execution stages. Output from my application's log shows the following, even though I have PORT set to 8000 in my Script Config:

Running STOPPING script.
PORT:
ENV_NAME           simplehttpserver
STAGE_NAME         canary
DEPLOY_ID          M-hE1FdZQVug0BvrwyFroQ
DEPLOY_STE         STOPPING
OPCODE             STOP
DEPLOY_TYP         STOP
BUILD_COMM         a2f66f12f8cc3c416bf5ee0bffe808d57b5c1fc0
BUILD_NAME         target
BUILD_REPO         snip
BUILD_BRAN         master
BUILD_ID           tmEbxK40QK6JQm_DzEBdbQ_a2f66f1
BUILD_URL          snip

For some extra context: I'm trying to use the STOPPING state to gracefully shutdown my application so I can do long-running tasks like perform database migrations for an application while its frontend environment is down.

If this isn't the intended use of the Terminate button, then I suppose I will have to find another way to do it ...

lilida pushed a commit that referenced this issue Mar 17, 2017
* optionally display STOPPING and STOPPED hosts

Previously, the environment overview page only showed hosts in the deploy
status that were not STOPPING or STOPPED. The only way to look at them is to
look in "Total" hosts.

First of all, it doesn't make sense to me to to have an agent state that shows
up in the TOTAL but isn't available in the board.

This adds an extra configuration parameter to display STOPPING and STOPPED
hosts. It is disabled by default, in case you have some reason you do not
display those states.

Related: #354

* default DISPLAY_STOPPING_HOSTS to true

* default DISPLAY_STOPPING_HOSTS to true in settings

I missed this earlier in d5e5efa -
changed it in manage.py but not in settings.py.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants