Skip to content

Managing overview on amazon

Jump to bottom
Jonathan Stray edited this page Aug 20, 2014 · 29 revisions

Getting a database shell

  1. overview-manage ssh production database to log in to the database instance
  2. sudo -u postgres psql overview

Creating a .psql database backup

This works even while the database is being modified. The backed-up database will have the exact same state as the database had when the backup started.

If you're backing up a database from a remote Postgres server:

  1. Find the host, username, database and password of the database to back up.
  2. pg_dump -v --no-owner -Fc -f backup.psql --host HOST --user USERNAME DATABASE and enter the password.

But if you're logged into the database instance, it's easier (as long as you're in a directory with the right permissions):

  1. sudo -u postgres pg_dump -v --no-owner -Fc -f backup.psql

(The --no-owner is necessary when backing up from Heroku, because the Heroku username, which is gibberish, won't exist on our Amazon AWS server and so the restore will fail.)

Restoring a .psql database backup

  1. Run DROP SCHEMA public CASCADE; CREATE SCHEMA public; GRANT ALL PRIVILEGES ON SCHEMA public TO overview; in a database shell.

  2. Run sudo -u postgres pg_restore -v -c -d DATABASE --no-owner FILENAME. This will work, though the owner will be postgres instead of overview.

  3. Run SELECT 'ALTER TABLE "' || tablename || '" OWNER TO overview;' FROM pg_tables WHERE schemaname = 'public'; in a database shell

  4. Copy/paste all resulting commands back into the same database shell.

  5. In a database shell, run:

     SELECT 'ALTER TYPE "' || typname || '" OWNER TO overview;' FROM (SELECT typname FROM pg_type WHERE oid IN (SELECT DISTINCT enumtypid FROM pg_enum)) x;

  6. Copy/paste all resulting commands back into the same database shell.

  7. overview-manage restart production

Editing configuration files on any instance

  1. Check out aws-overview-config
  2. Edit the appropriate file--for instance, templates/web/root/etc/init/overview-web-server.conf will map to /etc/init/overview-web-server.conf on the web instance. Look at config/config-sample.yml and config/secrets-sample.yml to see which variables are available to you; there are also database_ip and database_url.
  3. Commit and push the changes
  4. overview-manage deploy-config ENV.INSTANCE, e.g., production.web

You may overwrite any system file, but you can never delete configuration files through this system. (You'll have to log in to each system manually.)

It may be more practical to:

  1. SSH to an instance (e.g., overview-manage ssh production web)
  2. Edit the config file and then make it take effect--for instance, sudo restart overview-web-server
  3. Follow the above steps

This will save you from committing embarrassing typos.

Deploying a new version

See https://github.com/overview/aws-overview-tools/tree/master/manage

Checking deployed version

  1. overview-manage ssh production web
  2. ls /opt/overview/frontend

That gives the SHA1 commit that's deployed on web.

Updating overview-manage on the manage instance

  1. Push a new version to Git
  2. overview-manage ssh to connect to the manage instance, then cd /opt/overview/aws-overview-tools && git pull --rebase.
  3. Log out

It wouldn't be the end of the world if you modified the repository directly on the manage instance and then pushed. But there wouldn't be an author in the commit logs.

Granting or revoking access to the manage instance

Edit /home/ubuntu/.ssh/authorized_keys.

After revoking somebody's access, remember that they may still own the private keys on the manage instance. Those give access to Overview's GitHub projects (until they're open-source and come from a read-only https:// URL) and, if the firewall isn't on, the production and staging instances. So rotate those keys. by editing /home/ubuntu/.ssh/authorized_keys on all instances, making them all use a new private key from the manage instance. Delete and create a new manage key on the EC2 Management Console, too, so future instances will use the new one.

Clearing overview-manage's caches

(Written for https://www.pivotaltracker.com/n/projects/928628/stories/69190832)

overview-manage stores files in a few places.

Let's iterate over them:

The build-cache volume

This stores Ivy and Node caches, so that the build system doesn't need to download the same files over and over again from remote repositories every time you deploy.

To wipe it, just attach the volume on the manage instance, mkfs.ext4 it, and detach it.

/opt/overview/manage

This stores git checkouts and compiled code, so that repeated deploys don't need repeated builds.

We use instance storage. (In case of data loss, it would be faster to rebuild Overview than to revive the data.) So unmount /dev/xvdb, mkfs.ext4 it, and mount it.

/opt/overview/* on other instances

overview-manage deploys to other servers. You may want to delete some files from there, too.

But that's a dangerous thing. If the servers are running, they're using the code in there. So shut down the servers first.

Also: config files get deployed throughout the other instances. If you're trying for a truly clean setup, it might be quicker to just spin up a new instance.

Clone this wiki locally