Deploy a Django app to Heroku

This guide provides instructions on deploying a Django app to Heroku, DataMade's preferred platform for hosting dynamic applications.

The easiest way to properly set up your application code for Heroku is to use our Django template to create a fresh Heroku-enabled Django app. The template includes additional nice features like ES6 support and GitHub Actions configuration, but it is only appropriate for brand new apps. Once you've created your app, you can skip to learning how to Provision Heroku resources.

If you'd like to convert an existing app to Heroku, or if the template is unfeasible for some other reason, see Set up application code for Heroku.

Provision Heroku resources

If you initialized your application with DataMade's new-django-app Cookiecutter template, your app is already configured to run on Heroku. If you are migrating an existing app, or if you want to learn more about the configuration that comes with the template, see Set up application code for Heroku, below, before proceeding with this step.

The following instructions will help you deploy your properly configured application to the platform.

Install the Heroku CLI with the manifest plugin

The fastest way to get a project up and running on Heroku is to use the Heroku CLI. Before you start, make sure you have the CLI installed locally. Once you install the CLI, you'll need to switch over to the CLI's beta version. This allows you to use the manifest CLI plugin, which you must install:

heroku update beta

Confirm that you have the manifest plugin installed:

heroku manifest --help

Create apps and pipelines for your project

In order to deploy your project, you need to create Heroku apps for staging and production, and tie them together in a pipeline. Be sure that your application is committed to version control before you begin, and that the repo has a heroku.yml file. Otherwise Heroku won't use the correct buildpack — we use a container buildpack and not one of their default Python buildpacks.

To create these resources, start by defining the name of your app:

# This should be the same slug as your project's GitHub repo.
export APP_NAME=<your-app-name-here>

Then, run the following Heroku CLI commands to create a staging app and a pipeline:

heroku create ${APP_NAME}-staging -t datamade --manifest
heroku pipelines:create -t datamade ${APP_NAME} -a ${APP_NAME}-staging -s staging
heroku pipelines:connect ${APP_NAME} -r datamade/${APP_NAME}

Your CLI output should look like this:

heroku create ${APP_NAME}-staging -t datamade --manifest
Reading heroku.yml manifest... done
Creating ⬢ demo-app-staging... done, stack is container
Adding heroku-postgresql... done
https://demo-app-staging.herokuapp.com/ | https://git.heroku.com/demo-app-staging.git

heroku pipelines:add ${APP_NAME}-staging -a ${APP_NAME}-staging -s staging
Adding ⬢ demo-app-staging to datamade-app pipeline as staging... done

If you would like to set up a production app as well, run the following commands to create one and add it to your pipeline:

heroku create ${APP_NAME} -t datamade --manifest
heroku pipelines:add ${APP_NAME} -a ${APP_NAME} -s production

Once you have the environments you need, enable review apps for your pipeline:

# Note that these need to be two separate commands due to an open Heroku bug,
# since --autodeploy and --autodestroy require a PATCH request
heroku reviewapps:enable -p ${APP_NAME}
heroku reviewapps:enable -p ${APP_NAME} --autodeploy --autodestroy

Set configuration variables for review apps and deployments

Next, configure environment variables for staging and production apps. DJANGO_SECRET_KEY should be a string generated using the XKCD password generator, and DJANGO_ALLOWED_HOSTS should be a comma-separated string of valid hosts for your app.

heroku config:set -a ${APP_NAME}-staging DJANGO_SECRET_KEY=<random-string-here>
heroku config:set -a ${APP_NAME}-staging DJANGO_ALLOWED_HOSTS=.herokuapp.com

# Run these commands if you have a production application
heroku config:set -a ${APP_NAME} DJANGO_SECRET_KEY=<random-string-here>
heroku config:set -a ${APP_NAME} DJANGO_ALLOWED_HOSTS=<your-list-of-allowed-hosts-here>

Note that review app config vars cannot yet be set using the CLI, but you can set them in the Heroku dashboard by navigating to the pipeline home page and visiting Settings > Review Apps > Review app config vars in the nav.

You can also set them in the app.json file, but only set non-sensitive values since that file is committed to version control. See this code for an example, and the Heroku docs for more information about the app.json schema.

Also note that while DATABASE_URL is probably required by your application, you don't actually need to set it yourself. The Heroku Postgres add-on will automatically define this variable when it provisions a database for your application.

Configure deployments from Git branches

Heroku can deploy commits to specific branches to different environments (e.g. staging vs. production).

Follow the Heroku documentation to enable automatic deploys from main to your staging app. Be sure to check Wait for CI to pass before deploy to prevent broken code from being deployed!

For production deployments, create a long-lived deploy branch off of main and configure automatic deployments from deploy to production.

# create deploy branch (first deployment)
git checkout main
git pull origin main
git checkout -b deploy
git push origin deploy

# sync deploy branch with main and deploy to production (subsequent deployments)
git checkout main
git pull origin main
git push origin main:deploy

Upgrade production resources

Creating your production instance from our template Heroku artifacts will provision hobby-grade resources for your application. Ahead of launch, plan time to upgrade your dyno and database to at least the first production-grade tier. That's Standard-1x for dynos and Standard-0 for Postgres.

Consult the Heroku documentation on:

Setting dyno types
Upgrading Postgres with pg:copy

Set up Slack notifications

Heroku can send build notifications to Slack via the Heroku ChatOps integration. This integration should already be set up in our Slack channel, but if you need to install it again, see the official documentation.

To enable notifications for an app, run the following Slack command in the corresponding channel:

/h route ${PIPELINE_NAME} to ${CHANNEL_NAME}

For example, to enable notifications for the parserator pipeline in the #parserator channel, we would run /h route parserator to #parserator.

Enable additional services

If your app requires additional services, like Solr or PostGIS, you'll need to perform some extra steps to set them up for your pipeline.

Solr

In the absence of an affordable Solr add-on, we deploy apps that use Solr using our legacy AWS deployment pattern. Consult senior staff.

PostGIS

If your app requires PostGIS, you'll need to manually enable it in your database. Once your database has been provisioned, run the following command to connect to your database and enable PostGIS:

heroku psql -a <YOUR_APP> -c "CREATE EXTENSION postgis"

To automate this process, you can include a step like this in scripts/release.sh to make sure PostGIS is always enabled in your databases:

psql ${DATABASE_URL} -c "CREATE EXTENSION IF NOT EXISTS postgis"

Set up a custom domain

All Heroku apps are automatically delegated a subdomain under the heroku.com root domain, like example.heroku.com. This automatic Heroku subdomain is usually fine for review apps and staging apps, but production apps almost always require a dedicated custom domain like example.com.

When you're ready to deploy to production and publish your app publicly, you'll need to set up a custom domain. In order to do this, you need to register the custom domain in two places: in the Heroku dashboard, and in your (or your client's) DNS provider. Then, you'll need to instruct Heroku to enable SSL for your domain.

For detailed documentation on setting up custom domains, see the Heroku docs.

Step 1: Configure a custom domain on Heroku

The first step to setting up a custom domain is to instruct Heroku to use the domain for your app. Navigate to Settings > Domains in your app dashboard, choose Add domain, and enter the name of the custom domain you would like to use.

When you save the domain, Heroku should display the DNS target for your domain. Copy this string and use it in the next step to delegate the domain with your DNS provider.

Step 2: Configure a custom domain on a DNS provider

Note: If you're not comfortable with basic DNS terminology and you're finding this section to be confusing, refer to the CloudFlare docs on how DNS works.

Once you have a DNS target for Heroku, you need to instruct your DNS provider to direct traffic for your custom domain to Heroku.

If you're setting up a custom subdomain, like www.example.com or app.example.com, you'll need to create a CNAME record pointing to your DNS target with your DNS provider. For more details, see the Heroku docs on configuring DNS for subdomains.

If you're setting up a custom root domain, like example.com, you'll need to create the equivalent of an ALIAS record with your DNS provider. Not all DNS providers offer the same type of ALIAS record, so to provision this record you should visit the Heroku docs on configuring DNS for root domains and follow the instruction for your provider. At DataMade we typically use Namecheap, which allows you to create ALIAS records.

After creating the appropriate DNS record with your DNS provider, wait a few minutes for DNS to propagate and confirm that you can load your app by visiting your custom domain. Remember that Django will only serve domains that are listed in its ALLOWED_HOSTS settings variable, so you may have to update your DJANGO_ALLOWED_HOSTS config var on Heroku to accomodate your custom domain.

Step 3: Enable SSL

Once your custom domain is properly resolving to your app, navigate to Settings > SSL Certificates in your app dashboard, select Configure SSL, and Choose Automatic Certificate Management (ACM). Your app should now load properly when you visit it with the https:// protocol.

As a final step, we want to make sure that the app always redirects HTTP traffic to HTTPS. Heroku can't do this for us, so we need to configure the app code to do it. If you didn't use the Django template to create your app, add the following settings to your settings.py file:

if DEBUG is False:
    SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
    SECURE_SSL_REDIRECT = True

When you deploy this change and try to load your app with the http:// protocol, it should now automatically redirect you to https:// and display a valid certificate.

Manual SSL Certificates

For most of our Heroku sites, we go with Automatic Certificate Management (ACM) for our SSL certifications. However, for domains that already have an SSL certification for the entire domain like *.example.com, the SSL certificate must be set up manually:

Heroku provides documentation on how to create the necessary CSR file that SSL providers require: https://devcenter.heroku.com/articles/acquiring-an-ssl-certificate

General guidelines for custom domains

When setting up custom domains, follow these general guidelines:

Where possible, let the client register the domain name so we don't have to manage it.
Shorter domains are always better, but we usually defer to our clients' preferences when choosing a custom domain name.
If the client has a pre-existing root domain, always advise deploying on a subdomain like app.example.com instead of a path like example.com/app. Clients often ask for paths off of root domains, but they are typically quite hard to deploy.
If using a root domain, make sure to set up a www subdomain to redirect to the root.
Don't allow .herokuapp.com in DJANGO_ALLOWED_HOSTS in production, since we want the custom domain to be canonical for search engine optimization.

Set up application code for Heroku

In order to deploy a legacy Django application to Heroku, a few specific configurations need to be enabled in your application code.

Containerize your app

We use Heroku as a platform for deploying containerized apps, which means that your app must be containerized in order to use Heroku properly. If your app is not yet containerized, follow our instructions for containerizing Django apps before moving on.

There are two commands you should make sure to add to your Dockerfile in order to properly deploy with Heroku:

In the section of your Dockerfile where you install OS-level dependencies with apt-get, make sure to install curl so that Heroku can stream logs during releases (if you're inheriting from the official python images, curl will already be installed by default)
Toward the end of your Dockerfile, run python manage.py collecstatic --noinput so that static files will be baked into your container on deployment

For an example of a Django Dockerfile that is properly set up for Heroku, see the Minnesota Election Archive project.

Clean up old configurations

For new projects, you can skip this step. For existing projects that are being convered from our older deployment practices, you'll want to consolodate everything into settings.py and eventually delete your settings_local.py and supporting files. In switching to Heroku, the settings_local.py pattern is no longer necessary to store secret values as we'll be using environment variables instead.

In addition, you will want to delete the following files, as we won't be using Travis, Blackbox or CodeDeploy:

.travis.yml (we will be using GitHub Actions instead of Travis for CI)
appspec.yml
APP_NAME/settings_local.example.py (secret values are now stored as environment variables)
configs/nginx.xxx.conf.gpg files
configs/supervisor.xxx.conf.gpg files
configs/settings_local.xxx.py.gpg files
configs/settings_local.travis.py files
keyrings/live/pubring.kbx (Blackbox is no longer needed as we're using environment variables)
scripts/after_install.sh.gpg (no longer using AWS CodeDeploy)
scripts/before_install.sh.gpg
scripts/app_start.sh.gpg
scripts/app_stop.sh.gpg

For an example of a conversion, see this PR for the Erikson EDI project (private repo)

Serve static files with WhiteNoise

Apps deployed on Heroku don't typically use Nginx to serve content, so they need some other way of serving static files. Since our apps tend to have relatively low traffic, we prefer configuring WhiteNoise to allow Django to serve static files in production.

Follow the setup instructions for WhiteNoise in Django to ensure that your Django apps can serve static files. You will also need to include whitenoise in your requirements.txt as a dependency.

In order to be able to serve static files when DEBUG is False, you'll also want to make sure that RUN python manage.py collectstatic is included as a step in your Dockerfile. This will ensure that the static files are baked into the container. Since we typically mount application code into the container during local development, you'll also need to make sure that your static files are stored outside of the root project folder so that the mounted files don't overwrite them. One easy way to do this is to set STATIC_ROOT = '/static' in your settings.py file.

Read settings and secret variables from the environment

Heroku doesn't allow us to decrypt content with GPG, so we can't use Blackbox to decrypt application secrets. Instead, we can store these secrets as config vars, which Heroku will thread into our container environment.

The three most basic config vars that you'll want to set for every app include the Django DEBUG, SECRET_KEY, and ALLOWED_HOSTS variables. Update settings.py to read these variables from the environment:

SECRET_KEY = os.environ['DJANGO_SECRET_KEY']
DEBUG = False if os.getenv('DJANGO_DEBUG', True) == 'False' else True
allowed_hosts = os.getenv('DJANGO_ALLOWED_HOSTS', [])
ALLOWED_HOSTS = allowed_hosts.split(',') if allowed_hosts else []

Make sure to update your app service in docker-compose.yml to thread any variables that don't have defaults into your local environment:

services:
  app:
    environment:
      - DJANGO_SECRET_KEY=really-super-secret

For a full example of this pattern in a production app, see the Docker Compose file and Django settings file in the UofM Election Archive project.

Configure Django logging

When Gunicorn is running our app on Heroku, we generally want it to log to stdout and stderr instead of logging to files, so that we can let Heroku capture the logs and see view them with the heroku logs CLI command (or in the web console).

By default, Django will not log errors to the console when DEBUG is False (as documented here). To make sure that errors get logged appropriately in Heroku, set the following baseline logging settings in your settings.py file:

import os

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,  # Preserve default loggers
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
        },
    },
}

For more detail on Django's logging framework, see the documentation.

Create Heroku config files

If you're converting an existing app to use Heroku, create the following config files relative to the root of your repo. If you're setting up a Heroku deployment for an app that you created with the Django template, you should have these config files already, and you can safely skip to Create apps and pipelines for your project.

`heroku.yml`

The heroku.yml manifest file tells Heroku how to build and network the services and containers that comprise your application. This file should live in the root of your project repo. For background and syntax, see the documentation. Use the following baseline to get started:

# Define addons that you need for  your project, such as Postgres, Redis, or Solr.
setup:
  addons:
    - plan: heroku-postgresql
# Define your application's Docker containers.
build:
  docker:
    web: Dockerfile
# Define any scripts that you'd like to run every time the app deploys.
release:
  command:
    - ./scripts/release.sh
  image: web
# The command that runs your application. Replace 'app' with the name of your app.
run:
  web: gunicorn -t 180 --log-level debug app.wsgi:application

`release.sh`

In your app's scripts folder, define a script release.sh to run every time the app deploys. Use the following baseline script and make sure to run chmod u+x scripts/release.sh to make it executable:

#!/bin/bash
set -euo pipefail

python manage.py collectstatic --noinput
python manage.py migrate --noinput
python manage.py createcachetable && python manage.py clear_cache

If your app uses a dbload script to load initial data into the database, you can use release.sh to check if the initial data exists and run the data loading scripts if not. For example:

# Set ${TABLE} to the name of a table that you expect to have data in it.
if [ `psql ${DATABASE_URL} -tAX -c "SELECT COUNT(*) FROM ${TABLE}"` -eq "0" ]; then
    make all
fi

The release.sh file must be set to executable at the file system level. To do this, run chmod +x scripts/release.sh on your local machine and commit the change. Surprisingly, GitHub will recognize this change!

Note that logs for the release phase won't be viewable in the Heroku console unless curl is installed in your application container. Make sure your Dockerfile installs curl to see logs as scripts/release.sh runs.

`app.json`

In order to enable review apps for your project, the repo must contain an app.json config file in the root directory. For background and syntax, see the documentation. Use the following baseline to get started:

{
  "name": "your-app",
  "scripts": {},
  "env": {
    "DJANGO_SECRET_KEY": {
      "required": true
    },
    "DJANGO_ALLOWED_HOSTS": {
      "required": true
    }
  },
  "formation": {
    "web": {
      "quantity": 1,
      "size": "hobby"
    }
  },
  "environments": {
    "review": {
      "addons": ["heroku-postgresql:hobby-basic"]
    }
  },
  "buildpacks": [],
  "stack": "container"
}

Set up GitHub Actions for CI

For Heroku deployments, we use GitHub Actions for CI (continuous integration). Read the how-to to set up GitHub Actions.

Troubleshooting

I see `Application error` or a `Welcome to Heroku` page on my site

If your app isn't loading at all, check the dashboard to make sure that the most recent build and release cycle passed. If the build and release both passed, check the Dyno formation widget on the app Overview page to make sure that dynos are enabled for your web process.

Heroku CLI commands are failing without useful error messages

Sometimes a Heroku CLI command will fail without showing much output (e.g. Build failed). In these cases, you can set the following debug flag to show the raw HTTP responses from the Heroku API:

export HEROKU_DEBUG=1

The release phase of my build is failing but the logs are empty

Heroku can't stream release logs unless curl is installed in the application container. Double-check to make sure your Dockerfile installs curl.

If curl is installed and you still see no release logs, try viewing all of your app's logs by running heroku logs -a <YOUR_APP>. Typically this stream represents the most complete archive of logs for an app.

I need to connect to an app database from the command line

The Heroku CLI provides a command, heroku psql, that you can use to connect to your database in a psql shell. See the docs for using this command.

I need to share a database between multiple apps

You can use the Heroku CLI to accomplish this task. See the Heroku docs on sharing databases between applications.

I get emails saying "[warning] Database disruption imminent, row limit exceeded"

If a review app requires loading in data with more than 10,000 rows, Heroku will send an angry email to whoever "deployed" that review app saying that disruption of the database is imminent because of exceeded row limits.

If the email is indeed referring to a review app, you can safely ignore it because "database disruption" means that INSERT operations will be revoked in seven days and for most review apps this is beyond the amount of time the app will be active anyway. If the email is instead referring to a production app or a long-lived staging app, you should upgrade your Heroku Postgres plan for that instance to insure that database function continues.

In an ideal world it would be nice to configure apps that require >10,000 rows of data to use a larger Heroku Postgres plan for review apps. Unfortunately, there is not currently a way to set this type of configuration (and hence prevent these kinds of emails being sent for review apps) because Heroku defaults to the cheapest plan for review app addons.

Help! My staging pipeline doesn't have a database!

You might deploy a review app and everything works. Then you merge your code to main, which builds a new version to the staging pipeline. But for some reason, there is no database provisioned for staging.

Did you have the manifest CLI plugin installed when you first created the Heroku pipeline? If not, then it won't provision the Postgres add-on. See this step.

Here's an example where the manifest plugin was not installed when creating an app:

heroku create ${APP_NAME}-staging -t datamade --manifest
Creating ⬢ demo-app-staging... done
https://demo-app-staging.herokuapp.com/ | https://git.heroku.com/demo-app-staging.git

Here is an example where everything worked because the manifest plugin was installed:

heroku create ${APP_NAME}-staging -t datamade --manifest
Reading heroku.yml manifest... done
Creating ⬢ demo-app-staging... done, stack is container
Adding heroku-postgresql... done
https://demo-app-staging.herokuapp.com/ | https://git.heroku.com/demo-app-staging.git

heroku pipelines:add ${APP_NAME}-staging -a ${APP_NAME}-staging -s staging
Adding ⬢ demo-app-staging to datamade-app pipeline as staging... done

The difference is in the CLI's output. In the working example, note the output Reading heroku.yml manifest... done and Adding heroku-postgresql... done.

If that is not the problem, then make sure your app's heroku.yml is configured correctly. When Heroku builds your app to a pipeline, it uses the heroku.yml file to provision the resources (like Postgres or Solr).

Files

deploy-a-django-app.md

Latest commit

History