Adjust and investigate database pool settings

[sandbergja]

What maintenance needs to be done?

• Use the RAILS_MAX_THREADS environment variable to set the pool setting.
• Investigate the timeout setting to see if it is appropriate.

Level of urgency

• High
• Moderate
• Low

Why is this maintenance needed?

We have been experiencing timeout errors (see #1566). This may be related, and even if it's not, it's good to make sure these settings are appropriate.

Acceptance criteria

• We set the RAILS_MAX_THREADS environment variable on the boxes using princeton ansible
• We use the RAILS_MAX_THREADS environment variable to determine the size of the database connection pool for prod, staging, and qa.
• We investigate the timeout setting to make sure it is appropriate for our environment and postgres configuration.

Additional information

• The sidekiq documentation has some information about RAILS_MAX_THREADS, the number of sidekiq threads, and the db connection pool