Skip to content
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.

FEATURE ADDED - Integrated Docker development environment, works cross platform (including Windows!) #1818

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env-example
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ TEST_CLASSROOM_OWNER_ORGANIZATION_GITHUB_ID=
TEST_CLASSROOM_OWNER_ORGANIZATION_GITHUB_LOGIN=

WEBHOOK_SECRET=
API_SECRET=
API_SECRET=
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ vendor/bundle
!/log/.keep
/tmp
/coverage
yarn-error.log

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file came up in testing, and I realized it should be .gitignored.


# Ignore local environment variables
.env
Expand Down Expand Up @@ -46,3 +47,4 @@ spec/examples.txt

# mac
.DS_Store

40 changes: 40 additions & 0 deletions Dockerfile-sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
FROM ruby:2.4.2

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the docker file to build the docker image that will do the development. It starts with the base ruby image for the ruby version currently used in the project, but for easy forward compatibility it installs rbenv as well.


WORKDIR /usr/src/app

# weird we have to set these env variables to get rbenv
# to install without an error, but such is life.
ENV PATH "$PATH:/root/.rbenv/bin"
ENV PATH "$PATH:/root/.rbenv/shims"

#not sure what this bug is, should probably try taking it out later
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bug may be because we are using an older version of Ruby, and the container for that version is based on debian jessie which is no longer supported. Maybe this should be using a newer version (possibly alpine) instead.

RUN printf "deb http://archive.debian.org/debian/ jessie main\ndeb-src http://archive.debian.org/debian/ jessie main\ndeb http://security.debian.org jessie/updates main\ndeb-src http://security.debian.org jessie/updates main" > /etc/apt/sources.list

#install rbenv (though is this really necessary? Will try both ways.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image might be more efficient if we use the default Ruby that comes with the container rather than installing a different version. We're also hoping to upgrade to Ruby 2.6.3 in the nearish future which should make this process simpler.

RUN wget -q https://github.com/rbenv/rbenv-installer/raw/master/bin/rbenv-installer -O- | bash

#install yarn and node and other dependencies
RUN wget -q https://deb.nodesource.com/setup_10.x -O- | bash -

RUN wget -q https://dl.yarnpkg.com/debian/pubkey.gpg -O- | apt-key add -

RUN echo "deb http://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y yarn nodejs

RUN apt-get install -y netcat libpq-dev rinetd

#adding files needed to get the bootstrap script to run
# Add files we need to build dependencies
ADD Gemfil* ./
ADD package.json .
ADD yarn.lock .
ADD .ruby-version .
ADD bin .
ADD script/bootstrap-sc .
COPY vendor ./vendor
RUN bash bootstrap-sc

#Startup script and port forwarding
COPY config/rinetd.conf /etc/rinetd.conf

ENTRYPOINT bash
2 changes: 2 additions & 0 deletions Procfile.dev.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
rails: ./bin/rails server -p 5000 -b 0.0.0.0

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

had to make a special procfile to get the binding right on rails. If there's a way to parse environment variables internally in the procfile we could do that here.

sidekiq: ./bin/sidekiq -C config/sidekiq.yml
32 changes: 28 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,11 +111,33 @@ GitHub Classroom is a [Ruby on Rails](http://rubyonrails.org/) application.

New to Ruby? No worries! You can follow these instructions to install a local server.

#### macOS
#### Cross-platform option - Self-contained Docker Development Environment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we move the Docker setup instructions to their own file? The readme is getting a bit long with these additions.


If you do not want to install and maintain ruby on your local machine directly, you can run it all in a self-contained virtual machine by following these instructions.

##### Install Docker and Docker Compose

Every platform does this differently, so Google is your friend. On Windows you can use Docker Windows or Docker Toolbox, for example, depending on various preferences and software specs.

**Special Instructions for some Windows users** If you are using a verison of docker that runs on a virtual machine (Docker Toolbox on Windows, older versions of Docker for Mac) make sure the virtual machine has access to at least 2-3 GB of RAM and, ideally, 2 CPUs. You will also, in that case, need to run `docker-machine ssh` and type `sudo sysctl -w vm.max_map_count=262144` at that prompt to avoid errors. Finally, you may want to use NAT port forwarding to forward the docker machiné's port 5000 to your host machine's 5000, so that you can access the server at localhost:5000 rather than using the docker machine's IP.

##### Building the docker images

Open a command prompt or terminal window in the project folder, then type `docker-compose -f docker-compose.sc.yml up --build -d` to build all of the development environment. This might take a while! Go get some coffee.

##### Setting up the environment

If all went well up above, you should be ready! Type `docker attach classroom_ruby` to attach to your new, virtual linux environment that is all ready to go, then follow the setup instructions that starts at [Setup Classroom](#setup-github-classroom). You can edit all of the files in the project directly on your regular machine with the editor of your choice - the dev environment will see them internally and adjust accordingly!

##### Shutting the environment down and restarting

If you are done working on the code and want to shutdown the envrionment, use `docker-compose -f docker-compose.sc.yml down` to take it down. Later, use `docker-compose -f docker-compose.sc.yml up -d` to bring it back up, followed by `docker attach classroom_ruby` to get back in.

#### macOS (Native installation)

If you're using macOS and running the Homebrew package manager you're all set to go! Head down to [Setup Classroom](#setup-github-classroom)

#### Linux
#### Linux (Native installation)

##### Installing Docker and Docker Compose

Expand All @@ -133,9 +155,9 @@ In order to install the `pg` gem you have to have PostgreSQL on your system, all

If you're running an Debian/Ubuntu based GNU/Linux for example run: `apt-get install nodejs postgresql redis-server memcached`.

#### Windows
#### Windows (Native installation)

We really don't have a good story for running this on Windows, but Pull Requests are welcome :smile:
Running Ruby natively on Windows is hard if not impossible. The best bet on Windows is to use the self-contained environment described above.

### Setup GitHub Classroom

Expand Down Expand Up @@ -208,6 +230,8 @@ Now you should have all of the values filled in, great job!

### Running the application

If you are using the docker development environment, you will need to run `docker attach classroom_rubyrails` before running any of these. If for some reason that doesn't work, try `docker-compose run -f docker-compose.sc.yaml rubyrails`.

#### Optional
If you'd like to receive webhooks from GitHub you can run:

Expand Down
31 changes: 31 additions & 0 deletions config/rinetd.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# This file is used for the docker self contained environment, to enable 'localhost'
# forwarding within the ruby dev env

# this is the configuration file for rinetd, the internet redirection server
#
# you may specify global allow and deny rules here
# only ip addresses are matched, hostnames cannot be specified here
# the wildcards you may use are * and ?
#
# allow 192.168.2.*
# deny 192.168.2.1?


#
# forwarding rules come here
#
# you may specify allow and deny rules after a specific forwarding rule
# to apply to only that forwarding rule
#
# bindadress bindport connectaddress connectport
localhost 2345 postgresql 5432
localhost 9337 elasticsearch 9300
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually aren't using elasticsearch anymore so this can be removed 🎉

localhost 9227 elasticsearch 9200
localhost 22322 memcached 11211
localhost 9736 redis 6379

# logging information
logfile /var/log/rinetd.log

# uncomment the following line if you want web-server style logfile format
# logcommon
70 changes: 70 additions & 0 deletions docker-compose.sc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
version: '2.1'
services:
rubyrails:
container_name: "classroom_rubyrails"
build:
context: .
dockerfile: Dockerfile-sc
volumes:
- .:/usr/src/app

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.bundle and vendor are copied into the docker image, then not mounted back onto the host box. That way dependencies we shouldn't be editing don't clutter up the filesystem.

- /usr/src/app/.bundle
- /usr/src/app/vendor
ports:
- "5000:5000"
entrypoint: bash

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the dev environment simply opens a shell. Users attach to it and run script/setup and script/server etc just as they would for a native installation.

tty: true
stdin_open: true
environment:
- SELFCONTAINED=true

elasticsearch:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More elasticsearch config that can be removed 🎉 🎉

container_name: "classroom_elasticsearch"
image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2
ports:
- "9337:9300"
- "9227:9200"
volumes:
- classroom-data-elasticsearch-data:/usr/share/elasticsearch/data
- classroom-data-elasticsearch-logs:/user/share/elasticsearch/logs

memcached:
container_name: "classroom_memcached"
image: memcached:1.4.26-alpine
ports:
- "22322:11211"

postgresql:
container_name: "classroom_postgresql"
image: postgres:9.4.10-alpine
ports:
- "2345:5432"
restart: always
environment:
POSTGRES_USER: "postgres"
POSTGRES_PASSWORD: "postgres"
volumes:
- classroom-data-postgres-data:/var/lib/postgresql/data
- classroom-data-postgres-logs:/var/log/postgresql

redis:
container_name: "classroom_redis"
image: redis:3.2.4-alpine
ports:
- "9736:6379"
volumes:
- classroom-data-redis-data:/var/lib/redis
- classroom-data-redis-logs:/var/log/redis

volumes:
classroom-data-elasticsearch-data:
external: false
classroom-data-postgres-data:
external: false
classroom-data-postgres-logs:
external: false
classroom-data-elasticsearch-logs:
external: false
classroom-data-redis-data:
external: false
classroom-data-redis-logs:
external: false
31 changes: 31 additions & 0 deletions script/bootstrap-sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/bin/bash

# Usage: script/bootstrap-sc.sh
# Runs bootstrap for the self-contained docker profile, where
# ruby itself is also part of the docker-compose stack

# This script gets run by the dockerfile itself to install dependencies, etc.
# It assumes we are on debian jessie, as that is the ruby:2.4.2 image base

set -e

# install ruby with rbenv if necessary (should not be if dockerfile is correctly updated)
rubyversion=$(cat .ruby-version)
rubyinstalled=$(ruby -v)
unset BUNDLER_VERSION

if [[ $rubyinstalled != *$rubyversion* ]]; then
rbenv install $(cat .ruby-version)
rbenv global $(cat .ruby-version)
fi
#install updated bundler
gem install bundler

# install gems
bundle check 2>&1 || {
echo "==> Installing gem dependencies..."
bundle install --local --without production
}

#install yarn dependencies
yarnpkg install
17 changes: 15 additions & 2 deletions script/server
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,29 @@ set -e

cd "$(dirname "$0")/.."

if [ -f "$(which docker-compose)" ]; then
if [ "$SELFCONTAINED" != "true" ] && [ -f "$(which docker-compose)" ]; then
echo "===> Stopping existing containers..."
docker-compose stop
fi

if [ "$SELFCONTAINED" = "true" ]; then
unset BUNDLER_VERSION
# Set up port forwarding
if ! pgrep -x 'rinetd' > /dev/null
then
rinetd
fi
fi

# ensure everything in the app is up to date.
script/update

test -z "$RACK_ENV" &&
RACK_ENV='development'

# boot the app and any other necessary processes.
bundle exec foreman start -f Procfile.dev
if [ "$SELFCONTAINED" != "true" ]; then
bundle exec foreman start -f Procfile.dev
else
bundle exec foreman start -f Procfile.dev.sc
fi
29 changes: 21 additions & 8 deletions script/setup
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,32 @@
# script/setup: Set up application for the first time after cloning, or set it
# back to the initial first unused state.



set -e

cd "$(dirname "$0")/.."

if [ -f "$(which docker-compose)" ]; then
echo "===> Removing existing containers..."
docker-compose down
fi

script/bootstrap
if [ "$SELFCONTAINED" != "true" ]; then
# If not self contained, set up docker and dependencies
if [-f "$(which docker-compose)" ]; then
echo "===> Removing existing containers..."
docker-compose down
fi

echo "===> Waiting for the containers to mount..."
sleep 5
script/bootstrap

echo "===> Waiting for the containers to mount..."
sleep 5
else
# If we are self-contained, docker and dependencies are done, so check
# on the port forwarding.
unset BUNDLER_VERSION
if ! pgrep -x 'rinetd' > /dev/null
then
rinetd
fi
fi

echo "===> Preparing database..."
bin/rails db:setup
Expand Down
11 changes: 7 additions & 4 deletions script/update
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,13 @@ set -e

cd "$(dirname "$0")/.."

script/bootstrap
if [ "$SELFCONTAINED" != "true" ]; then
script/bootstrap

echo "===> Waiting for the containers to mount..."
sleep 5
echo "===> Waiting for the containers to mount..."
sleep 5

fi

echo "==> Updating database..."
bin/rake db:migrate db:test:prepare
Expand All @@ -18,4 +21,4 @@ echo "===> Remove old logs and tempfiles..."
bin/rails log:clear tmp:clear

echo "===> Restart application server..."
bin/rails restart
bin/rails restart