How we set up continuous integration on Jenkins

1. At https://cloud-images.ubuntu.com/locator/ec2/, pick the latest hvm:ebs-ssd instance for the region (us-east-1 in our case)
   1. Choose the dedicated CI VPC, with IAM role Jenkins-CI.
   2. Tag it Environment: ci
   3. Give it the jenkins-ci security group (inbound port 443)
   4. Launch it!
2. Name it Jenkins-CI
3. Give yourself temporary SSH access (via the security group) and SSH in.
4. Install Jenkins:
   1. sudo apt-get update && sudo apt-get dist-upgrade
   2. Follow the instructions at https://pkg.jenkins.io/debian-stable/ to install:

      wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
      sudo apt-add-repository 'deb https://pkg.jenkins.io/debian-stable binary/'
      sudo apt-get update
      sudo apt-get install jenkins 

5. Set up GitHub for authentication: at https://jenkins-ci.overviewdocs.com/securityRealm/finishLogin set up overview-jenkins-ci with a URL of https://jenkins-ci.overviewdocs.com.
6. Setup Jenkins:
   1. From your computer, ssh -L 8080:localhost:8080 ubuntu@[JenkinsIP] and browse to http://localhost:8080
   2. Copy/paste the administrator password in (as prompted by Jenkins)
   3. "Select plugins to install" => choose defaults, ...
      • plus: embeddable-build-status, JUnit Plugin, GitHub Plugin, GitHub Authentication Plugin
      • minus: Ant Plugin, Gradle Plugin
   4. Skip creating the administrator user. Click "Start Using Jenkins"
   5. "Manage Jenkins" => "Configure System":
      • # of executors: 0
      • Jenkins URL: https://jenkins-ci.overviewdocs.com
      • System Admin e-mail address: [email protected]
      • SMTP Server: email-smtp.us-east-1.amazonaws.com
      • Check User SMTP Authentication and enter Amazon's SMTP settings
   6. "Manage Jenkins" => "Configure Global Security"
      • Security realm => Github Authentication Plugin
      • Enter Client ID and Client Secret from the overview-jenkins-ci app page on GitHub
      • Authorization => GitHub Committer Authorization Strategy
      • Enter comma-separated Admin User Names
      • Check Use GitHub repository permissions, and don't fill in Participant in Organization
      • Check Grant READ permissions for Anonymous Users
      • Check Grant ViewStatus permissions for Anonymous Users
      • Save. You'll be locked out.
7. Adjust DNS to point to your new server.
8. Set up the HTTPS proxy:
   • Install programs

     sudo apt-get install haproxy
     sudo add-apt-repository ppa:certbot/certbot
     sudo apt-get update
     sudo apt-get install certbot
     sudo certbot certonly --standalone -d jenkins-ci.overviewdocs.com -m [email protected] --agree-tos -n

   • Fill in /etc/haproxy/haproxy.cfg:

     global
       log /dev/log  local0
       log /dev/log  local1 notice
       chroot /var/lib/haproxy
       user haproxy
       group haproxy
       daemon
       # https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy-1.5.14&openssl=1.0.1e&hsts=yes&profile=modern
       ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
       ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
       ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
       ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
     
     defaults
       log global
       mode http
       option httplog
       option dontlognull
       option forwardfor
       option http-server-close
       timeout connect 5000
       timeout client 50000
       timeout server 50000
       errorfile 502 /etc/haproxy/errors/502.http
     
     frontend jenkins-ci.overviewdocs.com
       bind :80
       bind :443 ssl crt /etc/haproxy/ssl.pem
       redirect scheme https if !{ ssl_fc }
       reqadd X-Forwarded-Proto:\ https
       use_backend jenkins
     
     backend jenkins
       server jenkins01 127.0.0.1:8080
     

   • Fill in /etc/letsencrypt/post-renew.sh:

     #!/bin/sh
     
     cat \
         /etc/letsencrypt/live/jenkins-ci.overviewdocs.com/privkey.pem \
         /etc/letsencrypt/live/jenkins-ci.overviewdocs.com/fullchain.pem \
         > /etc/haproxy/ssl.pem
     chown haproxy:haproxy /etc/haproxy/ssl.pem
     chmod 0600 /etc/haproxy/ssl.pem
     systemctl restart haproxy
     

   • Fill in /etc/letsencrypt/pre-renew.sh:

     #!/bin/sh
     systemctl stop haproxy
     

   • chmod +x /etc/letsencrypt/post-renew.sh /etc/letsencrypt/pre-renew.sh
   • /etc/letsencrypt/post-renew.sh (will actually start haproxy)
   • Fill in /etc/cron.daily/letsencrypt:

     #!/bin/sh
     
     exec /usr/bin/certbot renew \
         --pre-hook /etc/letsencrypt/pre-renew.sh \
         --post-hook /etc/letsencrypt/post-renew.sh \
         --quiet

   • chmod +x /etc/cron.daily/letsencrypt