Support automatic Let's Encrypt

.travis.yml

@@ -1,15 +1,29 @@
 language: python
 cache:
   - pip
+addons:
+  hosts:
+    - jupyter.test
 env:
   global:
     - ETCDCTL_API=3
+    - LEGO_CA_SERVER_NAME=pebble
+    - LEGO_CA_CERTIFICATES=$GOPATH/src/github.com/letsencrypt/pebble/test/certs/localhost/cert.pem
+    # ref: https://github.com/letsencrypt/pebble#testing-at-full-speed
+    - PEBBLE_VA_NOSLEEP=1
+    # ref: https://github.com/letsencrypt/pebble#invalid-anti-replay-nonce-errors
+    - PEBBLE_WFE_NONCEREJECT=0
+    # ref: https://github.com/letsencrypt/pebble#authorization-reuse
+    - PEBBLE_AUTHZREUSE=100
 
 # installing dependencies
 before_install:
   - pip install --upgrade setuptools pip
   - pip install --pre -r dev-requirements.txt --upgrade .
   - pip install pytest-cov
+  # Install pebble
+  - go get -u github.com/letsencrypt/pebble/...
+  - (cd $GOPATH/src/github.com/letsencrypt/pebble && go install ./...)
 install:
   - python -m jupyterhub_traefik_proxy.install --traefik --etcd --consul --output=./bin
   - export PATH=$PWD/bin:$PATH

dev-requirements.txt

@@ -1,3 +1,4 @@
+pyOpenSSL
 pytest
 pytest-asyncio
 codecov

docs/source/https.md

@@ -0,0 +1,52 @@
+# Enabling HTTPS
+
+In order to protect the data and passwords transfered over the public network, you should not run JupyterHub without enabling **HTTPS**.
+
+JupyterHub Traefik-Proxy supports **automatically** configuring HTTPS via [Let’s Encrypt](https://letsencrypt.org/docs/), or setting it
+up **manually** with your own key and certificate.
+
+## How-To HTTPS for JupyterHub managed Traefik-Proxy
+1. Via **Let'Encrypt**:
+    * Enable automatic https:
+    ```python
+    c.Proxy.traefik_auto_https=True
+    ```
+    * Set the email address used for Let's Encrypt registration:
+    ```python
+    c.Proxy.traefik_letsencrypt_email=""
+    ```
+    * Set the domain list:
+    ```python
+    c.Proxy.traefik_letsencrypt_domains=["jupyter.test"]
+    ```
+    * Set the the CA server to be used:
+    ```python
+    c.Proxy.traefik_acme_server="https://acme-v02.api.letsencrypt.org/directory"
+    ```
+    <span style="color:green">**Note 1**</span>
+
+    **TraefikProxy**, supports only the most common challenge type, i.e. the [HTTP-01 ACME challenge](https://letsencrypt.org/docs/challenge-types/#http-01-challenge).
+    If other challenge type is needed, one could setup the proxy to be externally managed to get access to all the Traefik's configuration options (including the
+    ACME challenge type).
+
+    <span style="color:green">**Note 2**</span>
+
+    When using Let's Encrypt for certificate aquisition, Traefik **must** to be able to receive the HTTP-01 request from the ACME server on **port 80**. This is a Let's Encrypt limitation as described on the [Let's Encrypt community forum](https://community.letsencrypt.org/t/support-for-ports-other-than-80-and-443/3419/72) and the [Traefik docs](https://docs.traefik.io/v1.7/configuration/acme/#acme-challenge).
+
+    The HTTP port used for the ACME HTTP-01 challenge is set by default to 80 in TraefikProxy. However, it can be configured for testing purposes (when using the ACME testing server, [Pebble](https://github.com/letsencrypt/pebble) for example) using:
+
+      ```python
+      c.Proxy.traefik_acme_challenge_port=8000
+      ```
+
+2. **Manually**, by providing your own key and certificate:
+
+    Providing a certificate and key can be done by configuring JupyterHub to enable SSL encryption as specified in [the docs](https://jupyterhub.readthedocs.io/en/stable/getting-started/security-basics.html?highlight=https#enabling-ssl-encryption). Example:
+    ```python
+      c.JupyterHub.ssl_key = '/path/to/my.key'
+      c.JupyterHub.ssl_cert = '/path/to/my.cert'
+    ```
+
+## How-To HTTPS for external Traefik-Proxy
+If the proxy isn't managed by JupyterHub, HTTPS can be enabled through Traefik's static configuration file.
+Checkout Traefik's documentation for [setting up ACME (Let's Encrypt) configuration](https://docs.traefik.io/v1.7/configuration/acme/)

docs/source/index.rst

@@ -39,11 +39,12 @@ Installation Guide
 Getting Started
 ---------------
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
 
    toml
    etcd
    consul
+   https
 
 API Reference
 -------------

docs/source/toml.md

@@ -69,9 +69,9 @@ c.TraefikTomlProxy.toml_dynamic_config_file="/path/to/dynamic_config_filename.to
 ---
 <span style="color:green">**Note !**</span>
 
-**When JupyterHub starts the proxy**, it writes the static config once, then only edits the routes config file. 
+* **When JupyterHub starts the proxy**, it writes the static config once, then only edits the routes config file.
 
-**When JupyterHub does not start the proxy**, the user is totally responsible for the static config and 
+* **When JupyterHub does not start the proxy**, the user is totally responsible for the static config and
 JupyterHub is responsible exclusively for the routes.
 ---
 

jupyterhub_traefik_proxy/proxy.py

@@ -23,7 +23,7 @@
 from subprocess import Popen
 from urllib.parse import urlparse
 
-from traitlets import Any, Bool, Dict, Integer, Unicode, default
+from traitlets import Any, Bool, Dict, Integer, List, Unicode, default
 from tornado.httpclient import AsyncHTTPClient
 
 from jupyterhub.utils import exponential_backoff, url_path_join, new_token
@@ -47,15 +47,35 @@ class TraefikProxy(Proxy):
     )
 
     traefik_api_validate_cert = Bool(
-        True,
-        config=True,
-        help="""validate SSL certificate of traefik api endpoint""",
+        True, config=True, help="""validate SSL certificate of traefik api endpoint"""
     )
 
     traefik_log_level = Unicode("ERROR", config=True, help="""traefik's log level""")
 
+    traefik_acme_challenge_port = Integer(
+        80, config=True, help="""http port for the acme challenge"""
+    )
+
+    traefik_auto_https = Bool(
+        False, config=True, help="""enable automatic HTTPS with Let's Encrypt"""
+    )
+
+    traefik_letsencrypt_email = Unicode(config=True, help="""Let's Encrypt email""")
+
+    traefik_letsencrypt_domains = List(config=True, help="""domains list""")
+
+    traefik_acme_server = Unicode(
+        "https://acme-staging-v02.api.letsencrypt.org/directory",
+        config=True,
+        help="""the CA server to use""",
+    )
+
+    traefik_acme_storage = Unicode(
+        "acme.json", config=True, help="""file used for certificates storage"""
+    )
+
     traefik_api_password = Unicode(
-        config=True, help="""The password for traefik api login"""
+        config=True, help="""the password for traefik api login"""
     )
 
     @default("traefik_api_password")
@@ -217,21 +237,53 @@ async def _setup_traefik_static_config(self):
         self.static_config = {}
         self.static_config["debug"] = True
         self.static_config["logLevel"] = self.traefik_log_level
+
         entryPoints = {}
+        scheme = urlparse(self.public_url).scheme
+        address = urlparse(self.public_url).netloc
+        port = urlparse(self.public_url).port
 
-        if self.ssl_cert and self.ssl_key:
-            self.static_config["defaultentrypoints"] = ["https"]
-            entryPoints["https"] = {
-                "address": ":" + str(urlparse(self.public_url).port),
-                "tls": {
-                    "certificates": [
-                        {"certFile": self.ssl_cert, "keyFile": self.ssl_key}
-                    ]
-                },
+        self.static_config["defaultentrypoints"] = [scheme]
+
+        # Traefik complains if we don't provide a port
+        if not port:
+            if scheme == "http":
+                address += ":80"
+            elif scheme == "https":
+                address += ":443"
+
+        entryPoints[scheme] = {"address": address}
+
+        if self.traefik_auto_https:
+            entryPoints["https"].update({"tls": {}})
+            entryPoints["acme_challenge"] = {
+                "address": ":" + str(self.traefik_acme_challenge_port)
             }
-        else:
-            self.static_config["defaultentrypoints"] = ["http"]
-            entryPoints["http"] = {"address": ":" + str(urlparse(self.public_url).port)}
+
+            acme = {
+                "email": self.traefik_letsencrypt_email,
+                "storage": self.traefik_acme_storage,
+                "entryPoint": "https",
+                "caServer": self.traefik_acme_server,
+                "httpChallenge": {"entryPoint": "acme_challenge"},
+            }
+
+            acme["domains"] = []
+
+            for domain in self.traefik_letsencrypt_domains:
+                acme["domains"].append({"main": domain})
+            self.static_config["acme"] = acme
+
+        if self.ssl_cert and self.ssl_key:
+            entryPoints["https"].update(
+                {
+                    "tls": {
+                        "certificates": [
+                            {"certFile": self.ssl_cert, "keyFile": self.ssl_key}
+                        ]
+                    }
+                }
+            )
 
         auth = {
             "basic": {
@@ -248,20 +300,17 @@ async def _setup_traefik_static_config(self):
         self.static_config["api"] = {"dashboard": True, "entrypoint": "auth_api"}
         self.static_config["wss"] = {"protocol": "http"}
 
-
     def _routespec_to_traefik_path(self, routespec):
         path = self.validate_routespec(routespec)
-        if path != '/' and path.endswith('/'):
-            path = path.rstrip('/')
+        if path != "/" and path.endswith("/"):
+            path = path.rstrip("/")
         return path
 
-
     def _routespec_from_traefik_path(self, routespec):
-        if not routespec.endswith('/'):
-            routespec = routespec + '/'
+        if not routespec.endswith("/"):
+            routespec = routespec + "/"
         return routespec
 
-
     async def start(self):
         """Start the proxy.
 

tests/config_files/pebble-config.json

@@ -0,0 +1,10 @@
+{
+  "pebble": {
+    "listenAddress": "0.0.0.0:14000",
+    "managementListenAddress": "0.0.0.0:15000",
+    "httpPort": 8000,
+    "tlsPort": 8443,
+    "ocspResponderURL": "",
+    "externalAccountBindingRequired": false
+  }
+}

tests/conftest.py

@@ -6,13 +6,36 @@
 import sys
 import time
 
+import json
 import pytest
 
 from jupyterhub_traefik_proxy import TraefikEtcdProxy
 from jupyterhub_traefik_proxy import TraefikConsulProxy
 from jupyterhub_traefik_proxy import TraefikTomlProxy
 
 
+@pytest.fixture
+async def autohttps_toml_proxy():
+    """Fixture returning a configured Let's Encrypt TraefikTomlProxy"""
+    proxy = TraefikTomlProxy(
+        public_url="https://jupyter.test:8443",
+        traefik_api_password="admin",
+        traefik_api_username="api_admin",
+        should_start=True,
+        traefik_auto_https=True,
+        traefik_letsencrypt_email="[email protected]",
+        traefik_letsencrypt_domains=["jupyter.test"],
+        traefik_acme_server="https://0.0.0.0:14000/dir",
+        traefik_acme_challenge_port=8000
+    )
+
+    await proxy.start()
+    yield proxy
+    await proxy.stop()
+    if os.path.exists("acme.json"):
+        os.remove("acme.json")
+
+
 @pytest.fixture
 async def no_auth_consul_proxy(consul_no_acl):
     """
@@ -190,6 +213,37 @@ def external_toml_proxy():
     traefik_process.wait()
 
 
+@pytest.fixture
+async def pebble():
+    config_file = "./tests/config_files/pebble-config.json"
+    gopath = os.environ["GOPATH"]
+    """
+    Insert the path to certs into the pebble config file.
+    We cannot pass $GOPATH directly as pebble doesn't know how to parse env vars
+    and we cannot hard-code it either as the test would error when run locally.
+    """
+    with open(config_file, "r+") as f:
+        data = json.load(f)
+        data["pebble"]["certificate"] = (
+            gopath + "/src/github.com/letsencrypt/pebble/test/certs/localhost/cert.pem"
+        )
+        data["pebble"]["privateKey"] = (
+            gopath + "/src/github.com/letsencrypt/pebble/test/certs/localhost/key.pem"
+        )
+        f.seek(0)
+        json.dump(data, f, indent=2)
+
+    pebble_server = subprocess.Popen(
+        ["pebble", "-config", "./tests/config_files/pebble-config.json"],
+        stdout=None,
+        stderr=None,
+    )
+    yield pebble_server
+
+    pebble_server.kill()
+    pebble_server.wait()
+
+
 @pytest.fixture(scope="session", autouse=True)
 def etcd():
     etcd_proc = subprocess.Popen("etcd", stdout=None, stderr=None)