Skip to content

Commit

Permalink
update with review comments
Browse files Browse the repository at this point in the history
  • Loading branch information
@bollwyvl
bollwyvl committed Sep 22, 2023
1 parent 8f0ff5e commit faf66f4
Show file tree
Hide file tree
Showing 7 changed files with 401 additions and 165 deletions.
54 changes: 18 additions & 36 deletions .github/workflows/test.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ on:
env:
# avoid warnings about config paths
JUPYTER_PLATFORM_DIRS: "1"
# avoid looking at every version of pip ever released
PIP_DISABLE_PIP_VERSION_CHECK: "1"

jobs:
build:
Expand Down Expand Up @@ -57,12 +59,7 @@ jobs:
path: ./dist

test:
name: |-
${{ matrix.os }}
${{ matrix.python-version }}
s${{ matrix.jupyter_server-version }}
lab${{ matrix.jupyterlab-version }}
nb${{ matrix.notebook-version }}
name: ${{ matrix.os }} ${{ matrix.python-version }} ${{ matrix.pip-extras }}
needs: [build]
timeout-minutes: 30
runs-on: ${{ matrix.os }}
Expand All @@ -75,31 +72,16 @@ jobs:
matrix:
os: [ubuntu-22.04]
python-version: ["3.8", "3.11"]
jupyter_server-version: ["1", "2"]
jupyterlab-version: ["3", "4"]
include:
# let jupyterlab-version 3 and 4 imply notebook-version 6 and 7
# respectively, to avoid doubling the amount of test runs
- jupyterlab-version: "3"
notebook-version: "6"
- jupyterlab-version: "4"
notebook-version: "7"
pip-extras: ["lab", "classic"]
exclude:
# windows should work for all test variations, but a limited selection
# is run to avoid doubling the amount of test runs
- os: windows-2022
python-version: "3.8"
jupyter_server-version: "1"
jupyterlab-version: "3"
notebook-version: "6"
- os: windows-2022
python-version: "3.11"
jupyter_server-version: "2"
jupyterlab-version: "4"
notebook-version: "7"
exclude:
# jupyterlab-version 4 requires jupyter_server-version 2
- jupyter_server-version: "1"
jupyterlab-version: "4"
pip-extras: classic
- os: windows-2022
python-version: "3.8"
pip-extras: lab

steps:
- uses: actions/checkout@v4
Expand All @@ -123,7 +105,7 @@ jobs:
#
# Pytest options are set in `pyproject.toml`.
run: |
pip install -vv $(ls ./dist/*.whl)\[acceptance\] 'jupyterlab~=${{ matrix.jupyterlab-version }}.0' 'jupyter_server~=${{ matrix.jupyter_server-version }}.0' 'notebook~=${{ matrix.notebook-version }}.0'
pip install -vv $(ls ./dist/*.whl)\[acceptance,${{ matrix.pip-extras }}\]
- name: List Python packages
run: |
Expand All @@ -136,19 +118,19 @@ jobs:
jupyter server extension list 2>&1 | grep -iE "jupyter_server_proxy.*OK" -
- name: Check server extension for notebook v6
if: matrix.notebook-version == '6'
if: contains(matrix.pip-extras, 'classic')
run: |
jupyter serverextension list
jupyter serverextension list 2>&1 | grep -iE "jupyter_server_proxy.*OK" -
- name: Check frontend extension for notebook v6
if: matrix.notebook-version == '6'
if: contains(matrix.pip-extras, 'classic')
run: |
jupyter nbextension list
PYTHONUNBUFFERED=1 jupyter nbextension list 2>&1 | grep -A1 -iE '.*jupyter_server_proxy.*enabled' | grep -B1 -iE "Validating.*OK"
- name: Check frontend extension for notebook v7+
if: matrix.notebook-version != '6'
if: ${{ !contains(matrix.pip-extras, 'classic') }}
run: |
jupyter notebook extension list
jupyter notebook extension list 2>&1 | grep -iE 'jupyter_server_proxy.*OK.*'
Expand All @@ -158,12 +140,12 @@ jobs:
jupyter lab extension list
jupyter lab extension list 2>&1 | grep -iE 'jupyter_server_proxy.*OK.*'
# we have installed a pre-built wheel and configured code coverage to
# inspect "jupyter_server_proxy", by re-locating to another directory,
# there is no confusion about "jupyter_server_proxy" referring to our
# installed package rather than the local directory
- name: Run tests
run: |
# we have installed a pre-built wheel and configured code coverage to
# inspect "jupyter_server_proxy", by re-locating to another directory,
# there is no confusion about "jupyter_server_proxy" referring to our
# installed package rather than the local directory
mkdir build
cd build
pytest -c ../pyproject.toml ../tests
Expand All @@ -173,7 +155,7 @@ jobs:
uses: actions/upload-artifact@v3
with:
name: |-
tests-${{ matrix.os }}-${{ matrix.python-version }}-${{ matrix.jupyterlab-version }}-${{ github.run_number }}
tests-${{ matrix.os }}-${{ matrix.python-version }}-${{ matrix.pip-extras }}-${{ github.run_number }}
path: |
./build/pytest
./build/coverage
Expand Down
197 changes: 164 additions & 33 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,47 @@ the README.md file.

## Local development setup

### `conda`

<details>

<summary>
<i>
Optional, but especially recommended on non-Linux platforms...
</i>
</summary>

Using the `conda` (or `mamba` or `micromamba`) package manager with packages from
[`conda-forge`](https://conda-forge.org/feedstock-outputs) can help isolate development
environments on nearly any operating system and architecture.

For example, after installing [`mambaforge`](https://conda-forge.org/miniforge),
create a new environment with all heavy development and test dependencies:

```yaml
mamba create --name=jupyter-server-proxy --channel=conda-forge "python=3.11" "nodejs=20" pip git geckodriver firefox
mamba activate jupyter-server-proxy
```

</details>

### Python package

```bash
# Clone the repo to your local environment
git clone https://github.com/jupyterhub/jupyter-server-proxy.git
# Change directory to the jupyter-server-proxy directory
cd jupyter-server-proxy
# Install package in development mode
pip install -e ".[test]"
# Link your development version of the extension with JupyterLab
jupyter labextension develop --overwrite .
# Install package in development mode, with the latest Jupyter clients
pip install -e ".[test,lab]"
# Link your development version of the extension with JupyterLab and Notebook
jlpm labextension develop --overwrite .
# Server extension must be manually installed in develop mode
jupyter server extension enable jupyter_server_proxy
```

## Testing

Run the tests:

```bash
Expand All @@ -32,75 +58,171 @@ These generate test and coverage reports in `build/pytest` and `build/coverage`.

### Acceptance tests

If you have `robotframework-jupyterlibary` installed, the acceptance tests will run.
In `tests/acceptance`, a number of
[`.robot` files](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html)
emulate a user starting a Jupyter server, opening a real browser, clicking on
screen elements, and seeing several working proxy servers in supported Jupyter clients.

To install these in addition to the [Python package](#python-package) test
dependencies, run:
These tests are slower and more resource intensive than the unit and integration
tests, and generate additional screenshots, browser logs, server logs, and report
HTML in `build/robot`.

```bash
pip install -e ".[acceptance]"
```
#### Extra browser dependencies

In addition, compatible versions of:
Compatible versions of [`geckodriver`](https://github.com/mozilla/geckodriver)
and [`firefox`](https://www.mozilla.org/en-US/firefox) need to be on `$PATH`.

- `geckodriver`
- `firefox`
These can be provisioned by [a `conda`-compatible installer](#conda), a system
package manager, or as a last resort, direct binary downloads.

Need to be on your `$PATH` and compatible with each other.
#### Acceptance test dependencies

To install the additional dependencies beyond the [Python package](#python-package)
test dependencies, and run the tests against the latest Jupyter clients:

```bash
pip install -e ".[test,acceptance,lab]"
pytest
```

To run _only_ the acceptance tests, use the `-k` switch:

```bash
pytest -k acceptance
```

These are slower than the rest of the `pytest` tests, and generate screenshots,
browser logs, server logs, and report HTML in `build/robot`.
#### Older Jupyter Clients

To run the acceptance tests against the previous major versions of Notebook
and JupyterLab, it is advisable to use a separate, isolated environment, testing the
as-built assets from `pyproject-build`.

After creating and activating such an environment with `virtualenv` or [`conda`](#conda):

```bash
pip install --find-links ./dist/ --no-index-url jupyter-server-proxy[test,acceptance,classic]
```

## Frontend Development

To support a wide range of clients, both JupyterLab and Notebook Classic extensions
are built and distributed, each with their own quirks.

### JupyterLab/Notebook extension

The `./labextension/` directory contains the extension for the
[`lumino`](https://github.com/jupyterlab/lumino/)-based JupyterLab and Notebook
clients.

#### `nodejs`

Building this extension requires a compatible version of
[`nodejs`](https://nodejs.org/en/download/package-manager), with a supported, long
term support (LTS) release recommended.

### JupyterLab extension
#### `jlpm`

The `jlpm` command is JupyterLab's pinned version of `yarn` that is
installed with JupyterLab.
The `jlpm` command is a vendored, pinned version of the [`yarn`](https://yarnpkg.com)
package manager. Installed with JupyterLab, it performs commands such
as installing `npm` dependencies listed in `labextension/package.json`, building
and watching the extension from source, and formatting web-related source code files.

> You may use `yarn` or `npm run` instead of `jlpm` below.
#### The built Lab extension

During a [`pyproject-build`](https://pypi.org/project/build/)
of the python package, a temporary JupyterLab and `jlpm` will be installed as part
of the `build-system`, executing roughly the commands:

```bash
cd labextension # Change to the root of the labextension
jlpm # Install dependencies (or `npm i`)
jlpm # Install dependencies
jlpm build:prod # Build:
# - `labextension/lib`
# - `jupyter_server_proxy/labextension`
# - `labextension/lib` with type checking
# - `jupyter_server_proxy/labextension` with minimization
```

During `pip install`, the built assets are copied to the user's
`{sys.prefix}/share/jupyter/labextensions/@jupyterhub/jupyter-server-proxy` to be
found by the application at startup.

#### Developing the Lab extension

For fine-grained access to the `jlpm` command and various build steps:

```bash
pip install -e .[lab] # Ensure a compatible jlpm
cd labextension # Change to the root of the labextension
jlpm
jlpm install:extension # Symlink into `{sys.prefix}/share/jupyter/labextensions`
```

You can watch the source directory and automatically rebuild the `labextension/lib`
Watch the source directory and automatically rebuild the `labextension/lib`
and `jupyter_server_proxy/labextension` folders:

```bash
cd labextension
# Watch the source directory in one terminal, automatically rebuilding when needed
jlpm run watch
jlpm watch
# Run JupyterLab in another terminal
jupyter lab
```

With the watch command running, every saved change will immediately be built locally
and available in your running JupyterLab. Refresh JupyterLab to load the change in
your browser (you may need to wait several seconds for the extension to be rebuilt).
While running `jlpm watch`, every saved change to a `.ts` file will immediately be
built locally and available in your running Jupyter client. "Hard" refresh JupyterLab or Notebook
with <kbd>CTRL-F5</kbd> or <kbd>⌘-F5</kbd> to load the change in your browser
(you may need to wait several seconds for the extension to be fully rebuilt).

#### Source Maps

By default, the `jlpm build` and `jlpm watch` commands generate
[source maps](https://firefox-source-docs.mozilla.org/devtools-user/debugger/how_to/use_a_source_map/)
for this extension to improve debugging using the browser development tools,
often revealed by pressing <kbd>F12</kbd>.

By default, the `jlpm run build` command generates the source maps for this
extension to make it easier to debug using the browser dev tools. To also generate
source maps for the JupyterLab core extensions, you can run the following command:
To also generate source maps for the JupyterLab core application, run the following command:

```bash
jupyter lab build --minimize=False
```

### Documentation
### Notebook Classic extension

The files in `jupyter_server_proxy/static` extend the Notebook Classic application's
_Tree_ page.

#### RequireJS

The Notebook Classic extension uses the [`require.js`](https://requirejs.org)
dependency injection system, and presently uses no dependencies beyond what is
provided by Notebook Classic.

#### The built Classic extension

During a user's `pip install`, the static assets are copied to
`{sys.prefix}/share/jupyter/nbextensions/jupyter_server_proxy`, to be
found by the application at startup.

#### Developing the Classic extension

While this extension is served as-is once installed, for live development the
extension assets must be linked:

```bash
pip install -e ".[classic]"
jupyter nbextension install --symlink --sys-prefix --py jupyter_server_proxy
```

After making changes, "hard" refresh the browser application, usually with
<kbd>CTRL-F5</kbd> or <kbd>⌘-F5</kbd>.

## Documentation

The documentation uses a fairly standard [Sphinx](https://www.sphinx-doc.org)
build chain, and requires `make` on Linux/MacOS, which cannot be installed with
`pip`.
`pip`

> `make` is available from [`conda-forge`](#conda) as `make` for Linux/OSX, and `m2-make`
> on Windows
In addition to any system packages, building the documentation requires
additional packages. To install the needed packages:
Expand Down Expand Up @@ -132,3 +254,12 @@ make linkcheck
```bash
make devenv
```

## Linting

During continuous integration (CI) the `pre-commit` package is used to run a
number of checks, with each tool in a private virtual environment. If it is able to,
the CI bot will push to a PR with fixes.

By installing `pre-commit` with `pip` or `conda`, you can have this same experience,
or inspect the configuration to try to recreate it yourself locally.
Loading

0 comments on commit faf66f4

Please sign in to comment.