visualization-notebooks

Jupyter Notebooks for 4CE data visualizations.

Development

First time setup and installation

Set up git hook to clear notebook outputs before commit:

cp ./pre-commit.sh ./.git/hooks/pre-commit
chmod +x ./.git/hooks/pre-commit

Set up environments using Conda and enable Jupyter widgets:

conda env create -f environment.yml
conda env create -f environment.r.yml

jupyter nbextension enable --py --sys-prefix lineup_widget

To use real datasets, create a data folder:

mkdir data

Add data files inside this folder, and make sure to properly set the data directory in notebooks/constants.py:

DATA_DIR = join("..", "data")

Running and editing notebooks

Enter the notebook directory:

cd notebooks

Start Jupyter with the Python (or R) kernel:

conda activate 4ce # or 4ce-r
jupyter notebook # or jupyter lab

Conversion from Altair to Vega-Lite for the Web

Plots created with Altair can be converted to the Vega-Lite JSON format, which enables these plots to be used on the website.

To streamline this process, any plots that should be used on the website should be passed to the for_website() function (defined here).

In a notebook, this would look like:

from web import for_website

plot = alt.Chart(df).mark_point().encode(x='day',y='num_patients')

# Specify the notebook name, and give each plot a unique name.
for_website(plot, "Labs", "Lab Values by Site")

plot

By default, the for_website function does nothing. Only when the environment variable C19_SAVE_FOR_WEB_DIR is set on your computer, will the output JSON files be generated.

This environment variable can be set like this:

export C19_SAVE_FOR_WEB_DIR=~/research/dbmi/covid/plots_for_website

Troubleshooting

Updating a conda environment

To add or remove a conda package, update the corresponding environment file. Then run

conda env update -f environment.yml # or environment.r.yml

Exporting

To export a notebook to HTML, with only the outputs and no code, run:

jupyter nbconvert --to html --no-input path/to/notebook.ipynb