Skip to content

Contributing

Jump to bottom
Benjamin Räthlein edited this page Jun 11, 2024 · 80 revisions

Thanks for your interest in helping improve Streamlit! 🎉

🤓 If you are looking for Streamlit's documentation, go here instead: https://docs.streamlit.io/

This wiki is for people who want to contribute code to Streamlit. There are also other ways to contribute, such as reporting bugs, creating feature requests, helping other users in our forums, Stack Overflow, etc., or just being an awesome member of the community!

Before contributing

If your contribution is more than a few lines of code, then prior to starting to code on it please post in the issue saying you want to volunteer, and then wait for a positive response. And if there is no issue for it yet, create it first.

This helps make sure:

  1. Two people aren't working on the same thing
  2. This is something Streamlit's maintainers believe should be implemented/fixed
  3. Any API, UI, or deeper architectural changes that need to be implemented have been fully thought through by Streamlit's maintainers
  4. Your time is well spent!

To be clear: if you open a PR that adds a new feature (and isn't just a bug fix or similar) without prior support from the Streamlit team, the chances of getting it merged are extremely low. Adding a new feature comes with a lot of baggage, such as thinking through the exact API, making sure it fulfills our standards, and maintaining it in the future – even if it's just a small parameter.

Style Guide

Check out Streamlit's style guide. We use Prettier and Ruff to autoformat code on presubmit, but some things go beyond what autoformatters can do. So please take a look!

How to build Streamlit

1. Set up your base environment

MacOS

# Some Apple dev tools (developer.apple.com/downloads)
$ xcode-select --install

# Install Homebrew
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Install the Protobuf compiler, graphviz, database, and virtual environment
$ brew install protobuf graphviz gawk mysql mysql-client pkg-config unixodbc postgresql pre-commit

Installing Node JS and yarn

We recommend that you manage your nodejs installation with nvm. After following the instructions linked above to install nvm, use the following command to install the latest supported node version

# Install node
nvm install node

Note: Node has added Corepack which is a manager of package managers 😱. It supports yarn! You can enable it by running the following:

corepack enable

You may need to brew install corepack depending on how you installed node.

ARM based Macs

If you are running a MacOS computer running on the new chipsets (e.g. M1), you may hit some trouble installing pyodbc This can be solved by installing unixodbc and setting some flags. See this comment for help. We found that it just needs to be set once. Be sure to make sure you have the correct version of unixodbc in the commands.

Ubuntu

# Set up the Yarn repo
## If `node -v` shows a version < 16.10
$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
$ source ~/.bashrc
$ nvm install node

## Install yarn
$ corepack enable
$ corepack prepare yarn@stable --activate

$ sudo apt-get update

# Install Pyenv for testing multiple Python versions
$ sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python3-openssl mysql-client libmysqlclient-dev unixodbc-dev
$ curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash

# Install some other deps
$ sudo apt install graphviz python3-distutils pre-commit

# Install Yarn, pip, Protobuf, npm
$ sudo apt install python3-pip protobuf-compiler libgconf-2-4
# (libgconf is for our e2e tests in Cypress)

Probably not needed, but in case you want to update your Node installation:

curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -

Windows

Streamlit's development setup is pretty Mac- and Linux-centric. If you're doing Streamlit development on Windows, we suggest spinning up a Linux VM (you can do this easily with VirtualBox, which is free); or working in a Linux Docker image.

Alternately, you can try using Microsoft's WSL ("Windows Subsystem for Linux"), which may work fine, or may result in you slowly dying from thousands of tiny papercuts (these steps were last tested on 2019-11-18):

# Install a Python version using pyenv with the `CONFIGURE_OPTS=--enable-shared` flag set:
$ CONFIGURE_OPTS=--enable-shared pyenv install 3.7.5 && pyenv global 3.7.5

$ python -m venv venv

Pre-commit hooks

When Streamlit's pre-commit detects that one of the linters has failed, it automatically lints the files and does not allow the commit to pass. Please review the changes after lint has failed and commit them again, the second commit should pass, because the files were linted after trying to do the first commit.

But you can run pre-commit hooks manually as needed.

  • Run all checks on your staged files by using: 
    pre-commit run
  • Run only isort check on your staged files by using: 
    pre-commit run isort
  • Run only isort checks on all files by using: 
    pre-commit run isort --all-files
  • Run all checks on all files by using: 
    pre-commit run --all-files

2. Grab the code

(You probably already know how to do this, but just in case...)

First fork the repo via the UI on Github and then do the following:

$ git clone https://github.com/${YOUR_NAME}/streamlit.git
$ cd streamlit
$ git remote add remote https://github.com/streamlit/streamlit.git
$ git checkout develop
$ git submodule update --init
$ git checkout -b ${BRANCH_NAME}

3. Create a new Python environment

Create a virtual environment for Streamlit using your favorite tool (virtualenv, pipenv, etc) and activate it. Here's how we do it with venv:

$ cd lib
$ python -m venv venv

Note that, with venv this process should be done from any directory, but it is recommended to do it from the lib/ directory to keep all python files in one directory.

$ source ./venv/bin/activate

4. (Optional) Install conda and conda-build

If you want to build Streamlit as a conda package on your local machine (needing to do this should be rare), you'll need to install a few extra dependencies so that the make conda-package target works.

  1. First, install conda using your favorite package manager or by following these instructions. Both anaconda and miniconda will work.
  2. Then, run conda install conda-build.

How to develop Streamlit

The basic developer workflow is that you run a React development server on port 3000 in one terminal and run Streamlit CLI commands in another terminal.

So that's:

1. One-time setup

$ make all-devel

Alternatively, if you're not planning on running all our tests locally, you can avoid installing a million test-only dependencies by calling this instead:

$ make mini-devel   # Alternative to "all-devel"

What all-devel does behind the scenes:

  1. make init to install Streamlit's Python and Javascript dependencies, and compile our Protobufs.
    • If you used mini-devel, then you'll install fewer Python dependencies. That's the only difference!
  2. make install to build and install Streamlit into your Python environment.
  3. make develop to make it so you can edit the Python source files inside lib/streamlit/ and not have to reinstall the Python package with make install every time.

(There's also a make all, which does a make all-devel plus a make build — which builds our static assets for when we want to ship the library to users. But this takes a long time and can be avoided for most development.)

2. Start the dev server

From a new terminal, run:

$ cd frontend
$ yarn start

Note that this server listens on port 3000 rather than 8501 (i.e. Streamlit's production port). Normally you don't have to worry about this, but it may matter when you're developing certain features.

3. Make sure your OS protoc version and Python's protobuf are the same version

Open another terminal, start your Python environment and make sure protoc version and Python's protobuf are the same version.

If you're using venv, that's:

$ cd lib
$ source ./venv/bin/activate

# Print out your System's Protoc version
$ protoc --version

# Print out your Python's environment Protobuf version, for example: 3.19.4
$ pip show protobuf

If the versions match, you're good to go, otherwise one need to install protoc in the same version as Python's environment protobuf.

MacOS

# Uninstall incompatible protobuf version
$ brew uninstall --ignore-dependencies protobuf
$ curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.20.0/protoc-3.20.0-osx-x86_64.zip
$ sudo unzip -o protoc-3.20.0-osx-x86_64.zip -d /usr/local bin/protoc
$ sudo unzip -o protoc-3.20.0-osx-x86_64.zip -d /usr/local 'include/*'
# Print out your System's Protoc version
$ protoc --version
On Linux (ARM) that's: 
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.20.0/protoc-3.20.0-linux-aarch_64.zip
sudo unzip -o protoc-3.20.0-linux-aarch_64.zip -d /usr/local bin/protoc
sudo unzip -o protoc-3.20.0-linux-aarch_64.zip -d /usr/local 'include/*'

# (optional) remove old version
rm /usr/bin/protoc
ln -s /usr/local/bin/protoc /usr/bin/protoc

# Print out your System's Protoc version
protoc --version
  • At last confirm that System's Protoc version and Python's environment Protobuf version are the same
$ cd lib
$ source ./venv/bin/activate

# Print out your System's Protoc version
$ protoc --version

# Print out your Python's environment Protobuf version, for example: 3.19.4
$ pip show protobuf

4. Run Streamlit

Open another terminal, start your Python environment and run Streamlit.

If you're using venv, that's:

$ cd lib
$ source ./venv/bin/activate
$ cd ..

# Now run any Streamlit command you want, such as:
$ streamlit hello

# If you want to run Streamlit without csrf protection for development purposes:
$ streamlit hello --server.enableXsrfProtection=false

5. What to do when you modify some code

When you modify JS, or CSS code...

Since we use that awesome dev server above, when you change any JS/CSS code everything should automatically just work without the need to restart any of the servers.

When you modify Python code...

When you modify Python code, you should kill the old Streamlit server, if any (ctrl-c on the terminal) and then restart it.

When you update protobufs...

If you ever modify our protobufs, you'll need to run the command below to compile the protos into libraries that can be used in Python and JS:

$ make protobuf

When Javascript or Python dependencies change...

make init

6. Running tests

You should always write unit tests and end-to-end tests! This true for new features, but also for bugs; this way when you fix a bug you can be sure it will not show up again. So bug-fixing is actually a great way to increase our test coverage where it actually matters.

Python unit tests

  • Run all with:

    $ make pytest

  • Run a specific test file with:

    $ PYTHONPATH=lib pytest lib/tests/streamlit/the_test_name.py

  • Run a specific test inside a test file with:

    $ PYTHONPATH=lib pytest lib/tests/streamlit/the_test_name.py -k test_that_something_works

  • Some tests require you to set up credentials to connect to Snowflake and install the snowflake-snowpark-python package, which is only compatible with Python 3.8. They are skipped by default when running tests. To enable them and disable all others, pass the --require-snowflake flag to pytest.

    $ PYTHONPATH=lib pytest --require-snowflake

JS unit tests

  • Run all with:

    $ make jstest

  • Run specific tests:

    $ cd frontend
$ yarn workspace @streamlit/lib test src/path/to/test_file.test.ts

NOTE: Making changes to a react component may cause unit snapshot tests (which are designed to catch unintended changes to jsx/tsx components) to fail. Once you've double-checked that all of the changes in the failing snapshot test are expected, you can follow the prompts that appear after running make jstest to update the snapshots, check them into source control, and include them in your PR.

End-to-end tests

End-to-end tests are tricky to work with due to how sensitive they are to small environment changes. Because of this, we have instructions for test environment setup and running e2e tests in a separate Running e2e tests and updating snapshots document.

7. Formatting, linting, and type-checking

We've set up various formatting, linting, and type-checking rules that our Continuous Integration checks to maintain code quality and consistency. Before merging a Pull Request, all formatting and linting rules must be satisfied and passed successfully.

Python

For Python, we use ruff for formatting & linting and mypy for type-checking.

Formatting

To format all Python code & sort the imports, run the following command:

make pyformat

Alternatively, you can use the ruff format command directly.

Linting

To run the linter, use the command below:

make pylint

Alternatively, you can use the ruff check command directly.

Type-checking

For type-checking, run:

make mypy

VS-Code Settings

For development in VS Code, we recommend installing the ruff and mypy extensions with the following configuration (in the settings.json):

{
    "[python]": {
        "editor.defaultFormatter": "charliermarsh.ruff",
        "editor.codeActionsOnSave": {
            "source.organizeImports": "explicit"
        },
        "editor.formatOnSave": true,
        "editor.formatOnPaste": true,
        "editor.formatOnType": true,
    },
    "ruff.organizeImports": true,
    "mypy-type-checker.importStrategy": "fromEnvironment",
}

Javascript / Typescript

For Javascript/Typescript, we utilize Prettier and ESLint.

Formatting

To format your code, run this command:

make jsformat

Linting

To initiate the linting process, use this command:

make jslint

Type-checking

For type-checking, run:

make tstypecheck

Troubleshooting

  • On Mac OS Mojave, pyenv fails to install python versions with "zlib not available"

    $ xcode-select --install
$ sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

  • Unable to run unittests (pytest not installed)

    Ensure the Pipfile's dev packages are installed

    $ cd lib
$ source ./venv/bin/activate
$ make python-init-dev-only

  • UserWarning: Failed to load image Python extension ...torchvision/image.so. Expected in ...site-packages/torch/lib/libtorch_cpu.dylib

    Ensure missing libraries (in this case libtorch_cpu.dylib) are present in expected location of your Python environment.

    MacOS:

    $ brew install libtorch

# Replace path of this command to match your's system environment
$ cd lib
$ cp /usr/local/lib/libtorch_cpu.dylib ./venv/lib/python3.8/site-packages/torch/lib/

  • Test test_streamlit_version fails, for example:

def test_streamlit_version(self):
        """Test streamlit.__version__."""
        self.assertEqual(__version__, get_version())
        AssertionError: '1.11.0' != '1.11.1'
        - 1.11.0
        ?      ^
        + 1.11.1
        ?      ^

To fix this make sure you have setup your Python's venv environments correctly, upgrade your dependencies or recreate your environment and repeat setup.

You might have double environments which mismatch for example by accident you could have created venv Python environment inside streamlit repository and second one inside streamlit/lib. Remove them

# cd into streamlit repository main catalogue
$ cd streamlit

# You might need to update your branch, for example: develop
$ git pull origin develop

# remove Python's environment in streamlit repository main directory
$ rm -rf venv 

$ cd lib

# remove Python's environment in streamlit lib directory
$ rm -rf venv

# repeat steps related to Python's environment from the start of Contributing tutorial
$ python3.8 -m venv
$ sources ./venv/bin/activate

# cd into streamlit repository main directory
$ cd ..

$ make all-devel

# Now you should be good to go again, check it out with:
$ make pytest

Introducing dependencies

We aim to only introduce dependencies in this project that have reasonable restrictions and comply with various laws. Code licensed outside of BSD and MIT must be approved by @jroes (who will subsequently get legal guidance).

Contributions that modify Python dependencies are tested on all supported Python versions. For details, see: CI Environment

Clone this wiki locally