Skip to content
This repository was archived by the owner on Mar 10, 2025. It is now read-only.
/ soopervisor Public archive

☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.

License

Notifications You must be signed in to change notification settings

ploomber/soopervisor

Folders and files

NameName
Last commit message
Last commit date

Latest commit

254d1e6 · Mar 10, 2025
Dec 29, 2021
Sep 18, 2024
Jan 5, 2024
Sep 18, 2024
Sep 18, 2024
Dec 29, 2022
Sep 26, 2022
Oct 20, 2022
Sep 18, 2024
Aug 15, 2022
Jan 5, 2023
Feb 23, 2022
Mar 10, 2025
Jun 9, 2022
Dec 31, 2021
Dec 29, 2022
Jul 18, 2023
Jun 9, 2022
Sep 18, 2024

Repository files navigation

Important: This project is no longer maintained.

Soopervisor

CI badge CI macOS badge CI Windows badge

Tip: Deploy AI apps for free on Ploomber Cloud!

Soopervisor runs Ploomber pipelines for batch processing (large-scale training or batch serving) or online inference.

pip install soopervisor

Check out the documentation to learn more.

Compatible with Python 3.7 and higher.

Supported platforms

From notebook to a production pipeline

We also have an example that shows how to use our ecosystem of tools to go from a monolithic notebook to a pipeline deployed in Kubernetes.

Usage

Say that you want to train multiple models in a Kubernetes cluster, you may create a new target environment to execute your pipeline using Argo Workflows:

soopervisor add training --backend argo-workflows

After filling in some basic configuration settings, export the pipeline with:

soopervisor export training

Depending on the selected backend (Argo, Airflow, AWS Batch, or AWS Lambda), configuration details will change, but the API remains the same: soopervisor add, then soopervisor export.

About Ploomber

Ploomber is a big community of data enthusiasts pushing the boundaries of Data Science and Machine Learning tooling.

Whatever your skillset is, you can contribute to our mission. So whether you're a beginner or an experienced professional, you're welcome to join us on this journey!

Click here to know how you can contribute to Ploomber.