Bytewax InfluxDB Connector

A source and sink connector for Bytewax.

You will need an influxdb account and the API key to use the connector.

To test the connector, use examples/input.py and examples/output.py. You may need to modify the date parameters for it to work.

To use the source and sink connector use them in the input and output operators like shown below.

inp = op.input(
    "inp",
    flow,
    InfluxDBSource(
        timedelta(seconds=5),
        "https://us-east-1-1.aws.cloud2.influxdata.com",
        DATABASE,
        TOKEN,
        "home",
        ORG,
        datetime.now(timezone.utc) - timedelta(days=5),
    ),
)
op.output(
    "out",
    inp,
    InfluxDBSink(
        host="https://us-east-1-1.aws.cloud2.influxdata.com",
        database=DATABASE,
        org=ORG,
        token=TOKEN,
        write_precision="s",
    ),
)

Setting up the project for development

Install just

We use just as a command runner for actions / recipes related to developing Bytewax. Please follow the installation instructions. There's probably a package for your OS already.

Install pyenv and Python 3.12

I suggest using pyenv to manage python versions. the installation instructions.

You can also use your OS's package manager to get access to different Python versions.

Ensure that you have Python 3.12 installed and available as a "global shim" so that it can be run anywhere. The following will make plain python run your OS-wide interpreter, but will make 3.12 available via python3.12.

$ pyenv global system 3.12

Install uv

We use uv as a virtual environment creator, package installer, and dependency pin-er. There are a few different ways to install it, but I recommend installing it through either brew on macOS or pipx.

Install just

We use just as a command runner for actions / recipes related to developing Bytewax. Please follow the installation instructions. There's probably a package for your OS already.

Development

We have a just recipe that will:

1. Set up a venv in venvs/dev/.

2. Install all dependencies into it in a reproducible way.

Start by adding any dependencies that are needed into pyproject.toml or into requirements/dev.in if they are needed for development.

Next, generate the pinned set of dependencies with

> just venv-compile-all

Create and activate a virtual environment

Once you have compiled your dependencies, run the following:

> just get-started

Activate your development environment and run the development task:

> . venvs/dev/bin/activate
> just develop

License

Bytewax InfluxDB Connector is commercially licensed with publicly available source code. Please see the full details in LICENSE.