Skip to content
forked from paradedb/wrappers

Postgres Foreign Data Wrapper development framework in Rust.

License

Notifications You must be signed in to change notification settings

kysshsy/wrappers

 
 

Repository files navigation

Wrappers

Wrappers is a development framework for Postgres Foreign Data Wrappers (FDW), written in Rust. Its goal is to make Postgres FDW development easier while keeping Rust language's modern capabilities, such as high performance, strong types, and safety.

Wrappers is also a collection of FDWs built by Supabase. We currently support the following FDWs, with more under development:

FDW Description Read Modify
HelloWorld A demo FDW to show how to develop a basic FDW.
BigQuery A FDW for Google BigQuery
Clickhouse A FDW for ClickHouse
Stripe A FDW for Stripe API
Firebase A FDW for Google Firebase
Airtable A FDW for Airtable API
S3 A FDW for AWS S3
Logflare A FDW for Logflare
Auth0 A FDW for Auth0
SQL Server A FDW for Microsoft SQL Server
Redis A FDW for Redis
AWS Cognito A FDW for AWS Cognito

Features

  • Minimum interface and easy to implement.
  • Support for rich data types.
  • Support both sync and async backends, such as RDBMS, RESTful APIs, flat files and etc.
  • Built on top of pgrx, providing higher level interfaces, without hiding lower-level C APIs.
  • WHERE, ORDER BY, LIMIT pushdown are supported.

Documentation

Installation

Wrappers is a pgrx extension, you can follow the pgrx installation steps to install Wrappers.

Basically, run below command to install FDW after pgrx is installed. For example,

cargo pgrx install --pg-config [path_to_pg_config] --features stripe_fdw

Developing a FDW

To develop a FDW using Wrappers, you only need to implement the ForeignDataWrapper trait.

pub trait ForeignDataWrapper {
    // create a FDW instance
    fn new(...) -> Self;

    // functions for data scan, e.g. select
    fn begin_scan(...);
    fn iter_scan(...) -> Option<Row>;
    fn end_scan(...);

    // functions for data modify, e.g. insert, update and delete
    fn begin_modify(...);
    fn insert(...);
    fn update(...);
    fn delete(...);
    fn end_modify(...);

    // other optional functions
    ...
}

In a minimum FDW, which supports data scan only, new(), begin_scan(), iter_scan() and end_scan() are required, all the other functions are optional.

To know more about FDW development, please visit the Wrappers documentation.

Basic usage

These steps outline how to use the a demo FDW HelloWorldFdw, which only outputs a single line of fake data:

  1. Clone this repo
git clone https://github.com/supabase/wrappers.git
  1. Run it using pgrx with feature:
cd wrappers/wrappers
cargo pgrx run pg15 --features helloworld_fdw
  1. Create the extension, foreign data wrapper and related objects:
-- create extension
create extension wrappers;

-- create foreign data wrapper and enable 'HelloWorldFdw'
create foreign data wrapper helloworld_wrapper
  handler hello_world_fdw_handler
  validator hello_world_fdw_validator;

-- create server and specify custom options
create server my_helloworld_server
  foreign data wrapper helloworld_wrapper
  options (
    foo 'bar'
  );

-- create an example foreign table
create foreign table hello (
  id bigint,
  col text
)
  server my_helloworld_server
  options (
    foo 'bar'
  );
  1. Run a query to check if it is working:
wrappers=# select * from hello;
 id |    col
----+-------------
  0 | Hello world
(1 row)

Running tests

In order to run tests in wrappers:

docker-compose -f .ci/docker-compose.yaml up -d
cargo pgrx test --features all_fdws,pg15

Limitations

  • Windows is not supported, that limitation inherits from pgrx.
  • Currently only supports PostgreSQL v14, v15 and v16.
  • Generated column is not supported.

Contribution

All contributions, feature requests, bug report or ideas are welcomed.

License

Apache License Version 2.0

crates.io badge docs.rs badge Test Status MIT/Apache-2 licensed Contributors

About

Postgres Foreign Data Wrapper development framework in Rust.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Rust 98.7%
  • Python 1.1%
  • Other 0.2%