Skip to content

Commit 49a3093

Browse files
committed
Update to DataFusion 42, Arrow 53.1 (RC), remove DataFrameDataSource Python logic (#513)
* Initial update to DataFusion 42 and Arrow 53.1 * Remove protobuf python code * Remove python protobuf instructions * fix jsonwriter tests * fix select tests * more sort test fixes * more test fixes * test / clippy fixes * more test/lint fixes * clippy/fmt fixes * Remove order col after evaluation * clippy fix * fixes * update baselines * fix sql tests * update baselines * sort * fmt * use published arrow crates
1 parent 668dfc8 commit 49a3093

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

68 files changed

+1651
-3027
lines changed

BUILD.md

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -103,16 +103,3 @@ pixi add my-new-package --build
103103
```
104104

105105
This will install the package and update the pixi.lock file with the new environment solution.
106-
107-
## Updating DataFusion protobuf
108-
When updating the version of DataFusion that VegaFusion depends on, the protobuf definition file at `python/vegafusion/proto/datafusion.proto` must be updated to match. This file should be downloaded from GitHub from https://raw.githubusercontent.com/apache/arrow-datafusion/27.0.0/datafusion/proto/proto/datafusion.proto, replacing `27.0.0` with the matching version of DataFusion.
109-
110-
Then Python bindings should be generated using `protoc`
111-
112-
```
113-
pixi shell
114-
cd vegafusion-python
115-
protoc --python_out=vegafusion ./proto/datafusion.proto
116-
```
117-
118-
This will update the `vegafusion-python/vegafusion/proto/datafusion_pb2.py` file.

0 commit comments

Comments
 (0)