compat: Support dask query-planning, drop Python 3.9 #171
Conversation
hoxbro
changed the title
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #171 +/- ##
==========================================
- Coverage 77.74% 77.62% -0.12%
==========================================
Files 50 50
Lines 4871 4872 +1
==========================================
- Hits 3787 3782 -5
- Misses 1084 1090 +6 ☔ View full report in Codecov by Sentry. |
| @@ -13,25 +13,22 @@ | |||
| from dask import delayed | |||
| from dask.dataframe.core import get_parallel_type | |||
| from dask.dataframe.extensions import make_array_nonempty | |||
| from dask.dataframe.partitionquantiles import partition_quantiles | |||
There was a problem hiding this comment.
This was removed when they dropped the legacy implementation:
| USE_PYGEOS = '0' | ||
|
|
||
| [environments] | ||
| test-39 = ["py39", "test-core", "test", "test-task", "example", "test-example"] |
There was a problem hiding this comment.
dask 2025.1 only supports Python 3.10.
|
I have tested a dev release with HoloViews and Datashader, and both test suites passed. |
| @@ -86,6 +81,9 @@ test-unit = 'pytest spatialpandas/tests -n logical --dist loadgroup' | |||
| [feature.test-example.dependencies] | |||
| nbval = "*" | |||
|
|
|||
| [feature.test-example.activation.env] | |||
| DASK_SCHEDULER = "single-threaded" | |||
There was a problem hiding this comment.
This is to try to avoid this kind of error (which also happens on main)
edit: It still seems to fail, but I will let it be for now.
| 'pandas', | ||
| 'pyarrow >=10', | ||
| 'pandas >=2.0', | ||
| 'pyarrow >=14.0.1', |
There was a problem hiding this comment.
These two are part of the minimum dependencies of dask
Resolves #146
Resolves #169
I don't want to support the legacy method, so I'm pinning to 2025.1, the first version with the dask-expr part of the dask library. This version only supports Python 3.10 and forward.
I haven't done any performance benchmarks. This is only to get spatialpandas to work with dask going forward.
reference: https://docs.dask.org/en/stable/dataframe-extend.html
The main functionality has been tested for both HoloViews and Datashader here: