Skip to content

CFA: Create Functionality #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Oct 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
cbcf7df
Changes in response to python gotchas
dwest77a Sep 25, 2024
055297c
Updated method for substitutions with multiple filenames
dwest77a Sep 25, 2024
47031f3
Initial commit of creator script
dwest77a Sep 25, 2024
a3442ef
Refactored creator to fix dimension issues, added extra steps to adhe…
dwest77a Sep 27, 2024
b6eafa7
Updated version, removed attempts to capture args
dwest77a Sep 27, 2024
4ca1692
Added logging to all functions, added ArrayMixin, cleaned up creator.py
dwest77a Oct 8, 2024
e2212d5
Updated version number
dwest77a Oct 8, 2024
edc9185
Added initial documentation for creator tool
dwest77a Oct 8, 2024
397cafe
Updated workflow action versions
dwest77a Oct 9, 2024
dc1fbbf
Update removes unneeded fragment addresses
dwest77a Oct 9, 2024
67f5f42
Organised Integration tests with pytest dependencies
dwest77a Oct 11, 2024
5185fbf
Upgraded to using poetry, upgraded to new version
dwest77a Oct 11, 2024
606a3ea
Renamed package for poetry usage
dwest77a Oct 11, 2024
fc7b848
Renamed package for poetry usage
dwest77a Oct 11, 2024
9119d03
Renamed to cfapyx, added creator documentation
dwest77a Oct 11, 2024
ea519df
Changed to cfapyx
dwest77a Oct 11, 2024
0ca86d0
Added variable/dimension properties
dwest77a Oct 11, 2024
8282c3f
Now can check properties before writing
dwest77a Oct 11, 2024
37b2be5
Added note on create new functionality
dwest77a Oct 11, 2024
09b8a1c
Ammended docs, added test for doc fail in backend
dwest77a Oct 11, 2024
1f67b6d
Updated poetry with sphinx version
dwest77a Oct 11, 2024
16605d8
Updated to use poetry
dwest77a Oct 11, 2024
d45841b
Upgraded docs workflow for poetry
dwest77a Oct 11, 2024
37580f7
Edited directory for documentation generation
dwest77a Oct 11, 2024
0789372
Edited directory for documentation generation 2
dwest77a Oct 11, 2024
703fd6e
Edited directory for documentation generation 3
dwest77a Oct 11, 2024
c10b1ef
Updated docs for creator module
dwest77a Oct 11, 2024
934ce45
Updated version
dwest77a Oct 11, 2024
7bfe702
Updated docs for inspiration, added new pie chart
dwest77a Oct 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,19 @@ jobs:

# Firstly, checkout repo
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v4
# Set up Python env
- name: Setup Python
uses: actions/setup-python@v2
uses: actions/setup-python@v5
with:
python-version: 3.11
# Install dependencies
- name: Install Python dependencies
run: |
python3 -m pip install --upgrade pip
pip3 install -r requirements.txt
pip3 install -e .
pip3 install poetry
poetry install
# Test with pytest
- name: Run pytest
run: |
pytest
poetry run pytest
64 changes: 39 additions & 25 deletions .github/workflows/sphinx.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,32 +12,46 @@ on:

jobs:
build:
# Linux version requirements
# Specify an OS for the runner
runs-on: ubuntu-latest

#Define steps
steps:
# Checkout and build the docs with sphinx
- uses: actions/checkout@v2

- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: "3.10"
# Firstly, checkout repo
- name: Checkout repository
uses: actions/checkout@v4
# Set up Python env
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.11
# Install dependencies
- name: Install Python dependencies
run: |
python3 -m pip install --upgrade pip
pip3 install poetry
poetry install

- name: Build documentation
run: |
mkdir gh-pages
touch gh-pages/.nojekyll
pushd docs/source/
poetry run sphinx-build -b html . _build
popd
cp -r docs/source/_build/* gh-pages/

- name: Build HTML
uses: ammaraskar/sphinx-action@master
with:
docs-folder: "docs/"
# pre-build-command: "mkdir /tmp/sphinx-log"
- name: Upload artifacts
uses: actions/upload-artifact@v1
with:
name: html-docs
path: docs/build/html/
# Deploys to the gh-pages branch if the commit was made to main, the
# gh-pages then takes over serving the html
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
if: github.ref == 'refs/heads/main'
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: docs/build/html
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: html-docs
path: gh-pages
# Deploys to the gh-pages branch if the commit was made to main, the
# gh-pages then takes over serving the html
- name: Deploy documentation
if: ${{ github.event_name == 'push' }}
uses: JamesIves/[email protected]
with:
branch: gh-pages
folder: gh-pages
1 change: 0 additions & 1 deletion CFAPyX/__init__.py

This file was deleted.

8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,20 @@
CFA python Xarray module for using CFA files with xarray.

See the [Documentation](https://cedadev.github.io/CFAPyX/) for more details.
CFAPyX on [Github](https://github.com/cedadev/CFAPyX)
cfapyx on [Github](https://github.com/cedadev/CFAPyX)

For use with the Xarray module as an additional backend.

> **_NOTE:_** The `create` functionality was added to version 2024.10.11 and is currently in alpha release. Please report any unexpected errors or issues using the GitHub Issues tab for this repository.

# Installation

```
pip install xarray==2024.6.0
pip install CFAPyX
pip install cfapyx
```

# Usage
# Usage as Xarray Engine

```
import xarray as xr
Expand Down
3 changes: 3 additions & 0 deletions cfapyx/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from .backend import CFANetCDFBackendEntrypoint

from .creator import CFANetCDF
17 changes: 12 additions & 5 deletions CFAPyX/backend.py → cfapyx/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@
from xarray.core.dataset import Dataset
from xarray import conventions

from CFAPyX.datastore import CFADataStore
from cfapyx.datastore import CFADataStore

import logging

logger = logging.getLogger(__name__)

def open_cfa_dataset(
filename_or_obj,
Expand All @@ -18,7 +22,7 @@ def open_cfa_dataset(
decode_coords=None,
use_cftime=None,
decode_timedelta=None,
cfa_options={},
cfa_options: dict=None,
group=None,
):
"""
Expand All @@ -42,6 +46,8 @@ def open_cfa_dataset(
parameter in ``cfa_options`` is false.
"""

cfa_options = cfa_options or {}

# Load the CFA datastore from the provided file (object not supported).
store = CFADataStore.open(filename_or_obj, group=group)

Expand Down Expand Up @@ -70,7 +76,7 @@ def open_cfa_dataset(

class CFANetCDFBackendEntrypoint(BackendEntrypoint):

description = "Open CFA-netCDF files (.nca) using CFA-PyX in Xarray"
description = 'Open CFA-netCDF files (.nca) using "cfapyx" in Xarray'
url = "https://cedadev.github.io/CFAPyX/"

def open_dataset(
Expand All @@ -84,7 +90,7 @@ def open_dataset(
decode_coords=None,
use_cftime=None,
decode_timedelta=None,
cfa_options={},
cfa_options=None,
group=None,
# backend specific keyword arguments
# do not use 'chunks' or 'cache' here
Expand All @@ -94,6 +100,8 @@ def open_dataset(
CFA aggregated variables into proper arrays.
"""

cfa_options = cfa_options or {}

return open_cfa_dataset(
filename_or_obj,
drop_variables=drop_variables,
Expand All @@ -113,7 +121,6 @@ class CFAStoreBackendEntrypoint(StoreBackendEntrypoint):
def open_dataset(
self,
cfa_xarray_store,
*,
mask_and_scale=True,
decode_times=True,
concat_characters=True,
Expand Down
Loading