Package Modernization (#21)

* scripts treated as module

* conda recipe verified

* bashcomplete

* spec attempts... moving to bl100-analysis0

* test

* rpm builds

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* ignore lint

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* PR comment response - change tool.black to tool.ruff

* add toml to environment.yml

* Adding in configuration for versioningit

* Remove second declaration of ruff configuration

* Change how tarball file is found and improve error message

* Add two more packages for building

* Removing init because this is not a package

* Change how things work

The final metadata in the wheel is somehow wrong

* Remove a bonus character

* Run builds on pull requests

* conda package builds locally

* add scripts/finddata to manifest

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* try with scripts/__init__.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* move cli - wheel installs locally

* Give clearer message if no instrument is specified

* add wheel validation to GH workflow

* Update .spec for entry point refactor; build versioning seems resolved

* Exit if copy fails

* Drop hh:mm from build version

* Ignore generated file

* Comment out argcomplete which does not exist as rhel9 python3.11 package yet

* Add patchfile and do lots of changes in specfile

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Set version obsoleted

* Remove debug print call

* Match python version used on rhel9

* Add shellcheck and fix errors

* Add pre-commit so developers can use it

* Add conda-forge channel

Co-authored-by: Jose Borreguero <[email protected]>

* Add more dependencies

Co-authored-by: Jose Borreguero <[email protected]>

* Add plotly to developer env and conda package

Co-authored-by: Jose Borreguero <[email protected]>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Pete Peterson <[email protected]>
Co-authored-by: Jose Borreguero <[email protected]>

.github/workflows/package.yml

@@ -0,0 +1,49 @@
+name: conda packaging and deployment
+
+on:
+  workflow_dispatch:
+  pull_request:
+  push:
+    branches: [main]
+    tags: ['v*']
+
+jobs:
+  linux:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        shell: bash -l {0}
+    steps:
+      - uses: actions/checkout@v3
+      - uses: conda-incubator/setup-miniconda@v2
+        with:
+          auto-update-conda: true
+          channels: conda-forge,defaults
+          mamba-version: "*"
+          environment-file: environment.yml
+          cache-environment-key: ${{ runner.os }}-env-${{ hashFiles('**/environment.yml') }}
+          cache-downloads-key: ${{ runner.os }}-downloads-${{ hashFiles('**/environment.yml') }}
+      - name: build and verify wheel
+        run: |
+          python -m build --wheel --no-isolation
+          check-wheel-contents dist/finddata-*.whl
+      - name: build conda package
+        run: |
+          # set up environment
+          cd conda.recipe
+          echo "versioningit $(versioningit ../)"
+          # build the package
+          VERSION=$(versioningit ../) conda mambabuild -c conda-forge --output-folder . .
+          conda verify noarch/finddata*.tar.bz2
+      - name: upload conda package to anaconda
+        shell: bash -l {0}
+        if: startsWith(github.ref, 'refs/tags/v')
+        env:
+          ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_TOKEN }}
+          IS_RC: ${{ contains(github.ref, 'rc') }}
+        run: |
+          # label is main or rc depending on the tag-name
+          CONDA_LABEL="main"
+          if [ "${IS_RC}" = "true" ]; then CONDA_LABEL="rc"; fi
+          echo pushing ${{ github.ref }} with label $CONDA_LABEL
+          anaconda upload --label $CONDA_LABEL conda.recipe/noarch/finddata*.tar.bz2

.gitignore

@@ -2,6 +2,8 @@
 
 # place the rpm gets built from
 tarsrc
+src/finddata/_version.py
+finddata*.tar.gz
 
 # C extensions
 *.so

.pre-commit-config.yaml

@@ -12,6 +12,12 @@ repos:
       exclude: "tests/cis_tests/.*"
     - id: trailing-whitespace
       exclude: "tests/cis_tests/.*"
+- repo: https://github.com/shellcheck-py/shellcheck-py
+  rev: v0.9.0.2
+  hooks:
+  - id: shellcheck
+    # follow sources even when they aren't specified on the command line
+    args: ['--external-sources']
 - repo: https://github.com/astral-sh/ruff-pre-commit
   rev: v0.5.5
   hooks:

MANIFEST.in

@@ -1,3 +1,3 @@
-include finddata finddata.bashcomplete README.md
-include versioneer.py
-include finddata/_version.py
+include src/finddata finddata.bashcomplete README.md
+include src/finddata/cli.py
+include src/finddata/_version.py

README → README.md

@@ -8,12 +8,8 @@ Put `finddata` somewhere in your path (i.e. `/usr/local/bin`).
 If you want to use `finddata.publish_plot()`, the configuration file `/etc/autoreduce/post_processing.conf` must be in place with the url and credentials.
 It is advised that this be readable only by the process using the functionality.
 
+`python -m build; pip install .`
+
 Optional bash completion
 ------------------------
-Put `finddata.bashcomplete` in `/etc/bash_completion.d/`.
-
-Development
------------
-```
-python -c "import setuptools; execfile('setup.py')" develop
-```
+`pip install .\[bashcompletion\]

conda.recipe/meta.yaml

@@ -0,0 +1,51 @@
+# load information from pyproject.toml
+{% set pyproject = load_file_data('pyproject.toml') %}
+{% set project = pyproject.get('project', {}) %}
+{% set license = project.get('license').get('text') %}
+{% set description = project.get('description') %}
+{% set project_url = pyproject.get('project', {}).get('urls') %}
+{% set url = project_url.get('homepage') %}
+# this will get the version set by environment variable
+{% set version = environ.get('VERSION') %}
+{% set version_number = version.split('+')[0] %}
+{% set git_describe_number = environ.get('GIT_DESCRIBE_NUMBER', '0') | string %}
+
+package:
+  name: finddata
+  version: {{ version_number }}
+
+source:
+  path: ..
+
+build:
+  noarch: python
+  number: {{ git_describe_number }}
+  string: py{{py}}
+  script: {{ PYTHON }} -m pip install . --no-deps --ignore-installed -vvv
+
+requirements:
+  host:
+    - python
+    - versioningit
+    - setuptools
+    - wheel
+
+  build:
+    - setuptools
+    - versioningit
+
+  run:
+    - python
+    - urllib3
+    - plotly
+
+test:
+  commands:
+    - finddata --help
+
+about:
+  home: {{ url }}
+  license: {{ license }}
+  license_family: MIT
+  license_file: ../LICENSE
+  summary: {{ description }}

environment.yml

@@ -0,0 +1,20 @@
+name: finddata
+channels:
+  - conda-forge
+dependencies:
+  - anaconda-client
+  - argcomplete
+  - boa
+  - conda-build < 4
+  - conda-verify
+  - check-wheel-contents
+  - libmamba
+  - libarchive
+  - pre-commit
+  # match what is used on rhel9
+  - python=3.11
+  - python-build
+  - toml
+  - versioningit
+  - urllib3
+  - plotly

finddata.spec

@@ -1,19 +1,24 @@
 %global srcname finddata
 %global summary Find data files using ONCat
 %define release 1
+# default python3=python3.9 on rhel9 which is too old
+%define python3_pkgversion 3.11
 
 Summary: %{summary}
 Name: python-%{srcname}
-Version: 0.9.1
+Version: 0.10.0
 Release: %{release}%{?dist}
-Source0: https://github.com/peterfpeterson/finddata/archive/v%{version}.tar.gz
+Source: %{srcname}-%{version}.tar.gz
 License: MIT
 Group: Development/Libraries
 BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-buildroot
 Prefix: %{_prefix}
 BuildArch: noarch
 Vendor: Pete Peterson <[email protected]>
-Url: https://github.com/peterfpeterson/finddata
+Url: https://github.com/neutrons/finddata
+Obsoletes: python3-finddata < 0.10
+
+Patch1: finddata-pyproject.patch
 
 %description
 Finddata uses ONCat to locate the full path of files on the NScD clusters.
@@ -24,36 +29,40 @@ Requires: python%{python3_pkgversion}
 Requires: python%{python3_pkgversion}-urllib3
 Requires: bash
 Requires: bash-completion
-Requires: python%{python3_pkgversion}-argcomplete
+#Requires: python{python3_pkgversion}-argcomplete
 %{?python_provide:%python_provide python%{python3_pkgversion}-%{srcname}}
 
-BuildRequires: python3-devel
-BuildRequires: python3-setuptools
+BuildRequires: python%{python3_pkgversion}-devel
+BuildRequires: python%{python3_pkgversion}-wheel
+BuildRequires: python%{python3_pkgversion}-pip
 
 %description -n python%{python3_pkgversion}-%{srcname}
 Finddata uses ONCat to locate the full path of files on the NScD clusters.
 
+# unpack tarball and apply patchfile - add -v to see individual commands
 %prep
-%setup -n %{srcname}-%{version} -n %{srcname}-%{version}
+%autosetup -p1
 
 %build
-%py3_build
+%pyproject_wheel
 
 %install
-%py3_install
+%pyproject_install
+# automatically scan files to install
+%pyproject_save_files finddata
+# install bash completions
+%{__mkdir} -p %{buildroot}%{_sysconfdir}/bash_completion.d/
+%{__install} -m 644 finddata.bashcomplete %{buildroot}%{_sysconfdir}/bash_completion.d/
 
-# testing is somehow broken, but the package did work
-#%check
-#%{__python3} setup.py test
-# ipywidgets is missing
-# skimage is missing
+# there are no tests - check that top level module can be imported
+%check
+%pyproject_check_import -t
 
 %clean
 rm -rf $RPM_BUILD_ROOT
 
-%files -n python%{python3_pkgversion}-%{srcname}
-%doc README
-%license LICENSE.txt
-%{python3_sitelib}/*
-/usr/bin/finddata
-/etc/bash_completion.d/finddata.bashcomplete
+%files -n python%{python3_pkgversion}-%{srcname} -f %{pyproject_files}
+%doc README.md
+%license LICENSE
+%{_bindir}/finddata
+%{_sysconfdir}/bash_completion.d/finddata.bashcomplete