Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up CI & miscellaneous updates for 3.10 #911

Merged
merged 8 commits into from
Jan 22, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 37 additions & 68 deletions .github/workflows/pytest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,8 @@ concurrency:
cancel-in-progress: true

env:
# Version used until 2024-11-20; disabled
# GAMS_VERSION: 29.1.0
# First version including a macOS arm64 distribution
GAMS_VERSION: 43.4.1

# See description in lint.yml
depth: 100

GAMS_VERSION: 43.4.1 # First version including a macOS arm64 distribution
depth: 100 # Must be large enough to include the most recent release
label: "safe to test"

jobs:
Expand All @@ -30,8 +24,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- if: >
github.repository != github.event.pull_request.head.repo.full_name &&
! contains(github.event.pull_request.labels.*.name, env.label)
!(
github.event_name == "schedule"
|| github.repository == github.event.pull_request.head.repo.full_name
|| contains(github.event.pull_request.labels.*.name, env.label)
)
run: |
echo "Pytest workflow will not run for branch in fork without label \`${{ env.label }}\`." >>$GITHUB_STEP_SUMMARY
exit 1
Expand Down Expand Up @@ -86,17 +83,17 @@ jobs:
- name: Fetch tags (for setuptools-scm)
run: git fetch --tags --depth=${{ env.depth }}

- uses: actions/setup-python@v5
- name: Set up uv, Python
uses: astral-sh/setup-uv@v5
with:
cache-dependency-glob: "**/pyproject.toml"
python-version: ${{ matrix.python-version }}
cache: pip
cache-dependency-path: "**/pyproject.toml"

- uses: ts-graphviz/setup-graphviz@v2
# TEMPORARY Work around ts-graphviz/setup-graphviz#630
if: ${{ ! startswith(matrix.os, 'macos-') }}
# Work around ts-graphviz/setup-graphviz#630
if: matrix.os != 'macos-13'

- name: Cache GAMS installer and R packages
- name: Cache GAMS installer
uses: actions/cache@v4
with:
path: |
Expand All @@ -110,17 +107,13 @@ jobs:
version: ${{ env.GAMS_VERSION }}
license: ${{ secrets.GAMS_LICENSE }}

- name: Install Python package and dependencies
- name: Install the package and dependencies
# By default, the below installs ixmp from the main branch. To run against
# other code, e.g. other branches for open PRs), temporarily edit as
# appropriate. DO NOT merge such changes to `main`.
run: |
pip install --upgrade "ixmp @ git+https://github.com/iiasa/ixmp.git@main"
pip install .[tests]

# TEMPORARY With Python 3.13 pyam-iamc resolves to 1.3.1, which in turn
# limits pint < 0.17. Override.
pip install --upgrade pint
uv pip install --upgrade "ixmp @ git+https://github.com/iiasa/ixmp.git@main"
uv pip install .[tests]

- name: Run test suite using pytest
env:
Expand All @@ -129,7 +122,7 @@ jobs:
run: |
pytest message_ix \
-m "not nightly and not tutorial" \
-rA --verbose --color=yes --durations=20 \
--color=yes --durations=20 -rA --verbose \
--cov-report=xml \
--numprocesses=auto --dist=loadgroup
shell: bash
Expand All @@ -140,9 +133,7 @@ jobs:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

tutorials:
if: >
github.event_name != 'workflow_run' ||
github.event.workflow_run.conclusion == 'success'
needs: [ check ]

strategy:
matrix:
Expand All @@ -158,33 +149,22 @@ jobs:

steps:
- name: Check out message_ix
if: github.event_name != 'workflow_run'
uses: actions/checkout@v4
with:
fetch-depth: ${{ env.depth }}
ref: ${{ github.event.pull_request.head.sha }}

- name: Check out message_ix (workflow_run)
if: github.event_name == 'workflow_run'
uses: actions/checkout@v4
with:
repository: ${{ github.event.workflow_run.repository.fullname }}
ref: ${{ github.event.workflow_run.head_branch }}

- uses: actions/setup-python@v5
- name: Set up uv, Python
uses: astral-sh/setup-uv@v5
with:
cache-dependency-glob: "**/pyproject.toml"
python-version: "3.13"
cache: pip
cache-dependency-path: "**/pyproject.toml"

- name: Set RETICULATE_PYTHON
# Use the environment variable set by the setup-python action, above.
run: echo "RETICULATE_PYTHON=$pythonLocation" >> "$GITHUB_ENV"
# Retrieve the Python executable set up above
run: echo "RETICULATE_PYTHON=$(uv python find)" >> "$GITHUB_ENV"
shell: bash

- uses: ts-graphviz/setup-graphviz@v2
# TEMPORARY Work around ts-graphviz/setup-graphviz#630
if: ${{ ! startswith(matrix.os, 'macos-') }}

- uses: r-lib/actions/setup-r@v2
id: setup-r
with:
Expand All @@ -206,34 +186,25 @@ jobs:
version: ${{ env.GAMS_VERSION }}
license: ${{ secrets.GAMS_LICENSE }}

- name: Install Python package and dependencies
- name: Install the package and dependencies
# By default, the below installs ixmp from the main branch. To run against
# other code, e.g. other branches for open PRs), temporarily edit as
# appropriate. DO NOT merge such changes to `main`.
run: |
pip install --upgrade "ixmp @ git+https://github.com/iiasa/ixmp.git@main"
pip install .[tests]

# TEMPORARY With Python 3.13 pyam-iamc resolves to 1.3.1, which in turn
# limits pint < 0.17. Override.
pip install --upgrade pint
uv pip install --upgrade "ixmp @ git+https://github.com/iiasa/ixmp.git@main"
uv pip install .[tests]

- name: Install R dependencies and tutorial requirements
run: |
install.packages("remotes")
remotes::install_cran(
c("dplyr", "IRkernel", "reticulate"),
dependencies = TRUE,
# force = TRUE,
)
install.packages(c("dplyr", "IRkernel", "reticulate"))
IRkernel::installspec()
shell: Rscript {0}

- name: Run test suite using pytest
run: |
pytest message_ix \
-m "tutorial" \
-rA --verbose --color=yes --durations=20 \
--color=yes --durations=20 -rA --verbose \
--cov-report=xml \
--numprocesses=auto --dist=loadgroup
shell: bash
Expand All @@ -250,13 +221,11 @@ jobs:

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: # Same as the "Latest version supported by message_ix", above
python-version: "3.13"

- name: Force recreation of pre-commit virtual environment for mypy
if: github.event_name == 'schedule' # Comment this line to run on a PR
run: gh cache list -L 999 | cut -f2 | grep pre-commit | xargs -I{} gh cache delete "{}" || true
env: { GH_TOKEN: "${{ github.token }}" }

- uses: pre-commit/[email protected]
- uses: astral-sh/setup-uv@v5
with: { cache-dependency-glob: "**/pyproject.toml" }
- uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: pre-commit|${{ env.UV_PYTHON }}|${{ hashFiles('.pre-commit-config.yaml') }}
lookup-only: ${{ github.event_name == 'schedule' }} # Set 'true' to recreate cache
- run: uvx pre-commit run --all-files --color=always --show-diff-on-failure
13 changes: 3 additions & 10 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,10 @@
repos:
- repo: local
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.14.1
hooks:
- id: mypy
name: mypy
always_run: true
require_serial: true
pass_filenames: false

language: python
entry: bash -c ". ${PRE_COMMIT_MYPY_VENV:-/dev/null}/bin/activate 2>/dev/null; mypy $0 $@"
additional_dependencies:
- mypy >= 1.12.0
- asyncssh
- git+https://github.com/iiasa/ixmp.git@main
- importlib_resources
Expand All @@ -20,9 +14,8 @@ repos:
- Sphinx
- types-PyYAML
- types-requests
args: ["."]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.4
rev: v0.9.1
hooks:
- id: ruff
- id: ruff-format
Expand Down
24 changes: 16 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
[![Documentation build](https://readthedocs.com/projects/iiasa-energy-program-message-ix/badge/?version=stable)](https://docs.messageix.org/en/stable/)
[![Build status](https://github.com/iiasa/message_ix/actions/workflows/pytest.yaml/badge.svg)](https://github.com/iiasa/message_ix/actions/workflows/pytest.yaml)
[![Test coverage](https://codecov.io/gh/iiasa/message_ix/branch/main/graph/badge.svg)](https://codecov.io/gh/iiasa/message_ix)
[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CODE_OF_CONDUCT.md)
[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CODE_OF_CONDUCT.md)


MESSAGEix is a versatile, dynamic, model framework for energy-engineering-economy-environment (E4) systems research.
Expand All @@ -20,11 +20,11 @@ The `message_ix` Python package—also fully usable from R—includes:
- Application programming interfaces (APIs) and tools for model building and scientific programming,
- Extensive documentation and a complete test suite.

The framework is built on IIASA's [*ix* modeling platform (`ixmp`)](https://github.com/iiasa/ixmp), which provides data warehouse features for high-powered numerical scenario analysis.
The framework is built on IIASA's [*ix* modeling platform (`ixmp`)](https://github.com/iiasa/ixmp), which provides data warehouse features for numerical scenario analysis.

## Documentation

Complete documentation of the framework is avaiable at **https://docs.messageix.org/**.
Complete documentation of the framework is available at **https://docs.messageix.org**.
This includes:

- **Installation** and recommended pre-requisite learning.
Expand All @@ -39,23 +39,31 @@ This includes:
Other forms of documentation:

- The online documentation is built automatically from the contents of the
[`message_ix` GitHub repository](https://github.com/iiasa/messag_ix).
[`message_ix` GitHub repository](https://github.com/iiasa/message_ix).
- Documentation for the ‘latest’ or ‘stable’ release is shown by default.
- Use the chooser to access the [‘latest’ version](https://docs.messageix.org/en/latest/), corresponding to the ``main`` branch and including the latest development code; or to access docs for a specific release, e.g. `v3.2.0`.
- For offline use, the documentation can be built from the source code.
See the file [`doc/README.rst`](doc/README.rst)

## Commitment to our Community

- We aim to create a community that is welcoming to everyone. Please respect our [Code of Conduct](CODE_OF_CONDUCT.md) at all times when interacting with `message_ix` and the community.
- We aim to create a community that is welcoming to everyone.
Please respect our [Code of Conduct](CODE_OF_CONDUCT.md) at all times when interacting with `message_ix` and the community.
- We warmly welcome all contributions to our community. Please take a look at our [guidelines for contributing to development](https://docs.messageix.org/en/latest/contributing.html).
- If you are looking to understand how the `message_ix` team operates and how we decide what to do with your contributions, please read our [governance guidelines](GOVERNANCE.md).
- We highly value code security. To learn more about our security practice and advice, please visit our [security policy](.github/SECURITY.md).
- We strive to entertain a welcoming and inclusive community. If you struggle to use `message_ix`, please check our [guidelines for getting support](SUPPORT.md).
- We highly value code security.
To learn more about our security practice and advice, please visit our [security policy](.github/SECURITY.md).
- We strive to entertain a welcoming and inclusive community.
If you struggle to use `message_ix`, please check our [guidelines for getting support](SUPPORT.md).
- We also:
- Host an annual [MESSAGEix Community Meeting](https://iiasa.ac.at/search?search_api_fulltext=MESSAGEix+Community+Meeting).
- Publish, at least once per year, a community newsletter.
You are cordially invited to subscribe by filling [this form](https://iiasa.ac.at/signup) and ticking the box for “MESSAGEix Community”.


## License

Copyright © 2018–2024 IIASA Energy, Climate, and Environment (ECE) Program
Copyright © 2018–2025 IIASA Energy, Climate, and Environment (ECE) Program

The MESSAGEix framework is licensed under the Apache License, Version 2.0 (the "License"); you may not use the files in this repository except in compliance with the License. You may obtain a copy of the License in [`LICENSE`](LICENSE) or at <http://www.apache.org/licenses/LICENSE-2.0>.

Expand Down
2 changes: 1 addition & 1 deletion RELEASE_NOTES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Next release
GitHub community guidelines
---------------------------

Add community guidelines for interaction on GitHub (:pull:`871`).
Add community guidelines for interaction on GitHub (:pull:`871`, :pull:`911`).
Please familiarize yourself with these to foster an open and welcoming community!

All changes
Expand Down
2 changes: 1 addition & 1 deletion doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# -- Project information ---------------------------------------------------------------

project = "MESSAGEix"
copyright = "2018–2024, IIASA Energy, Climate, and Environment (ECE) Program"
copyright = "2018–%Y, IIASA Energy, Climate, and Environment (ECE) Program"
author = "MESSAGEix Developers"

# The major project version, used as the replacement for |version|.
Expand Down
Loading
Loading