Skip to content

Commit

Permalink
Build library with Java version specified in .tool-versions
Browse files Browse the repository at this point in the history
This allows projects to specify, with an `asdf` (https://asdf-vm.com/)-formatted
`.tool-versions` file, the Java version to be used by the workflow for building
the project- `gha-scala-library-release-workflow` has always used a recent LTS
version Java for the build (eg Java 17), and this has sometimes been _too recent_
(guardian/atom-maker#94) for some projects at the Guardian.

We _want_ projects to update to Java 21 (see guardian/support-frontend#5792,
guardian/scala-steward-public-repos#67 etc), but tying
dozens of projects tightly to what `gha-scala-library-release-workflow` is using
will make updating that version pretty hard. If, at some later point, _some_ projects
want to experiment with Java 25, we shouldn't have to force all other projects to
be compatible with that first.

## How to express the version of Java required...

### Configuration proliferation is a problem...

One option would have been to simply add a new input parameter to the workflow,
specifying the required Java version, but that has a downside: it proliferates the
number of places in a project where the desired Java version is declared - and there
are many in use at the Guardian, with no well-agreed canonical source-of-truth:

* GHA `ci.yml`, in the [`java-version`](https://github.com/guardian/etag-caching/blob/7ecc04981f5a42a0f2ecb10631f28da571a49836/.github/workflows/ci.yml#L22) field of `actions/setup-java` - this has been my favourite in the past, as whatever CI runs with is usually pretty close to the truth
* In sbt, the `scalacOptions` of `-target`, `-release`, `-java-output-version` (currently `-release` preferred, tho' once [support](scala/scala#10654) is pervasive, `-java-output-version` is probably best) and the `javacOptions` of `-target` & `-source` - these all effectively set lower-bounds on the version of Java supported, rather than guaranteeing a minimum upper bound of Java version the way CI does.
* In apps running on Amigo AMI images; the Java version baked into the AMI, usually [referenced](https://github.com/guardian/mobile-apps-api/blob/3231e6bf064163c6d0e72c8dc862678c68eb0b62/mobile-fronts/conf/riff-raff.yaml#L10) by `riff-raff.yaml`
* In AWS Lambdas; the cloudformation [`Runtime`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.html#cfn-lambda-function-runtime) parameter, often set [by CDK](https://github.com/guardian/mobile-save-for-later/blob/1ac12e4c0100edb976ebae9e2a9975ad2321e26e/cdk/lib/mobile-save-for-later.ts#L44)

### ...`asdf`'s `.tool-versions` flle offers some consolidation

Benefits of using `.tool-versions`:

* Developers will automatically get the right Java version if they have `asdf` installed
* actions/setup-java#606 has added early support for reading the version of Java used in CI by `setup-java` from the `.tool-versions` flle - unfortunately, it's of limited use to us at the moment because of actions/setup-java#615, but it's a promising start _(`setup-java` also has support for `jEnv` files, but they are not commonly used at the Guardian)_
* Format of the file is simple enough that it can be easily understood and used, even if `asdf` is not in use - **including the file doesn't _force_ developers to use `asdf`** (I know some devs have been having some problems with it, and maybe there are/will be better alternatives out there), but it clearly documents the Java version to be used.

This does mean that **all library repos need to add a `.tool-versions` file** before this PR is merged. Helpful error messaging has been added with this PR to handle cases where the file is missing or invalid:

#### Only Java _major_ version is guaranteed

Note that although `asdf` requires a fully-specified Java version (eg `21.0.3.9.1`) in the `.tool-versions` file, currently the workflow will only match the *major* version of Java specified in the file (eg `21`), and will _always_ use the AWS Corretto distribution of Java. This is due to [limitations](actions/setup-java#615) in [`actions/setup-java`](https://github.com/actions/setup-java).
  • Loading branch information
rtyley committed Apr 30, 2024
1 parent 8648ce9 commit d967417
Show file tree
Hide file tree
Showing 2 changed files with 55 additions and 8 deletions.
30 changes: 26 additions & 4 deletions .github/workflows/reusable-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,12 +110,31 @@ jobs:
name: 🎊 Test & Version
needs: init
runs-on: ubuntu-latest
outputs:
library_build_major_java_version: ${{ steps.establish_java_for_library_build.outputs.library_build_major_java_version }}
steps:
- uses: actions/checkout@v4
- id: establish_java_for_library_build
name: Establish library build Java version
run: |
if [ ! -f .tool-versions ]; then
echo "::error title=Missing .tool-versions file::gha-scala-library-release-workflow requires an asdf-format .tool-versions file to establish the Java version for the build."
exit 1
fi
LIBRARY_BUILD_MAJOR_JAVA_VERSION=$( grep -Eo 'java [[:alnum:]-]+-[[:digit:]]+' .tool-versions | rev | cut -d'-' -f1 | rev )
echo "Using Java $LIBRARY_BUILD_MAJOR_JAVA_VERSION"
if [ -z "${LIBRARY_BUILD_MAJOR_JAVA_VERSION}" ]; then
echo "::error title=Missing Java version in .tool-versions file::Could not establish the library's required Java version - the '.tool-versions' file should have a line like 'java corretto-21.0.3.9.1'."
exit 1
fi
cat << EndOfFile >> $GITHUB_OUTPUT
library_build_major_java_version=$LIBRARY_BUILD_MAJOR_JAVA_VERSION
EndOfFile
- uses: actions/setup-java@v4 # don't 'cache: sbt', at least until https://github.com/actions/setup-java/pull/564 is merged
with:
distribution: corretto
java-version: 17
java-version: ${{ steps.establish_java_for_library_build.outputs.library_build_major_java_version }}
# - name: Debug MIMA assessment
# run: |
# sbt "show versionPolicyFindIssues"
Expand All @@ -140,7 +159,10 @@ jobs:
ls -lR $GITHUB_WORKSPACE
- name: Job summary
run: |
echo "# Release $(git describe --tags --abbrev=0)" >> $GITHUB_STEP_SUMMARY
cat << EndOfFile >> $GITHUB_STEP_SUMMARY
# Release $(git describe --tags --abbrev=0)
Library built with Java ${{ steps.establish_java_for_library_build.outputs.library_build_major_java_version }}.
EndOfFile
- uses: actions/cache/save@v4
with:
path: repo-with-unsigned-version-update-commits.git
Expand Down Expand Up @@ -239,7 +261,7 @@ jobs:
create-artifacts:
name: 🎊 Create artifacts
needs: [init, push-release-commit]
needs: [init, generate-version-update-commits, push-release-commit]
runs-on: ubuntu-latest
outputs:
ARTIFACT_SHA256SUMS: ${{ steps.record-hashes.outputs.ARTIFACT_SHA256SUMS }}
Expand All @@ -250,7 +272,7 @@ jobs:
- uses: actions/setup-java@v4 # don't 'cache: sbt', at least until https://github.com/actions/setup-java/pull/564 is resolved
with:
distribution: corretto
java-version: 17
java-version: ${{ needs.generate-version-update-commits.outputs.library_build_major_java_version }}
- name: Generate artifacts
run: |
cat << EndOfFile > sbt-commands.txt
Expand Down
33 changes: 29 additions & 4 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,30 @@ Your repo will require access to [release credentials](credentials/supplying-cre
[pass on those secrets](https://github.com/guardian/etag-caching/blob/9935da29e76b8b89759bcfe967cc7c1c02aa1814/.github/workflows/release.yml#L11-L13)
to the workflow.

## Java version

[Example `.tool-versions`](https://github.com/guardian/etag-caching/blob/main/.tool-versions)

Your repository *must* contain an [`asdf`](https://asdf-vm.com/)-formatted `.tool-versions` file
in the root of the repository, specifying the Java version to be used by the workflow for
building your project, eg:

```
java corretto-21.0.3.9.1
```

Note that although `asdf` requires a fully-specified Java version (eg `21.0.3.9.1` - use
`asdf list-all java` to list all possible Java versions), currently the workflow will only
match the *major* version of Java specified in the file (eg `21`), and will _always_ use the
AWS Corretto distribution of Java. This is due to
[limitations](https://github.com/actions/setup-java/issues/615) in
[`actions/setup-java`](https://github.com/actions/setup-java).

As recommended [below](#recommended-sbt-settings), you should also specify a `-release` flag in
`scalacOptions` to ensure that your library is compiled for any older versions of Java you wish
to support, even if you're taking advantage of a more recent version of Java for _building_ the
library.

## `sbt`

### Recommended `sbt` plugins
Expand Down Expand Up @@ -79,10 +103,11 @@ to the workflow.
to Maven Central.
* `scalacOptions` should include `-release:11` (available with Scala [2.13.9](https://www.scala-lang.org/news/2.13.9)
and above, also known as `-java-output-version`
[in Scala 3](https://www.scala-lang.org/blog/2022/04/12/scala-3.1.2-released.html#changes-to-other-compatibility-flags)) -
the workflow will always use one of the most recent LTS releases of Java
[supported by Scala](https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html),
but the generated class files will be compatible with whichever version of Java you target.
[in Scala 3](https://www.scala-lang.org/blog/2022/04/12/scala-3.1.2-released.html#changes-to-other-compatibility-flags)), or whatever minimum version of Java you want to support.
The workflow will _build_ your project with whatever Java version you declare in [`.tool-versions`](#java-version) -
but while this can be a relatively new version of Java, in order for your compiled code to support
_older_ versions of Java, and avoid `UnsupportedClassVersionError` errors, you'll
need to set this flag. See also [Scala/Java compatibility](https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html).
* Top-level 'release' module - if your project has a [multi-module](https://www.scala-sbt.org/1.x/docs/Multi-Project.html)
build this could be called 'root', or, if your project only has one module, it and your
artifact-producing module could be the same thing, and just use top-level settings.
Expand Down

0 comments on commit d967417

Please sign in to comment.