Spack V1 Migration Feature: Reserved Spack Definitions #332

CodeGat · 2025-11-26T03:13:43Z

Closes #330
Will be merged into #326

Background

Root specs are very meaningful in our infrastructure. This comes with a few downsides, mostly in the cases where we have multiple specs in the speclist, or regular users want to use prerelease manifests in their own instances but can't due to prerelease-specific modifications of the root spec.

In this PR, we move away from important information being in the .spack.specs[0] field, and instead use reserved, dummy definitions at the top of the file:

spack:
  definitions:
  - _name: [access-om2]
  - _version: [2025.11.000]
  specs:
  - access-om2
  # ...

This means that speclists can be free from infra-specific information, and they are open to future modification.

The PR

Rename the get-spack-root-spec action to the more apt get-spack-manifest
- Also update the outputs - simplified to deployment-{name|version} outputs, equivalent to .spack.definitions[]._{name|version}[0] fields.
Update all usages of the get-spack-root-spec action to get-spack-manifest

Testing

Testing Details in this collapsed section

Unit Tests

Done via pytest, all pass.

Prerelease (`!redeploy`)

Expected failure, since the schema version updates are not yet on the main branch (since comment triggers are technically in the context of the main branch). But, the only points of difference (The status setting in https://github.com/ACCESS-NRI/ACCESS-TEST/actions/runs/20014778511/job/57390074093 and https://github.com/ACCESS-NRI/ACCESS-TEST/actions/runs/20014778511/job/57390091047) succeeded as expected.

Software Deployment Repository Testing

Done for Software Deployment Repostiories in ACCESS-NRI/system-tools#19

Prerelease (Draft)

Succeeded, see https://github.com/ACCESS-NRI/system-tools/actions/runs/20151461268?pr=19

Prerelease (Ready For Review)

Succeeded. see https://github.com/ACCESS-NRI/system-tools/actions/runs/20217336546?pr=19

Prerelease (`!redeploy`)

Expected failure, since the schema version updates are not yet on the main branch (since comment triggers are technically in the context of the main branch). But, the only points of difference (The status setting in https://github.com/ACCESS-NRI/system-tools/actions/runs/20217823516/job/58033899157 and https://github.com/ACCESS-NRI/system-tools/actions/runs/20217823516/job/58033912451) succeeded as expected.

CodeGat · 2025-12-04T02:54:34Z

This looks about ready to go!

aidanheerdegen

An initial pass, but might need a peer-review to ask some more questions.

aidanheerdegen · 2025-12-15T04:08:44Z

scripts/spack_manifest/getter.py

+        #   {'something': ['else']}
+        # ]}
+        # Into a much easier to parse:
+        # ['name': 'access-om2', 'version': '2025.02.100']


Suggested change

# ['name': 'access-om2', 'version': '2025.02.100']

# [{'name': 'access-om2'}, {'version': '2025.02.100'}]

Speaking of which, why an array of single key/value dicts, and not just a dict with multiple key/value pairs?

Whoops, they actually were dict[str, Any] anyway! Done in b60c71d

aidanheerdegen · 2025-12-15T04:13:50Z

scripts/spack_manifest/getter.py

+                        # A bit of a strange case, but in future we may want to handle other reserved
+                        # definitions as lists rather than single strings
+                        case "name" | "version" | _:
+                            reserved_definitions[reserved_name_no_underscore] = (
+                                reserved_value_list[0]
+                            )
+


If the other case isn't being handled currently why bother with this logic? Could just mention in a comment.

Fair enough, done in b60c71d

aidanheerdegen · 2025-12-15T04:16:09Z

scripts/spack_manifest/getter.py

+            if len(package_spec.get("require", {})) != 0 and package_spec["require"][
+                0
+            ].startswith("@"):
                packages_with_versions_defined.append(package_name)


Don't know if this was an automatice code-formatter, but that is horrendous line breaking.

Thanks, black. Done in b60c71d

aidanheerdegen · 2025-12-15T04:28:30Z

tests/scripts/spack_manifest/injection/inputs/prerelease.spack.yaml

+  - _name: [access-om2]
+  - _version: [2024.03.0]
  specs:


This has different indentation to the spack.yaml input. Don't know if this is relevant.

It wouldn't affect the tests, but it's good to be consistent - done in 984b2d3

aidanheerdegen · 2025-12-15T04:29:21Z

tests/scripts/spack_manifest/injection/inputs/spack.yaml

  # add package specs to the `specs` list
+  definitions:
+    - _name: ["access-om2"]
+    - _version: ["2024.03.0"]


So we support/test for reserved definitions other than than _name and _version?

We don't test for them yet, since _name and _version are the only two I can think of so far.

…only deployment-[name|version]

… the bump

… usage

…remove --keep-root-spec-intact flag

CodeGat · 2025-12-19T00:24:42Z

After the rebase onto v7, I note that it still works fine 👍 See ACCESS-NRI/ACCESS-TEST#60 (comment)

Update reserved defs getter return value, remove case statement, revert autoformatter

CodeGat self-assigned this Nov 26, 2025

CodeGat added the type:feature New feature label Nov 26, 2025

CodeGat added this to Spack `0.22` to `1.0` Migration Nov 26, 2025

CodeGat added priority:high version:MAJOR Requires an update to Model Deployment Repositories CI for:v8 Applies to v8 labels Nov 26, 2025

github-project-automation bot moved this to New Issues 🌅 in Spack `0.22` to `1.0` Migration Nov 26, 2025

CodeGat mentioned this pull request Nov 26, 2025

TEST: spack-v1-migration-reserved-defs branch ACCESS-NRI/ACCESS-TEST#60

Open

CodeGat force-pushed the spack-v1-migration-reserved-defs branch from d9694d0 to 857bbd2 Compare November 27, 2025 02:32

CodeGat moved this from New Issues 🌅 to Review 👏 in Spack `0.22` to `1.0` Migration Dec 1, 2025

CodeGat marked this pull request as ready for review December 1, 2025 02:35

This was referenced Dec 4, 2025

Spack v1 Migration: Reserved Definitions Schema Update ACCESS-NRI/schema#60

Merged

Less restrictive target for gh ACCESS-NRI/system-tools#17

Merged

aidanheerdegen self-requested a review December 15, 2025 03:56

aidanheerdegen reviewed Dec 15, 2025

View reviewed changes

CodeGat force-pushed the spack-v1-migration branch from 3da2716 to a685e21 Compare December 18, 2025 06:04

CodeGat requested a review from harshula as a code owner December 18, 2025 06:04

CodeGat added 11 commits December 19, 2025 09:41

Rename action from get-spack-root-spec to get-spack-manifest, return …

97a595f

…only deployment-[name|version]

cd.yml: Use get-spack-manifest, rework job names

db3dcc2

ci-comment.yml: Use get-spack-manifest, simplify update and commit of…

25b789c

… the bump

deploy-1-setup.yml: Use get-spack-manifest, remove check on root spec…

223d56d

… usage

deploy-2-start.yml: Remove unused manifest information

05539f6

fix: Add manifest path to action

82d2072

Update modules.py, prerelease.py to use reserved defs _name and _version

3b97faf

Update tests

33d1f02

Make reserved definitions flow-style for module injection too

2afe94d

Remove references to RootSpecs, use ReservedDefinitions.get("name"), …

1365242

…remove --keep-root-spec-intact flag

Update pkg_repo_url logic to use spack.repo.PATH.get_pkg_class()

db7e3db

CodeGat force-pushed the spack-v1-migration-reserved-defs branch from e871355 to db7e3db Compare December 18, 2025 22:41

CodeGat added 2 commits December 19, 2025 11:47

getter.py fixes from code review

b60c71d

Update reserved defs getter return value, remove case statement, revert autoformatter

Update test input manifest formatting

984b2d3

CodeGat requested a review from aidanheerdegen December 19, 2025 00:52

	# ['name': 'access-om2', 'version': '2025.02.100']
	# [{'name': 'access-om2'}, {'version': '2025.02.100'}]

Spack V1 Migration Feature: Reserved Spack Definitions #332

Are you sure you want to change the base?

Spack V1 Migration Feature: Reserved Spack Definitions #332

Uh oh!

Conversation

CodeGat commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

The PR

Testing

Unit Tests

MDR Testing

Prerelease (Draft)

Prerelease (Ready For Review)

Prerelease (!redeploy)

Software Deployment Repository Testing

Prerelease (Draft)

Prerelease (Ready For Review)

Prerelease (!redeploy)

Uh oh!

CodeGat commented Dec 4, 2025

Uh oh!

aidanheerdegen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CodeGat commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CodeGat commented Nov 26, 2025 •

edited

Loading

Prerelease (`!redeploy`)

Prerelease (`!redeploy`)