test(sbom): make self-test discoverable + independent SPDX oracle - #16
Open
MarkAtwood wants to merge 2 commits into
Open
test(sbom): make self-test discoverable + independent SPDX oracle#16MarkAtwood wants to merge 2 commits into
MarkAtwood wants to merge 2 commits into
Conversation
Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
The self-test's checks lived only in helper functions driven by a check() accumulator, with no test_-prefixed callables or TestCase. `pytest tests/` (or `python -m unittest tests/test_sbom.py`) therefore collected the module and ran NOTHING while exiting 0 -- a silent loss of the only reproducibility / path-scrub / end-to-end coverage if CI ever switched runners. Add a TestSelfTest(unittest.TestCase) exposing test_unit and test_integration (integration skips when gen-sbom is absent), and switch the CI step to the discovery path so it is exercised. The `python tests/test_sbom.py` entry point is unchanged. Also add an independent SPDX oracle: validate_sbom.py is a shallow same-repo check, so the integration path now also runs the external pyspdxtools validator when installed (skipped cleanly otherwise).
There was a problem hiding this comment.
Pull request overview
This PR makes the SBOM self-test reliably executable under standard test runners by adding discoverable unittest entry points, and strengthens SPDX validation by optionally using an external validator as an independent oracle when available.
Changes:
- Add a
unittest.TestCasewrapper (TestSelfTest) sopytestandpython -m unittestexecute the existing self-test logic instead of collecting zero tests. - Extend integration validation to optionally run
pyspdxtoolsagainst the generated SPDX JSON when the tool is installed. - Update the GitHub Actions self-test workflow to run the SBOM self-test via
python -m unittest.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/test_sbom.py |
Adds discoverable unittest entry points and an optional external SPDX validator run during integration tests. |
.github/workflows/selftest.yml |
Switches the CI self-test step to execute via python -m unittest so tests are actually collected and run. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+152
to
+154
| rc = subprocess.call(["pyspdxtools", "--infile", spdx], | ||
| stdout=subprocess.DEVNULL, | ||
| stderr=subprocess.DEVNULL) |
sameehj
force-pushed
the
master
branch
4 times, most recently
from
July 24, 2026 14:09
3ab77f9 to
9bdf5b7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The self-test's checks lived only in helper functions driven by a
check()accumulator — notest_-prefixed callables orTestCase. Sopytest tests/(orpython -m unittest tests/test_sbom.py) collected the module and ran nothing while exiting 0 — a silent loss of the only reproducibility / path-scrub / end-to-end coverage if CI ever switched runners.TestSelfTest(unittest.TestCase)withtest_unit/test_integration(integration skips cleanly when gen-sbom is absent); thepython tests/test_sbom.pyentry point is unchanged. Switched the CI step to the discovery path (python -m unittest) so it's exercised — now runs 2 real tests instead of 0.validate_sbom.pyis a shallow same-repo check, so the integration path also runs externalpyspdxtoolswhen installed (skipped otherwise, staying dependency-free).Addresses SBOM-gpex.13. Note:
test_sbom.pyis also touched by #12 (capture_macros checks inunit_tests()) — different regions, trivially mergeable.