test(advisory): port gen-advisory unit suite + canonical advisories tree - #7
Open
MarkAtwood wants to merge 3 commits into
Open
test(advisory): port gen-advisory unit suite + canonical advisories tree#7MarkAtwood wants to merge 3 commits into
MarkAtwood wants to merge 3 commits into
Conversation
Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
The consolidation dropped the live advisory data that gen-advisory reads by default (advisories/records/*.json + advisories/vex-overlay.json), so a bare `gen-advisory` / `make advisory` had no input. Vendor it from the wolfSSL source of truth. Also the fixture the ported test's canonical-tree case needs.
Port the stdlib gen-advisory test gate and its frozen testdata/ CVE fixtures into central/ (beside gen-advisory), and wire it into selftest.yml. Restores CI coverage for the advisory generator dropped during the wolfGlass consolidation.
This was referenced Jul 23, 2026
There was a problem hiding this comment.
Pull request overview
Restores and formalizes test coverage for the central/gen-advisory generator by porting the previously-stdlib unit/semantic test suite and adding committed, hermetic CVE record fixtures plus a canonical advisories/ input tree that the generator can consume by default.
Changes:
- Add
central/test_gen_advisory.py(stdlib-only) covering record parsing, CSAF/CycloneDX emission invariants, overlay vocabulary, and CLI behavior/reproducibility. - Add committed CVE JSON fixtures under
central/testdata/and vendor canonical advisory inputs underadvisories/(records +vex-overlay.json). - Wire the advisory unit suite into
.github/workflows/selftest.yml(syntax check + unittest execution).
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| central/testdata/README.md | Documents the hermetic CVE fixture set used by the advisory generator tests. |
| central/testdata/CVE-2026-5999.json | Adds a synthetic CVE record fixture to exercise CVSS v3.1 → CSAF scores[] emission. |
| central/testdata/CVE-2026-5778.json | Adds a real CVE record fixture (CVSS v4) for generator tests. |
| central/testdata/CVE-2026-5501.json | Adds a real CVE record fixture (CVSS v4) for generator tests. |
| central/test_gen_advisory.py | Adds the ported unit + semantic test suite for central/gen-advisory. |
| advisories/vex-overlay.json | Adds the canonical VEX overlay consumed by default advisory generation flows. |
| advisories/records/CVE-2026-5778.json | Vendors the canonical CVE record into the default advisories tree. |
| advisories/records/CVE-2026-5501.json | Vendors the canonical CVE record into the default advisories tree. |
| .github/workflows/selftest.yml | Runs advisory unit tests in CI and includes the file in syntax-check compilation. |
Comments suppressed due to low confidence (1)
central/test_gen_advisory.py:10
- This docstring still references
scripts/test_gen_sbom.pyand a non-existent.github/workflows/advisory.yml. Since this repo’s unit gate is wired throughtests/test_gen_sbom.pyand.github/workflows/selftest.yml, the wording should be updated to avoid pointing readers at files that aren’t present.
These tests are pure stdlib (no network, no pip deps) so they form the cheap
PR gate, mirroring scripts/test_gen_sbom.py. They cover three things the
JSON-schema validators in .github/workflows/advisory.yml do NOT:
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+2
to
+6
| """Unit + semantic tests for scripts/gen-advisory. | ||
|
|
||
| Run from the repo root: | ||
|
|
||
| python3 -m unittest scripts/test_gen_advisory.py |
Comment on lines
+3
to
+5
| CVE Program records (CVE JSON 5.x) used by `scripts/test_gen_advisory.py` and | ||
| the `.github/workflows/advisory.yml` jobs. Committed so the tests are hermetic | ||
| (no network fetch from cve.org at test time). |
| @@ -0,0 +1,21 @@ | |||
| { | |||
| "_comment": "Canonical wolfSSL VEX overlay consumed by `make advisory` / `scripts/gen-advisory`. Keyed by CVE id; carries the determinations the CVE Program record cannot express (analysis state, justification, fixed versions, remediation, optional FIPS product, optional build-reachability hedge). Constrained by scripts/advisory-vex-overlay.schema.json. To model a wolfCrypt FIPS module as a separate product, add a \"fips\" block per the format in scripts/advisory-vex-overlay.example.json using the real validated module version and CMVP certificate number (do NOT copy the illustrative placeholder values from the example).", | |||
sameehj
force-pushed
the
master
branch
4 times, most recently
from
July 24, 2026 14:09
3ab77f9 to
9bdf5b7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Restores advisory-generator test coverage dropped in the wolfGlass consolidation: ports the 720-line stdlib
test_gen_advisory.pygate + itstestdata/fixtures intocentral/, wires it intoselftest.yml, and vendors the canonicaladvisories/tree (records +vex-overlay.json) thatgen-advisory/make advisoryreads by default. 62 tests pass.Base of the advisory fix stack — the three advisory fix PRs (#1 path-traversal, CDX per-product, overlay validation) build on this because they add tests to the file introduced here. Merge this first.