feat: add OCI artifact export via postprocessor#6574
feat: add OCI artifact export via postprocessor#6574sozercan wants to merge 1 commit intomoby:masterfrom
Conversation
Add support for exporting OCI artifacts through a postprocessor model. Frontends signal artifact metadata on the build result, and an artifact processor runs between the build and export phases to assemble an OCI layout via LLB. The transformed result is then handled by any built-in exporter (image, oci, local). The artifact frontend runs in a standard frontend sandbox, provenance is captured for all postprocessor steps, and layer paths are normalized and validated to prevent path traversal. Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
7f443df to
9f888e0
Compare
|
This doesn't seem to have much to do with #6393 Instead, it looks like most of the changes are in the container image exporter and the new built-in "artifact" frontend. If you want to work on the (external) postprocessors you can confirm your design in #6393 and make sure it covers the example use cases, eg. ability to write both artifact conversion(eg iso), deploy based postprocessors. We should also have a plan how this might be used to for reproducibility converters and sbom in the future to make the current solutions more flexible. This shouldn't contain any new "artifact" exporting. If you want to support non-image OCI artifacts in current builtin exporters then that should have a separate design issue, where we can confirm the protocol between the build result metadata and the exporter. To make these use cases efficient don't we also need some sort of "non-tar" storage type so we can pull these artifacts, change their composition and then export (unlike image layers, you can't run containers on top of them of course). CI is also failing, but that doesn't look like the biggest issue atm. |
Summary
Add support for exporting OCI artifacts through a postprocessor model, following the architecture discussed in #6393 (comment)
Instead of implementing artifact export as a gateway exporter at the same level as built-in exporters, this models it as a postprocessor step that runs between the build and export phases:
containerimage.artifact,containerimage.artifact.type,containerimage.artifact.config.mediatype,containerimage.artifact.layers)ArtifactProcessordetects this metadata and invokesSolvePostprocessor(), which runs theartifact.v0frontend in a standard frontend sandboxThis enables frontends to produce OCI artifacts that can be pushed directly to registries, exported as OCI tarballs, or written to local directories — without the frontend needing to implement any export logic itself.
Design
solver.gogains aProcessortype and a loop that runs registered processors between build and export. The artifact processor is one such processor.artifact.v0frontend is registered viaforwarder.NewGatewayForwarder, the same mechanism used fordockerfile.v0. No custom execution paths.Exporter/ExporterInstanceinterfaces remain unchanged. NollbBridge,executor, orFrontendResultparameters added.Postprocess []Parametersfield in both v0.2 and v1.0 formats.Validation
This has been tested end-to-end with AIKit, which uses this to package HuggingFace models as CNCF ModelPack OCI artifacts:
The AIKit integration PR: kaito-project/aikit#748
Buildkit image:
docker.io/sozercan/buildkit:oci-layout-passthrough-b6f47d1e0Resulting artifact examples: