Skip to content

feat: add checking in remote repo if manifest is a chart #3811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions examples/podinfo-flux/helm-oci/podinfo-helmrelease.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: podinfo-oci
namespace: flux-system
spec:
interval: 5m0s
releaseName: podinfo-oci
chartRef:
kind: OCIRepository
name: podinfo-helm
targetNamespace: podinfo-helm-oci
11 changes: 11 additions & 0 deletions examples/podinfo-flux/helm-oci/podinfo-source.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIRepository
metadata:
name: podinfo-helm
namespace: flux-system
spec:
interval: 30s
url: oci://ghcr.io/stefanprodan/charts/podinfo
ref:
tag: 6.4.0
16 changes: 15 additions & 1 deletion examples/podinfo-flux/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ components:
required: true
manifests:
- name: flux-install
namespace: flux
namespace: flux-system
files:
- https://github.com/fluxcd/flux2/releases/download/v2.4.0/install.yaml
images:
Expand Down Expand Up @@ -48,6 +48,20 @@ components:
# Note: this is a helm OCI artifact rather than a container image
- ghcr.io/stefanprodan/charts/podinfo:6.4.0

- name: podinfo-via-flux-helm-oci
description: Example deployment via flux (helm oci repo) using the famous podinfo example
required: true
manifests:
- name: podinfo
namespace: podinfo-helm-oci
files:
- helm-oci/podinfo-source.yaml
- helm-oci/podinfo-helmrelease.yaml
images:
- ghcr.io/stefanprodan/podinfo:6.4.0
# Note: this is a helm OCI artifact rather than a container image
- ghcr.io/stefanprodan/charts/podinfo:6.4.0

- name: podinfo-via-flux-oci
description: Example deployment via flux (native oci) using the famous podinfo example
required: true
Expand Down
1 change: 1 addition & 0 deletions packages/zarf-agent/chart/templates/webhook.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ webhooks:
- "v1"
- "v1beta1"
sideEffects: None
timeoutSeconds: 20
- name: agent-flux-helmrepo.zarf.dev
namespaceSelector:
matchExpressions:
Expand Down
2 changes: 1 addition & 1 deletion site/src/content/docs/ref/init-package.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ The `zarf-agent` is a [Kubernetes Mutating Webhook](https://kubernetes.io/docs/r

The `zarf-agent` is responsible for modifying [Kubernetes PodSpec](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec) objects [Image](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Container.Image) fields to point to the Zarf Registry. This allows the cluster to pull images from the Zarf Registry instead of the internet without having to modify the original image references.

The `zarf-agent` modifies the following [flux](https://fluxcd.io/flux/) resources: [GitRepository](https://fluxcd.io/docs/components/source/gitrepositories/), [OCIRepository](https://fluxcd.io/flux/components/source/ocirepositories/), & [HelmRepository](https://fluxcd.io/flux/components/source/helmrepositories/) to point to the local Git Server or Zarf Registry. HelmRepositories are only modified if the `type` key is set to `oci`.
The `zarf-agent` modifies the following [flux](https://fluxcd.io/flux/) resources: [GitRepository](https://fluxcd.io/docs/components/source/gitrepositories/), [OCIRepository](https://fluxcd.io/flux/components/source/ocirepositories/), & [HelmRepository](https://fluxcd.io/flux/components/source/helmrepositories/) to point to the local Git Server or Zarf Registry. HelmRepositories are only modified if the `type` key is set to `oci`. During the mutation of OCIRepositories, a call is made to the zarf registry to determine the media type of the OCI artifact. If the artifact is a helm chart the mutation will __NOT__ include the crc32 hash as including the hash interferes with the Flux deployment of the chart.

> Support for mutating OCIRepository and HelmRepository objects is in [`alpha`](/roadmap#alpha) and should be tested on non-production clusters before being deployed to production clusters.

Expand Down
55 changes: 54 additions & 1 deletion src/internal/agent/hooks/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,21 @@
// Package hooks contains the mutation hooks for the Zarf agent.
package hooks

import "github.com/zarf-dev/zarf/src/internal/agent/operations"
import (
"context"
"encoding/json"
"fmt"

ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/zarf-dev/zarf/src/internal/agent/operations"
"github.com/zarf-dev/zarf/src/internal/packager/images"
"github.com/zarf-dev/zarf/src/pkg/state"
"oras.land/oras-go/v2"
"oras.land/oras-go/v2/registry"
orasRemote "oras.land/oras-go/v2/registry/remote"
"oras.land/oras-go/v2/registry/remote/auth"
orasRetry "oras.land/oras-go/v2/registry/remote/retry"
)

func getLabelPatch(currLabels map[string]string) operations.PatchOperation {
if currLabels == nil {
Expand All @@ -13,3 +27,42 @@ func getLabelPatch(currLabels map[string]string) operations.PatchOperation {
currLabels["zarf-agent"] = "patched"
return operations.ReplacePatchOperation("/metadata/labels", currLabels)
}

func getManifestConfigMediaType(ctx context.Context, zarfState *state.State, imageAddress string) (string, error) {
ref, err := registry.ParseReference(imageAddress)
if err != nil {
return "", err
}
client := &auth.Client{
Client: orasRetry.DefaultClient,
Cache: auth.NewCache(),
Credential: auth.StaticCredential(ref.Registry, auth.Credential{
Username: zarfState.RegistryInfo.PullUsername,
Password: zarfState.RegistryInfo.PullPassword,
}),
}

http, err := images.ShouldUsePlainHTTP(ctx, ref.Registry, client)
if err != nil {
return "", err
}

registry := &orasRemote.Repository{
PlainHTTP: http,
Reference: ref,
Client: client,
}

_, b, err := oras.FetchBytes(ctx, registry, imageAddress, oras.DefaultFetchBytesOptions)

if err != nil {
return "", fmt.Errorf("got an error when trying to access the manifest for %s, error %w", imageAddress, err)
}

var manifest ocispec.Manifest
if err := json.Unmarshal(b, &manifest); err != nil {
return "", fmt.Errorf("unable to unmarshal the manifest json for %s", imageAddress)
}

return manifest.Config.MediaType, nil
}
213 changes: 213 additions & 0 deletions src/internal/agent/hooks/common_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: 2021-Present The Zarf Authors

package hooks

import (
"context"
crand "crypto/rand"
"encoding/binary"
"fmt"
"math/rand"
"net"
"os"
"strconv"
"strings"
"testing"
"time"

"github.com/defenseunicorns/pkg/helpers/v2"
v1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/stretchr/testify/require"
"github.com/zarf-dev/zarf/src/pkg/state"
"github.com/zarf-dev/zarf/src/pkg/transform"
"github.com/zarf-dev/zarf/src/test/testutil"
"oras.land/oras-go/v2"
"oras.land/oras-go/v2/registry/remote"
)

const (
// Kubernetes’ compiled-in default if the apiserver flag
// --service-node-port-range is not overridden.
defaultNodePortMin = 30000
defaultNodePortMax = 32767
// Hard safety cap so we never spin forever if someone mis-configures a range.
maxAttemptsFactor = 2
)

func populateLocalRegistry(ctx context.Context, t *testing.T, localURL string, artifact transform.Image, copyOpts oras.CopyOptions) {
localReg, err := remote.NewRegistry(localURL)
require.NoError(t, err)

localReg.PlainHTTP = true

remoteReg, err := remote.NewRegistry(artifact.Host)
require.NoError(t, err)

src, err := remoteReg.Repository(ctx, artifact.Path)
require.NoError(t, err)

dst, err := localReg.Repository(ctx, artifact.Path)
require.NoError(t, err)

_, err = oras.Copy(ctx, src, artifact.Tag, dst, artifact.Tag, copyOpts)
require.NoError(t, err)

hashedTag, err := transform.ImageTransformHost(localURL, fmt.Sprintf("%s/%s:%s", artifact.Host, artifact.Path, artifact.Tag))
require.NoError(t, err)

_, err = oras.Copy(ctx, src, artifact.Tag, dst, hashedTag, copyOpts)
require.NoError(t, err)
}

func setupRegistry(ctx context.Context, t *testing.T, port int, artifacts []transform.Image, copyOpts oras.CopyOptions) (string, error) {
localURL := testutil.SetupInMemoryRegistry(ctx, t, port)

for _, art := range artifacts {
populateLocalRegistry(ctx, t, localURL, art, copyOpts)
}

return localURL, nil
}

type mediaTypeTest struct {
name string
image string
expected string
artifact []transform.Image
Opts oras.CopyOptions
}

func TestConfigMediaTypes(t *testing.T) {
t.Parallel()
port, err := helpers.GetAvailablePort()
require.NoError(t, err)

linuxAmd64Opts := oras.DefaultCopyOptions
linuxAmd64Opts.WithTargetPlatform(&v1.Platform{
Architecture: "amd64",
OS: "linux",
})

tests := []mediaTypeTest{
{
// https://oci.dag.dev/?image=ghcr.io%2Fstefanprodan%2Fmanifests%2Fpodinfo%3A6.9.0
name: "flux manifest",
expected: "application/vnd.cncf.flux.config.v1+json",
image: fmt.Sprintf("localhost:%d/stefanprodan/manifests/podinfo:6.9.0-zarf-2823281104", port),
Opts: oras.DefaultCopyOptions,
artifact: []transform.Image{
{
Host: "ghcr.io",
Path: "stefanprodan/manifests/podinfo",
Tag: "6.9.0",
},
},
},
{
// https://oci.dag.dev/?image=ghcr.io%2Fstefanprodan%2Fcharts%2Fpodinfo%3A6.9.0
name: "helm chart manifest",
expected: "application/vnd.cncf.helm.config.v1+json",
image: fmt.Sprintf("localhost:%d/stefanprodan/charts/podinfo:6.9.0", port),
Opts: oras.DefaultCopyOptions,
artifact: []transform.Image{
{
Host: "ghcr.io",
Path: "stefanprodan/charts/podinfo",
Tag: "6.9.0",
},
},
},
{
//
name: "docker image manifest",
expected: "application/vnd.oci.image.config.v1+json",
image: fmt.Sprintf("localhost:%d/zarf-dev/images/hello-world:latest", port),
Opts: linuxAmd64Opts,
artifact: []transform.Image{
{
Host: "ghcr.io",
Path: "zarf-dev/images/hello-world",
Tag: "latest",
},
},
},
}

for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
ctx := testutil.TestContext(t)
url, err := setupRegistry(ctx, t, port, tt.artifact, tt.Opts)
require.NoError(t, err)

s := &state.State{RegistryInfo: state.RegistryInfo{Address: url}}
mediaType, err := getManifestConfigMediaType(ctx, s, tt.image)
require.NoError(t, err)
require.Equal(t, tt.expected, mediaType)
})
}
}

// GetAvailableNodePort returns a free TCP port that falls within the current
// NodePort range.
//
// The range is discovered in this order:
// 1. The env var SERVICE_NODE_PORT_RANGE (format "min-max") – matches the
// kube-apiserver flag name & format.
// 2. The Kubernetes default range 30000-32767.
//
// The function randomly probes ports in that range until it finds one the OS
// will allow us to bind. If every port in the range is in use it returns an
// error.
func GetAvailableNodePort() (int, error) {
minPort, maxPort, err := nodePortRange()
if err != nil {
return 0, err
}

// Seed a *local* rand.Rand so concurrent callers don’t step on each other.
seed := int64(binary.LittleEndian.Uint64(random64()))
r := rand.New(rand.NewSource(seed))

size := maxPort - minPort + 1
maxAttempts := size * maxAttemptsFactor // statistically enough even on busy hosts

for i := 0; i < maxAttempts; i++ {
port := r.Intn(size) + minPort
l, err := net.Listen("tcp", fmt.Sprintf(":%d", port))
if err != nil {
continue // busy; try another candidate
}
_ = l.Close() //nolint: errcheck
return port, nil
}
return 0, fmt.Errorf("unable to find a free NodePort in range %d-%d after %d attempts", minPort, maxPort, maxAttempts)
}

// nodePortRange resolves the active NodePort range.
func nodePortRange() (int, int, error) {
if v := os.Getenv("SERVICE_NODE_PORT_RANGE"); v != "" {
parts := strings.SplitN(strings.TrimSpace(v), "-", 2)
if len(parts) == 2 {
minPort, err1 := strconv.Atoi(parts[0])
maxPort, err2 := strconv.Atoi(parts[1])
if err1 == nil && err2 == nil && minPort > 0 && maxPort >= minPort {
return minPort, maxPort, nil
}
}
return 0, 0, fmt.Errorf("invalid SERVICE_NODE_PORT_RANGE value %q (expected \"min-max\")", v)
}
return defaultNodePortMin, defaultNodePortMax, nil
}

// random64 returns 8 cryptographically-secure random bytes. We fall back to
// time.Now if /dev/urandom becomes unavailable (extremely rare).
func random64() []byte {
var b [8]byte
if _, err := crand.Read(b[:]); err != nil {
binary.LittleEndian.PutUint64(b[:], uint64(time.Now().UnixNano()))
}
return b[:]
}
Loading
Loading