Support dependecies with version constraints in crank validate

[jtyr]

Description of your changes

This PR continuation of the PR #5570 where I have pointed out that the crossplane beta validate doesn't work with Configuratons that contain version constraints. This PR is trying to address that.

I have:

• Read and followed Crossplane's contribution process.
• Run make reviewable to ensure this PR is ready for review.
• Added or updated unit tests.
• Added or updated e2e tests.
• Linked a PR or a docs tracking issue to document this change.
• Added backport release-x.y labels to auto-backport this PR.

The PR was tested like this:

Clone the repo, apply the patch and build a new crank binary:

cd /tmp
gh repo clone crossplane/crossplane
cd crossplane
curl -s https://patch-diff.githubusercontent.com/raw/crossplane/crossplane/pull/5669.patch | git apply
make submodules
make go.build

Create directory structure for the test:

mkdir -p /tmp/{apis/foo.custom-api.example.org,test/{functions,validation}}
cd /tmp/test

Create files:

# Configuration for the "foo" package
cat <<END > apis/configuration.yaml
apiVersion: meta.pkg.crossplane.io/v1alpha1
kind: Configuration
metadata:
  name: foo
spec:
  dependsOn:
    - function: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform
      version: ">=v0.4.0"
    - provider: xpkg.upbound.io/upbound/provider-aws-s3
      version: ">=v1.3.1"
  crossplane:
    version: ">=v1.15.2"
END

# Foo composition
cat <<END > apis/foo.custom-api.example.org/composition.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: xfoos.custom-api.example.org
spec:
  compositeTypeRef:
    apiVersion: custom-api.example.org/v1alpha1
    kind: XFoo
  mode: Pipeline
  pipeline:
    - step: patch-and-transform
      functionRef:
        name: crossplane-contrib-function-patch-and-transform
      input:
        apiVersion: pt.fn.crossplane.io/v1beta1
        kind: Resources
        resources:
          - name: storage-bucket
            base:
              apiVersion: s3.aws.upbound.io/v1beta1
              kind: Bucket
              spec:
                forProvider:
                  region:  # Patched
            patches:
              - type: FromCompositeFieldPath
                fromFieldPath: spec.location
                toFieldPath: spec.forProvider.region
END

# XRD for the Foo composition
cat <<END > apis/foo.custom-api.example.org/definition.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xfoos.custom-api.example.org
spec:
  group: custom-api.example.org
  names:
    kind: XFoo
    plural: xfoos
  claimNames:
    kind: Foo
    plural: Foos
  versions:
    - name: v1alpha1
      schema:
        openAPIV3Schema:
          type: object
          required:
            - spec
          properties:
            spec:
              type: object
              required:
                - location
              properties:
                location:
                  type: string
END

cat <<END > test/claim.yaml
apiVersion: custom-api.example.org/v1alpha1
kind: Bar
metadata:
  name: mybar
spec:
  location: us-east-2
END

# Bar composition
cat <<END > test/composition.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: bar
spec:
  compositeTypeRef:
    apiVersion: custom-api.example.org/v1alpha1
    kind: Bar
  mode: Pipeline
  pipeline:
    - step: patch-and-transform
      functionRef:
        name: crossplane-contrib-function-patch-and-transform
      input:
        apiVersion: pt.fn.crossplane.io/v1beta1
        kind: Resources
        resources:
          - name: foo
            base:
              apiVersion: custom-api.example.org/v1alpha1
              kind: Foo
              spec:
                location:  # Patched
            patches:
              - type: FromCompositeFieldPath
                fromFieldPath: spec.location
                toFieldPath: spec.location
END

# Function used bu the Bar composition
cat <<END > test/functions/patch-and-transform.yaml
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: crossplane-contrib-function-patch-and-transform
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform:v0.4.0
END

# Configuration for the "foo" package
cat <<END > test/validation/configuration.yaml
apiVersion: pkg.crossplane.io/v1beta1
kind: Configuration
metadata:
  name: foo
spec:
  package: localhost:5000/foo:v0.1.0
END

# XRD for the Bar composition
cat <<END > test/validation/definition.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xbar.custom-api.example.org
spec:
  group: custom-api.example.org
  names:
    kind: XBar
    plural: xbars
  claimNames:
    kind: Bar
    plural: Bars
  versions:
    - name: v1alpha1
      schema:
        openAPIV3Schema:
          type: object
          required:
            - spec
          properties:
            spec:
              type: object
              required:
                - location
              properties:
                location:
                  type: string
END

Run local Docker registry:

docker run -d -p 5000:5000 --restart always --name registry registry:2

Package and push the foo package:

crossplane xpkg build --package-root apis --package-file foo.xpkg
crossplane xpkg push --package-files foo.xpkg localhost:5000/foo:v0.1.0

Validate the Bar composition using the upstream crank binary:

$ crossplane beta render test/claim.yaml test/composition.yaml test/functions --include-full-xr | crossplane beta validate test/validation -
package schemas does not exist, downloading:  xpkg.upbound.io/upbound/provider-aws-s3:>=v1.3.1
crossplane: error: cannot download and load cache: cannot cache package dependencies: cannot download package xpkg.upbound.io/upbound/provider-aws-s3:>=v1.3.1: cannot get config: parsing reference "xpkg.upbound.io/upbound/provider-aws-s3:>=v1.3.1": could not parse reference: xpkg.upbound.io/upbound/provider-aws-s3:>=v1.3.1

Validate the Bar composition using the patched crank binary:

$ crossplane beta render test/claim.yaml test/composition.yaml test/functions --include-full-xr | /tmp/crossplane/_output/bin/linux_amd64/crank beta validate test/validation -
package schemas does not exist, downloading:  xpkg.upbound.io/crossplane-contrib/function-patch-and-transform:>=v0.4.0
package schemas does not exist, downloading:  xpkg.upbound.io/upbound/provider-aws-s3:>=v1.3.1
[✓] custom-api.example.org/v1alpha1, Kind=Bar, mybar validated successfully
[✓] custom-api.example.org/v1alpha1, Kind=Foo, foo validated successfully
Total 2 resources: 0 missing schemas, 2 success cases, 0 failure cases

[jtyr]

@phisco @negz @jbw976 Please could you review this PR? It would be great if it would make it into the v1.16.0 release 🤞

[negz]

@jtyr Unfortunately this won't make it into v1.16 - feature freeze for that release was two weeks ago and it's being released tomorrow. Sorry!

[negz]

Would it make sense to split this logic out into a separate function?

[jtyr]

Done in the last commit. Please feel free to review it.

[jtyr]

@phisco @negz @jbw976 Please could you review this PR? It would be great if it would make it into the next release 🤞

[negz]

Perhaps @ezgidemirel would be a good first reviewer for this, given she implemented the validate subcommand.