RFC for Maven and Artifactory support

[emmjohnson]

Readable

[netlify]

✅ Deploy Preview for elated-stonebraker-105904 canceled.

Name	Link
🔨 Latest commit	5774deb
🔍 Latest deploy log	https://app.netlify.com/sites/elated-stonebraker-105904/deploys/6245cf4faead4300087145a3

[scothis]

What does the resource we expect the supply chain to stamp out for maven source look like? It may be worth hitting pause on this until we have a better sense of what's resources are being choreographed.

[scothis]

I'm not familiar with maven URLs like this, granted it's been several years since I've been deep in this ecosystem. This example is just a jar on an http server. That jar/http server may happen to be part of a maven repository, but that'd be coincidence.

Maven artifacts are identified by a groupId, artifactId, version (which may be a snapshot and are a bit more complicated to resolve) and a type (often jar). These coordinates are resolved against a set of repositories. Not every repository will hold every artifact, there's no way to know without asking. Each repository has a base URL that the artifacts are then resolved from. The groupId and artifactId translate into path segments under the base URL. For each artifact there's a maven-metadata.xml index that lists the specific versions and the artifacts. The version key may be listed directly in this index, or if a snapshot the newest match is used. Only after using the repository URL, the groupId, the artifactId, the index file for the artifact and the version, can you then resolve the fully qualified URL for the artifact.

Client specific configuration about how to talk to each repository (like auth) are managed in a separate config file, usually located at ~/.m2/settings.xml.

[emmjohnson]

Thanks @scothis. Definitely outside of my element here. We did modify the example to drop the .jar. So you are suggesting we need something like:

source:
  maven:
    groupid:
    artifactid:
    url:
    index:
    version:
    config:

[emmjohnson]

We searched around on an artifactory and I'm not sure I saw all of the fields that you listed. Would love it if you had an example you could point me to.

[scothis]

We searched around on an artifactory and I'm not sure I saw all of the fields that you listed. Would love it if you had an example you could point me to.

https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#dependency-management

[scothis]

Thanks @scothis. Definitely outside of my element here. We did modify the example to drop the .jar. So you are suggesting we need something like:

source:
  maven:
    groupid:
    artifactid:
    url:
    index:
    version:
    config:

My suggestion is to not change the Workload resource until we have a sense of what the resource being stamped out looks like. I would expect something like this, but that's possibly insufficient based on the how the stamped resource defines the model. Most prominently this model assumes the set of repositories to resolve against are defined somewhere else.

spec:
  source:
    maven:
      groupId:
      artifactId:
      version:
      type:

[rszumlakowski]

We are aiming for the name semver since that's what the FluxCD GitRepository resource uses. We are using the GitRepository resource to model this behaviour.

If the semver field is left out then the controller would always return the highest version of the artifact.

I agree that the packaging should be optional and the default should be jar.

[xyloman]

We are aiming for the name semver since that's what the FluxCD GitRepository resource uses. We are using the GitRepository resource to model this behaviour.

I thought a lot about this and consistency is important. However, the audience of the MavenResource is likely to get confused by the field semver. Other fields that exist on the resource map to their counter parts in the pom.xml (e.g. groupId, artifactId, packaging etc.) However, semver does not match. I propose for usability sake the semver field be named version to be consistent with the maven pom instead of consistent with the GitRepository resource. Furthermore, semver for the GitRepository resource makes sense since git uses that terminology.

[scothis]

The semver field is an option on a Flux GitRepository and not the only way to specify a commit. You can also specify a commit directly by its commit ref, tag or the HEAD of a branch. Curiously, Cartographer does not expose semver as an option for git source and nobody has asked for it yet.

The version field in maven has its own range syntax.

[xyloman]

@scothis that is a great point on the range syntax that maven has is slightly different than the syntax that was shared above. I would assume that keeping the syntax similar to what Maven users are familiar with.

[rszumlakowski]

At Cartographer office hours today we decided to go use the Maven field name version and the Maven range syntax rather than the semver field as previously discussed.

[rszumlakowski]

I favour this second approach. We have been called on to support not just general Maven artifacts but specifically Artifactory artifacts, too. I wouldn't be surprised if there are customer use cases for Nexus artifacts or even things like NPM, too. This second approach seems more scaleable.

[scothis]

Artifactory is a server side implementation of many different types of artifact repositories. Clients should have no knowledge that an artifact is hosted by an Artifactory repository. A client for one type of artifact repository (maven) has no ability to pull artifacts in a different type of repository (npm, oci, etc) unless it also happens to understand that repository api (unlikely).

[rszumlakowski]

The reason that we are calling out Artifactory repositories specifically is that customers are asking for Artifactory support explicitly.

[sclevine]

customers are asking for Artifactory support explicitly

Let's focus on use cases for Cartographer as an open source project in these spaces 🙂

[rszumlakowski]

I have removed references to "Artifactory" from my design document. Our team has decided to focus on Maven functionality first without any Artifactory-specific functionality. That should suit the open source project, right?

[sclevine]

Was more calling out the unqualified reference to "customers" -- Artifactory is fine :)

[heyjcollins]

@rashedkvm @atmandhol

[rashedkvm]

@emmjohnson should the reference to Artifactory be more specific and not confused as a source type here? Similar to the comment in #767 (comment)