[MNG-5971] Imported dependencies should be available to inheritance processing

[jira-importer]

Stephane Nicoll opened MNG-5971 and commented

When a project extends from a parent with a dependencyManagement section, it is not always possible to properly override (and align) the version to use for a group of dependencies.

We typically use Bill Of Materials to gather a group of modules and make sure their versions are consistent.

The following project demonstrates the issue: https://github.com/snicoll-scratches/maven-dependency-management

The first commit is a working use case where the parent uses a bom with version A and we use the same bom with version B in the child. Version B is used as expected.

The second commit demonstrates the faulty scenario. Rather than using a bom in the parent, we use a direct dependency (provided by that bom). We still use the bom with a different version. In that case all the dependencies but the one provided by the parent are overridden (leading to mixed versions for the dependencies provided by the BOM).

It looks like the distance is still used to compute the version while the graph of dependencies should be flatten at each step for a proper override.

Thoughts? Thanks!

---

Affects: 3.3.3

Attachments:

• bom-cloud.zip (50.23 kB)

Issue Links:

• MNG-5547 import scope of dependencyManagement doesn't seem to work
  ("is duplicated by")

• MNG-5947 dependencyManagement import section does not resolve dependencies using "nearest" definition
  ("is duplicated by")

• MNG-6246 Inconsistent override behaivor with BOM vs dependency
  ("is duplicated by")

• MNG-6161 Dependencies' management via <scope>import</scope> should take precedence over inherited definitions
  ("is duplicated by")

• MNG-5947 dependencyManagement import section does not resolve dependencies using "nearest" definition

• MNG-5982 The POM for ... is invalid, transitive dependencies ... while property was overriden

• MNG-7852 Use all the versions for dependency resolution rather than "nearest first" or "declared first"

• MNG-6079 3.4 regression: cannot override version of a dependencyManagement in a submodule any more
  ("supercedes")

Remote Links:

• Page
  

6 votes, 20 watchers

[jira-importer]

Christian Schulte commented

Imports never overwrite anything already present in the model. See DefaultDependencyManagementImporter for how the import is implemented. You want the BOM to always overwrite what is already there. Others want to overwrite what is in the BOM. Regarding the example, just do not put things you want to import into the pom as that will overwrite what gets imported.

[jira-importer]

Christian Schulte commented

Is your use case really "override what is already there" or did you file this issue because you noticed something you wanted to import has not been imported? I am currently thinking about introducing a new include scope which, instead of import, blindly includes everything specified without performing any duplicate checking or overriding. For your example project that means that you could use include instead of import to produce an invalid model due to duplicate dependencies in the final model. That way you would notice the conflict because Maven would fail due to an invalid model you need to fix. Would that be an option?

[jira-importer]

Stephane Nicoll commented

"Override what is already there". This is quite a big issue in Spring Boot land today. We are promoting a "starter-parent" concept where a set of plugins are pre-configured with sensible defaults and "core" dependencies can be managed via properties (see the spring-boot-dependencies and spring-boot-starter-parent).

This works fine until we started to integrate other components on top of Spring Boot that have their own boms and some specific requirements regarding dependencies. Spring Cloud Services is a typical case that is causing trouble: that project need to override the version of one dependency provided in the parent and there is no way in Maven to do that. What we document is to add the bom and you should be good to go.

Failing is not an option and It looks like we have different concept of what a BOM is. In my opinion, a BOM is way to share the dependency management of a set of dependencies. As such, when a BOM is added in a project, it should behave exactly as if all the dependencies that it contains were added to the project. Since adding the dependency explicitly (rather than via the import of a bom) works, I guess this was intentional. I honestly don't get why it should behave differently.

Taking a bit of distance of the actual problem, I think our use case is actually quite nice: we want to provide default dependencies management and a set of nice defaults for plugins that are typically used in a Spring Boot application. The only way I know to do that is via a parent pom. That's what we did. Now we are stuck if the user wants to override the version of a dependency because the bom import does not work. I feel that use case is legit. What would you recommend then?

[jira-importer]

Christian Schulte commented

To clarify further. You are referring to the following example setup taken from Spring Boot Quickstart

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.3.2.RELEASE</version>
</parent>
<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
</dependencies>

What issue are you trying to solve? If a user adds something to the <dependencies> or <dependencyManagement> sections leading to overriding something coming from org.springframework.boot:spring-boot-starter-parent, you want Maven to ignore that and use the parent data? Something along what the Java @Override annotation or the final modifier would be used for? You need to flag a dependency final in the parent so that it is not possible to override it down the hierarchy?

??that project need to override the version of one dependency provided in the parent and there is no way in Maven to do that??

Just add that dependency to the correct "level" (here: the parent) where it will correctly override things? Could you please explain what needs to be added to <dependencies> or <dependencyManagement> to demonstrate the issue.

Put another way. The import produces a conflict with something already in the model. Maven resolves that conflict automatically by applying a "target model is dominant" strategy. You are asking for Maven to apply a "source model is dominant" strategy. I am suggesting to add a new include scope which forces the user to resolve such conflicts manually. I think that is the best solution to this. If there is a conflict, the user needs to tell Maven what to do manually otherwise Maven fails due to an invalid model it cannot fix automatically.

[jira-importer]

Stephane Nicoll commented

Okay let me try to clarify. I don't think I am asking for anything funky. I want that anything that is defined closer "wins" regardless of how it is added.

Think it this way (with a A -> B -> C hierarchy where C is the child and A the grand-parent). A resolves its dependency management (including the ones from BOM). That gives a unified dependency management. Then B applies and does the same thing, adding or overriding some entries if necessary. Then C does. To me it's much more consistent to do things that way because you know that whatever you define at a given level will override the values from the parent if necessary.

Imports never overwrite anything already present in the model.

This very issue is about the fact that adding the dependency directly works, adding a bom that defines said dependency doesn't. IMO this is inconsistent considering how we're using BOM (= share a dependency management section).

Answering to your question. Of course I don't want that. I just want that whatever is defined in the user project overrides what the parent provides. So let's say that the parent provides com.foo:bar:1.0.0. If the user adds com.foo:bar:2.0.0 then all is well. If the user adds a bom that contains a dependency management for com.foo.bar:2.0.0 it doesn't. That's the problem I am trying to raise here.

In my mind there is no conflict: I am using the tools Maven provides to have a decent dependency management. The only mechanism I know to share dependency management is the BOM. Is there another one? (Changing the parent is not an option). I don't think so. If we can't use that mechanism to override versions (while we can by adding the dependency directly), it looks like something is missing in Maven.

I am ok with whatever alternatives Maven offers that would allow me to share the dependency management. I just don't think there is one.

[jira-importer]

Christian Schulte commented

Please take a look at DefaultModelBuilder around line 376 to 409. As I understand the issue, you want the import to be performed into the intermediate model (included where it is declared) so that those imported values are available to inheritance processing. Did I get that correctly?

I looked at spring-boot-dependencies-1.3.2.RELEASE.pom and spring-boot-starter-parent-1.3.2.RELEASE.pom. The issue is not present in there, right?

A -> B -> C

In A you want to import X:AID:VER bringing in GID:AID:1.0
In B you want to import Y:AID:VER bringing in GID:AID:2.0
In C you want to import Z:AID:VER bringing in GID:AID:3.0

The effective model of C should have GID:AID:3.0 overriding GID:AID:2.0 from B overriding GID:AID:1.0 from A. Correct?

[jira-importer]

Stephane Nicoll commented

Please stop looking at two poms in Spring Boot, the issue is more complicated than that and I've attached a sample project in my original description. Can we stop discussing technical and discuss feature please? :)

So yes, I would like that a BOM defined at a given level is included in the dependency management of said level, overriding what is provided by the parent if necessary. And that whole thing should become the dependency management of that level, that can be also overridden further down the road if necessary. Long story short: I want bom imports to behave exactly as if I added each dependency contained in the bom directly in the project.

Your example is correct.

No back on the feature. BOM is an effective way of sharing dependencies: instead of having to copy 10 <dependency> with their respective versions, I put them all in a bom and I import that bom with scope import. This is a very effective way of sharing a dependency management. All I am asking here is that it behaves consistently in Maven.

You already said that the current behaviour is on purpose (and other users don't want the bom to override dependencies) which means the bom would be some kind of "default if not exist" thing. I honestly don't understand the use case the way we (and others in the community) use the bom.

On my own personal opinion, the use case I am describing is much more logical than the current behaviour: when you import a BOM in a project, it would be as if you had copy/pasted all the dependencyManagement section of that BOM in the project. So I am seeing the BOM as a shortcut. Does that make sense?

[jira-importer]

Christian Schulte commented

We cannot change the way the import scope is defined. We could introduce a new scope include behaving exactly the way you describe. Since that scope would be processed very early (before inheritance and interpolation), you cannot use any features like property expansion.

<dependency>
  <groupId>must be a constant</groupId>
  <artifactId>must be a constant</artifactId>
  <version>must be a constant</version>
  <scope>include</scope>
  <type>pom</type>
</dependency>

Will provide a 3.4.0-SNAPSHOT later today.

[jira-importer]

Andy Wilkinson commented

Since that scope would be processed very early (before inheritance and interpolation), you cannot use any features like property expansion.

Given that I can do this:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.example.foo</groupId>
            <artifactId>alpha</artifactId>
            <version>${foo.version}</version>
        </dependency>
        <dependency>
            <groupId>com.example.foo</groupId>
            <artifactId>bravo</artifactId>
            <version>${foo.version}</version>
        </dependency>
    </dependencies>
</dependencyManagement>           

I'd really like to be able to move that dependency management out into a reusable bom and do this:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.example.foo</groupId>
            <artifactId>foo-bom</artifactId>
            <version>${foo.version}</version>
            <scope>include</scope>
            <type>pom</type>
        </dependency>
    </dependencies>
</dependencyManagement>

If the accepted goal for include is to provide a shorthand for declaring the managed dependencies directly, I'd expect that the same rules for the use of properties would apply. Is that not possible?

[jira-importer]

Christian Schulte commented

For the include scope it will not be possible to use anything requiring inheritance processing. Properties would be available only after inheritance processing has been performed. The import scope is processed after inheritance processing and interpolation. So there the complete model features are available.

[jira-importer]

Christian Schulte commented

You can grab a recent 3.4.0-SNAPSHOT from the Jenkins job for testing. Just change import to include to see how that goes. Please report back any issues you are encountering. It's still lacking any unit testing and I haven't written integration tests yet. Watch out for the SNAPSHOT to contain this commit: af2c42c2c16b6d6f15867646fd82b224359a5910

[jira-importer]

Christian Schulte commented

If Maven would support multiple inheritance

<parents>
  <parent>
  ...
  </parent>
</parents>

none of these features would be needed, right?

[jira-importer]

Arnaud Heritier commented

I agree that the import scope which add depMgt entries which cannot be redefined was always a nightmare especially nowdays where BOMs are often used and thus you can override a part of the depMgt with a BOM if the entries were already defined in another one.
Changing the behavior is difficult, but I'm not sure if it isn't better than an new scope which will be difficult to explain: import vs include

[jira-importer]

Andy Wilkinson commented

For the include scope it will not be possible to use anything requiring inheritance processing. Properties would be available only after inheritance processing has been performed.

Is that due to a technical restriction or a design choice that's been made? If I can use a property when defining a managed dependency directly, why can't I also use a property when I'm using the new include scope? Put another way, once property interpolation is available can't the model be processed such that any include-scoped dependency is replaced with the content of the bom that it references?

[jira-importer]

Christian Schulte commented

It's a technical restriction. The very purpose of the include scope is to be processed before inheritance. This is what this issue is about. The import scope is implemented in a way that it is one of the last steps involved during model building. After inheritance processing, property substitution, path normalisation etc. (when the model already is completely built). That's what has lead to this issue and that's why you can use properties for the import scope. For the include scope, the following currently would not be possible.

<dependency>
  <groupId>${groupId.property}</groupId>
  <artifactId>${artifactId.property}</artifactId>
  <version>${version.property}</version>
  <scope>include</scope>
  <type>pom</type>
</dependency>

The values for these properties need to be available during reading the model before any further processing takes place. If it will be released that way, it will not take long until the first issue is filed stating that is a bug. I haven't found a way to make that work without breaking the very purpose of the include scope yet. Building the complete model once to remember the values to be used for the include scope to then start the model building over again with those values may be solution. Inheritance processing would need to be re-implemented to support only processing properties.