dependencies.txt

Dependencies
============

Each gradle project can have multiple (named) "configurations"
and each configuration can have dependencies attached to it.

There are some standard conventions so, for example, the Java plugin
adds standard configurations such as "api", "implementation",
"testImplementation" and others. These configurations can also inherit
from each other; more about this topic can be found here:

https://docs.gradle.org/current/userguide/dependency_management_for_java_projects.html#dependency_management_for_java_projects
https://docs.gradle.org/current/userguide/java_library_plugin.html#sec:java_library_separation
https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plugin_and_dependency_management

Solr typically uses three configurations and attach project
dependencies to them:

api  - makes a dependency available for main classes, tests and any
  other modules importing the project (exportable dependency),

implementation - makes a dependency available for main classes, tests
  but will *not* export the dependency for other modules (so their
  compilation classpath won't contain it).

testImplementation - makes a dependency only available for test  classes.


Adding a library dependency
---------------------------

Let's say we wish to add a new dependency on library "foo.bar:baz" in
version 1.2 to :solr:core. Let's assume this library is only
used internally by the project. For new dependencies, we would add
the dependency and its version to gradle/libs.versions.toml first:

[versions]
...
foo-bar-baz = "1.2"
...

[libraries]
...
foo-bar-baz = { module = "foo.bar:baz", version.ref = "foo-bar-baz" }

Note that the used names separated by dashes are later referenced with dots
instead of dashes, but more on that later.

The chosen name for the module should more or less reflect the module's
group name and module id in a way that it groups related dependencies under
the same "prefix" (see below). There is no specific convention here and
group prefixes for domain names like "com" and "io" are avoided, as they
do not add any value and increase the size of the reference / alias name.

The :solr:core project is configured by solr/core/build.gradle, and we would
add (or modify) the dependency block as follows:

dependencies {
  implementation libs.foo.bar.baz
}

In the project we use the default name "libs" that is used to reference
the version catalog gradle/libs.versions.toml.

The "implementation" here is a named configuration; we don't need to declare
it because it is declared for us by the java-library plugin.

In case the IDE does not auto-completes the reference, you may have
to sync your project so that the newly added library is found.

As mentioned before, we can use the dashes to group related libraries
together. So let's assume we have another dependency on "foo.bar:biz",
which is part of the same project as "foo.bar:baz" and therefore share
the same version.

In this case we would want to use the same version for both libraries
and add them as follows to the version catalog gradle/libs.versions.toml:

[versions]
...
foo-bar = "1.2" # Use a shared name for both libraries
...

[libraries]
...
foo-bar-biz = { module = "foo.bar:biz", version.ref = "foo-bar" }
foo-bar-baz = { module = "foo.bar:baz", version.ref = "foo-bar" }

This way, both libraries use the same version reference and updates
would affect both.

Adding new libraries requires additional actions. The first you want
to do is to run versionCatalogFormat to sort the version catalog.

This command does also remove unused libraries. You can use "# @keep"
with a reason why the library should not be removed. This is sometimes
necessary if the usage of a library is not identified by the plugin,
like when using it with "classpath [dependency]" or for version alignment.

The next you want to update the lockfiles file using the following command:

gradlew resolveAndLockAll --write-locks

Starting in Solr 10, we also have to update a kotlin-js-store lockfile
that is used by the new Admin UI WebAssembly target. This can be done by
running:

gradlew kotlinUpgradeYarnLock

Since we are responsible to provide and maintain the versions of
libraries, the lock file will reflect the versions of the version
catalog.

The locking will fail if multiple versions of the same dependency are found.
This may be the case if libraries have a used library as transitive
dependency with a different version.

Update Lucene prerelease
------------------------

The main branch of Apache Solr uses a prerelease of Apache Lucene that
is published by ASF Jenkins on:
  https://nightlies.apache.org/solr/lucene-prereleases/

If you want to upgrade Lucene to a newer build proceed like the following:

- Go to https://ci-builds.apache.org/job/Solr/job/Lucene-prerelease-main/
- Log in with ASF account
- Press on "build now" on left side
- Wait until build is finished.... (can take long time, if other builds are
  queued)
- remember the build number of Jenkins (left side, first build in list,
  prefixed by '#')
- Edit gradle/libs.versions.toml and change Lucene's version to '9.0.0-prereleaseX',
  with 'X' is the jenkins build number
- Edit ./gradle/globals.gradle and change jenkins build number, too
  (this directs the repository to the one created by latest build):
    def lucenePrereleaseBuild = 'X'
- Run: gradlew resolveAndLockAll (as described before)


Lucene local dependency substitution
-------------------------------

Because of the fundamental nature of Solr's dependency on Lucene, Solr
facilitates dependency substitution to support developing against a local
Lucene version (e.g., for joint Solr and Lucene development).

A local Lucene dependency substitution is resolved by evaluating the
following conditions, in priority order:

  1) A "-Plucene.dev.version=[version]" property, resolving Lucene artifacts
     from a local Maven repository.
  2) A non-empty property "-Plucene.dev.path=[path]" pointing to a local
     path. Relative paths are resolved against the Solr root project
     directory.
  3) An auto-wired 'lucene' subdirectory, if present. To skip auto-wiring,
     pass a blank value for condition 2: "-Plucene.dev.path=".

Note: Lucene dependencies substituted via conditions 2 and 3 require the
local Lucene repository to use the same Gradle version as the Solr root
project.

Local Maven repository Lucene artifacts (for use with the
"-Plucene.dev.version" project property) may be created by running the
following command from within a local Lucene project:

  gradlew mavenToLocal

As is the case with project properties generally, the relevant properties
can be invoked on the command line, or configured via the gradle.properties
file (the latter approach is particularly useful for making IDEs aware of
associated dependency substitutions).


Inspecting current dependencies
-------------------------------

The tree of dependencies of a project (in all configurations) can
be dumped by the following command (example):

gradlew -p solr/solrj dependencies

But this can be a bit overwhelming; we will most likely be interested
in just the "publicly visible" and "classpath-visible" configurations.

The publicly visible project dependencies (classes shared by other
modules importing our module) can be displayed with:

gradlew -p solr/solrj dependencies --configuration api

And the "private" set of dependencies (real classpath) can be dumped
with:

gradlew -p solr/solrj dependencies --configuration runtimeClasspath

Additionally, you can always inspect the lockfile of a module to see
which versions are used for which configuration.

Excluding a transitive dependency
---------------------------------

Let's say "foo.bar:baz" has a transitive dependency on project
"foo.bar:irrelevant" and we know the transitive dependency is not
crucial for the functioning of "foo.bar:baz". We can exclude it
by adding an exclusion block to the original declaration:

dependencies {
  implementation(libs.foo.bar.biz, {
    exclude group: "foo.bar", module: "irrelevant"
  })
}

Note the brackets - they are important and prevent accidental
mistakes of applying the exclusion to the wrong scope.


Updating dependency checksums and licenses
-----------------------------------------

The last step is to make sure the licenses, notice files and checksums
are in place for any new dependencies. This command will print what's
missing and where:

gradlew licenses

To update JAR checksums (sha1) for licenses use:

gradlew updateLicenses

Note that this Gradle task does only update the checksums and does not make
any changes to license or notice files.

Whenever the library's LICENSE or NOTICE file changes, these changes should
be reflected in our copies under solr/licenses. Currently it is necessary to
manually review and update these files. This process is subject to change,
progress can be tracked in SOLR-15929.

License and notice files may be picked from the libraries' repositories.
When looking up the libraries in Maven Central (https://search.maven.org)
almost all projects have a direct reference to the source code (right side),
usually a GitHub repository, where you can find the License and Notice file
in the root directory.

Remember to check out the correct tag / release before copying any license
or notice file. Some multi-module projects that publish multiple artifacts
may have subdirectories for each artifact. These directories sometimes
hold a different license for that specific artifact, so make sure to copy
the right license file. Other multi-module projects may have only a single
license and notice file for all modules, like netty, so multiple dependencies
fromt he same group may reference the same license and notice file.

Other places where you may find a license and notice file are in the pom.xml
file as a URL under a <license> tag if there is no reference to a repository
in Maven Central, or in the artifact downloaded by maven when the library
is added as a dependency (in IntelliJ IDEA the libraries can be found
in the project view under External Libraries at the bottom).