Skip to content

A Conversation about versions and setting them in build configuration

Jump to bottom
Martijn Verburg edited this page Jan 4, 2022 · 1 revision

The following email thread between Matthias Klose and Gil Tene was captured in an OpenJDK mailing list and reproduced here to help give folks some history on the the various version number confusions.

On May 27, 2019, at 6:34 AM, Matthias Klose [email protected] wrote:

Hi guys,

The following is meant to be constructive and informational, so please don't read it as anything other than that. With that said, any and all "you are plain wrong, and here is why" followups to what I say below are welcome.

until recently we didn't have any source code releases for OpenJDK at all.

Not sure what you mean by this. Can you clarify what you mean when you use the term "source code releases"? And how the thing you mean differs from historical practice in OpenJDK?

By my understanding OpenJDK, as a source code project, has been producing source code releases pretty much since OpenJDK 6 was first released, and has never stopped doing so, with continuing update releases happening within major version project on a pretty regular basis.

Version formats have changed across major java versions, tagging and development processes going into a release also evolved and changed over time. But at all points in the past several years, it's been pretty clear when a release happened, and what the source code for that release actually was.

Binary builds of those released sources have been around for quite a while as well. But in the past, these may not have been created consistently by the project lead for every update.

What is "new" (AFAIK), specifically in 8u and 11u (and not in other projects thus far), is the posting of project-lead blessed, TCK-tested builds of "vanilla" released (as well as EA) sources of 8u and 11u (see e.g. https://mail.openjdk.java.net/pipermail/jdk8u-dev/2019-April/009105.html and subsequent discussions) at some external location. That's a very positive development, which I certainly hope will last and continue. But as noted in that e-mail thread, those binaries are simply good, fully-TCK-tested builds of the released sources, posted to a known location. They did not change the release process in any way.

I appreciate the effort to have source releases, however the gap to produce binaries based on the source release is way too high. Instead of having a dozen of configuration options to build a binary, please can we have a default option which builds consumable packages by default?

As things stand, across the various java version projects, the default build modes for all sources (including releases) produce consumable-by-the-builder things, intentionally labeled in ways that would scare people away from using the bits as a release. This is intentional, and is aimed to make sure that things that look like an actual release will only happen through intent, assertive actions and real choices about reported versions.

E.g., when I build things using the defaults, I'd get stuff like:

# bash ./configure
# make all
# build/linux-x86_64-normal-server-release/jdk/bin/java -version
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-gil_2019_05_27_10_48-b00)
OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode)

And that's a good thing. It prevents the accidental leakage of unfinished bits to others.

To produce things you would give other people ("consumable" by others), you need to intentionally make assertive (non-default) choices about configuration. Normally, stating your update version, build version, and milestone (or "pre" version) at the very least, and likely things like vendor version, etc. Non of those have good "defaults", as you don't want any of those to appear "by accident" in a build.

The same is true for the sources of the actual quarterly releases, which are no more than a snapshot of the sources at the release point.

To create a binary build that does not carry the "-internal" identification, and that has specific update and build numbers, you have to assertively declare those in the build configuration. But that's not very complicated. E.g.

# bash ./configure --with-update-version=972 --with-build-number=b41 --with-milestone="snapshot"
# make clean; make all
# build/linux-x86_64-normal-server-release/jdk/bin/java -version
openjdk version "1.8.0_972-snapshot"
OpenJDK Runtime Environment (build 1.8.0_972-snapshot-b41)
OpenJDK 64-Bit Server VM (build 25.972-b41, mixed mode)

And with the special milestone called "fcs":

# bash ./configure --with-update-version=982 --with-build-number=b42 --with-milestone="fcs"
# make clean; make all
# build/linux-x86_64-normal-server-release/jdk/bin/java -version
openjdk version "1.8.0_982"
OpenJDK Runtime Environment (build 1.8.0_982-b42)
OpenJDK 64-Bit Server VM (build 25.982-b42, mixed mode)

Note:

For OpenJDK 8u "fcs" is a special milestone. It is the specific thing you need to do to get rid of the milestone part of the versions string. Any other value (including "", which will revert to "internal") will show up in the version, because of this in common/autoconf/spec.gmk.in :

# These variables need to be generated here so that MILESTONE and
# JDK_BUILD_NUMBER can be overridden on the make command line.
ifeq ($(MILESTONE), fcs)
  RELEASE=$(JDK_VERSION)$(BUILD_VARIANT_RELEASE)
else
  RELEASE=$(JDK_VERSION)-$(MILESTONE)$(BUILD_VARIANT_RELEASE)
endif

11u is a little different. The release versioning is more specifically defined (JEP322), and the term for that "disqualifier" part for the version that tells you it is not an actual release (what 8u sets using --with-milestone) is the "pre-release identifier" ($PRE) part of the version string (set --with-version-pre).

The default behaves the same with regards to "-internal" in the sense that $PRE defaults to "internal", a =nd must be assertively set to "" to get rid of it. But in 11u a default for the UPDATE portion of the JEP322 version string is typically baked into make/autoconf/version-numbers (as opposed to having to be set via --with-update-version in 8u)

so e.g.:

# bash ./configure
# make all
# ./build/linux-x86_64-normal-server-release/jdk/bin/java -version
openjdk version "11.0.3-internal" 2019-04-16
OpenJDK Runtime Environment (build 11.0.3-internal+0-adhoc.gil.11u1103rel)
OpenJDK 64-Bit Server VM (build 11.0.3-internal+0-adhoc.gil.11u1103rel, mixed mode)

But when you build something that is fit for others to consume, you will still want to provide a build number with --with-build-number, use -with-version-pre="" if you want to say it is an actual release (use e.g. "ea", or just leave blank to make it "internal" otherwise), and in addition set --with-version-opt to control that thing that defaults to "adhoc.$USERNAME.$basedirname". Finally, --with-vendor-version-string="18.9" is considered "vanilla" (the year.month for Java SE 11), and that's where non-vanilla distros tend put their name in.

So e.g.:

# bash ./configure --with-version-build=42 --with-version-opt="" --with-version-pre="" --with-vendor-version-string="18.9"
# make all
# ./build/linux-x86_64-normal-server-release/jdk/bin/java -version
openjdk version "11.0.3" 2019-04-16
OpenJDK Runtime Environment 18.9 (build 11.0.3+42)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.3+42, mixed mode)
  • I see it's important to display the version string as the first line of java -version. The source release should set that correctly.

See above discussion about why (by long standing OpenJDK historical practice) version strings are controlled by the build configuration, and not by the source code of a release.

  • The OpenJDK source release ships with the vendor set to Oracle. Distributors set that to Azul, AdoptJDK, Debian, and probably other values. The binaries built by the OpenJDK itself set that to some sort of version string. An "unknown" vendor causes issue, because some software (LibreOffice, Gradle) uses or at least used that to check for a valid java installation.

I'll assume you are referring to the "18.9" part in the version string reported by the "blessed by project lead" builds of the OpenJDK 11u 11.0.3 release:

# ./openjdk-11.0.3+7/bin/java -version
openjdk version "11.0.3" 2019-04-16
OpenJDK Runtime Environment 18.9 (build 11.0.3+7)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.3+7, mixed mode)

That 18.9 part is the "vendor version string" as defined by JEP322 (a new property that did not exist in 8u). The choice of "18.9" (the year.month that Java SE 11 was released) is both the actual example vendor version string shown in the JEP, and follows the precedent already set for "vanilla" OpenJDK builds of 11, 11.0.1, and 11.0.2 produced by the previous project lead during the first 6 months of 11u. (note that those were the OpenJDK build posted by Oracle, not the Oracle JDK builds).

Since "18.9" was there as a vendor version string form the very start of OpenJDK 11, we can assume that anything that knows how to parse JEP322 versions will not have a problem with it.

  • The version number should be used for both the source release and the binary package. E.g. the 11.0.3 source release is missing the -ga modifier.

I'll assume that you are referring to version string reported by the binary, e.g. in response to -version, and not to package names, file names, or tags in source control.

The version string conventions for each existing Java SE version are already established, and they (unfortunately) differ by major version. Thus far, for all Java SE versions I know of which had an OpenJDK project (6, 7, 8, 9, 10, 11, 12), neither Oracle nor OpenJDK builds have included a positive (e.g. "-ga") indicator in the version strings of actually-released versions. Instead, the long standing convention has been to include a disqualifier ("-ea", "-internal", "-rc1") in non-released builds. The build system defaults to using the -internal qualifier in all versions to date.

Thus the current output from the project-lead blessed builds of 8u212 is:

# java -version
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (build 1.8.0_212-b04)
OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)

(which, since there is no "disqualifier" in the version, means that it reports as a build of an actual release).

The technical term for this "disqualifier" has varied over the years. As shown above, 8u calls it "milestone", 11u calls it a "pre-release identifier". But in all versions so far, not having it there, means that it is a release.

Changing version string formats and conventions for indicating things within a major java version is a generally bad idea. Adding "-ga" to actual update releases of existing Java SE major versions is likely to blow up a whole bunch of things. For one: a common ways to determine that a version of OpenJDK or Oracle JDK is an actual released version (as opposed to some ea/beta/rc1/internal/experimental thing) is to verify that there is no "-XYZ" part (the milestone or $PRE) in the version string.

Making changes to conventions across versions is quite possible, and has been done multiple times (e.g. JEP 322, JEP 223). JEP 322 is what we currently follow for 11u and above. If it insufficient for some reason, and we want to change it yet again for e.g. OpenJDK 14, that's something to discuss, I guess., But such changes should not affect existing Java SE versions (12 and below).

  • To include a package in a Linux distro, you have to use a monotonically increasing version number, and you have to follow the versioning constraints for the distro. So sometimes you have to juggle with the version numbers. E.g. for Debian/Ubuntu a second dash is problematic for version comparisons. I consider uploads to a development distro as essential, so I have to plan for these version numbers as well. Last time when I asked on the mailing lists, people seemed to be fine with the versioning, however if needed, we could document such versioning on the OpenJDK wiki?

This is a good point (make sure your package names retain monotonically growing version numbers), but since these rules apply to the package names as seen by the distro (and not to the version strings) simple distro-specific file name conversion conventions seem to suffice. This is often necessary simply because conventions and file name requirements can vary widely between OSs (some don't like dots, some don't like dashes, some deal with capitalization in interesting ways).

E.g. you can convert every dash in the version string to to an underscore in the .deb or .rpm package names.

An interesting problem shows up if the parsing of the version string related fields appears to "go backwards" in your monotonic comparisons (e.g. if the sorting is text based rather than numeric, and 11.0.10 would be considered earlier than 11.0.9, or 8u92 was considered to come after 8u102). I can think of a few ways to resolve that with package name conventions, but I don't really know if this is a problem. Does debian compare numeric sequence substrings in package name numerically, or lexicographically?

  • It would be very helpful to see directly in the binary how the build was configured. GCC is showing this information with gcc -v Python is showing that with python -c 'import sysconfig; print(sysconfig.get_config_var("CONFIG_ARGS"))' Or maybe this already exists?

Usually binaries in a distro come with a changelog, however sometimes even that is stripped away by re-distributors of binaries.

  • The configure system has some issues with invalid configure arguments, e.g. configuring --with-version-build='' leads to a failing build.

Setting --with-version-build="" works on 8u (it is a string).

In 11u, you'd use --with-version-build=0 (it is an integer).

Clone this wiki locally