Sourcehut (sr.ht) builds

Overview

Sourcehut is a popular development hosting service targetting more traditional developers. It provides Mercurial and Git source hosting, a CI workflow service, mailing list hosting, ticket tracking, and overview (aggregated services) project hosting. It is not a free service, but all of the services are open source and can be self-hosted.

Sourcehut has first-class support for Mercurial and Git, but not Jujutsu. This page contains tricks & tips for using Sourcehut with Jujutsu, using only the git commands available in the CI environment.

builds

Sourcehut uses .build.yml configuration files similar to Github workflows. Sourcehut's build server builds containers based on one of a half-dozen supported base images, and executes build commands in busybox invocations. It sets up a number of environment variables, creates the environment, installs dependencies, and then executes scripts defined by the user.

Extracting Version information from repository metadata

BLUF: VERSION=$(git ls-remote | grep $(git ls-remote | grep $GIT_REF | cut -f1) | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+')

A common build action is to get a version number from the build environment. Under Mercurial, one might use a command such as hg log -r tip --template '{bookmarks} | grep -oE '\S+$' to get a version from the bookmarks on the checked-out source. Most of the commands for getting the same information from git are predicated on the assumption that the user is using tags. This can fail when the user is using Jujutsu bookmarks, which are recorded as git branches.

Sourcehut builds sets an environment variable $GIT_REF to the reflog name that triggered the build. This is a project-level configurable variable -- for example, a project could only trigger builds on commits to a development branch, in which $GIT_REF would always be refs/heads/development. By default, $GIT_REF will always be refs/heads/master, since the default branch is master. In particular, when jj git push --allow-new v1.2.3 is used to push a bookmark, while both branches point to the same commit in git, in builds $GIT_REF will always be the configured trigger branch -- nominally master. Since the git check-outs are sparse, no other information is available locally. Hoever, it is still possible to derive the bookmark version, and this is by querying the external repository.

What the command above does is get all of the remote heads for the repository; looks for the $GIT_REF entry; finds all other entries with the same reference, and looks for a semver match.

The ls-remote command is actually quite fast, as the build servers are close to the repositories, and this command executes quickly. It assumes the versions are in semver format; if your versions are in a different format, you will need to adjust the pattern. In particular, it does not expect any extensions allowed by semver, such as v0.0.0-test. Again, adjust the pattern for your use cases. It may also pick the wrong version if you push multiple branch names on master. Finally, you'll want to check for failed matches and assign a default version for cases where you push non-bookmarked commits, e.g.:

VERSION=$(git ls-remote | grep $(git ls-remote | grep $GIT_REF | cut -f1) | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+')
[[ $VERSION == "" ]] && VERSION=dev-$(git ls-remote | grep $GIT_REF | cut -f1)