Support GitHub Merge Queue (merge_group events)

[MPV]

Issue Type

• Feature enhancement

Summary

GitHub has somewhat recently released a feature called a "Merge Queue".

It auto-merges all queued PRs (after having made sure PRs are up-to-date/rebased with green CI checks).

However it seems it sends a custom webhook (merge_group instead of push / pull_request).

How might this be added support for in GoCD?

See:

• https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/merging-a-pull-request-with-a-merge-queue
• https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue
• https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue#triggering-merge-group-checks-with-github-actions
• https://docs.github.com/en/webhooks-and-events/webhooks/webhook-events-and-payloads?actionType=checks_requested#merge_group
• https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#merge_group

Steps to Reproduce

1. Enable GitHub Merge Queue on a repo
2. Create a PR
3. GoCD runs on PR and marks check as green (✅)
4. Add the PR into the Merge Queue (equivalent to clicking Merge/Rebase/Squash when Merge Queue now is enabled)

Expected Results

Step 5: GoCD should re-run the checks (as requested by GitHub)
Step 6. GitHub will auto-merge the PR

Actual Results

Step 5: Doesn't happen
Step 6: Doesn't happen

Possible Fix

Only push event seems supported:

gocd/api/api-webhook-v1/src/main/java/com/thoughtworks/go/apiv1/webhook/controller/PushWebhookControllerV1.java

Lines 95 to 104 in 1e1d710

	protected String github(Request request, Response response) {
	final GitHubRequest req = new GitHubRequest(request);
	validate(req, cat(GitHub.PUSH, GitHub.PING));
	if (is(GitHub.PING, req)) {
	return acknowledge(response);
	}
	return notify(response, req.parsePayload(GitHubPush.class), req.scmNamesQuery());
	}

...although there seems to be some support for pull_request too...?

gocd/api/api-webhook-v1/src/main/java/com/thoughtworks/go/apiv1/webhook/controller/validation/GitHub.java

Lines 31 to 33 in 1e1d710

	public static final Set<String> PUSH = Set.of("push");
	public static final Set<String> PR = Set.of("pull_request");
	public static final Set<String> PING = Set.of("ping");

[chadlwilson]

There's support for PR webhook events via config repo plugins to respond to PR events, by say, dynamically generating new pipelines with materials pointing to the PR's ref. I believe it only is used/supported by https://github.com/gocd-contrib/gocd-groovy-dsl-config-plugin#support-for-branches-and-prs

gocd/api/api-webhook-v1/src/main/java/com/thoughtworks/go/apiv1/webhook/controller/ConfigRepoWebhookControllerV1.java

Lines 81 to 101 in 0f58107

	String github(Request req, Response res) {
	final GitHubRequest github = new GitHubRequest(req);
	validate(github, cat(GitHub.PUSH, GitHub.PR, GitHub.PING));
	if (is(GitHub.PING, github)) {
	return acknowledge(res);
	}
	final Payload payload = determinePayload(github);
	if (payload instanceof GitHubPR) {
	final GitHubPR pr = (GitHubPR) payload;
	if (!pr.isInteresting()) {
	LOGGER.debug("[WebHook] Ignoring {} because we are not interested in action: {}", pr.descriptor(), pr.action());
	return success(res);
	}
	}
	return triggerRepoUpdate(res, payload, repoFromRequest(req));
	}

Right now, GoCD has no integrated holistic support for both reacting to events and then reporting back checks - these typically happen via different plugins. I'm not super familiar with the Merge Queue, but if it just expects notifications against the commit SHA for the special branches it creates then it should work as well (or as badly) as the current support via the community https://github.com/gocd-contrib/gocd-build-status-notifier

How would GoCD be anticipated to "support" merge group events from your perspective? Do you use the Groovy DSL Config Repo plugin? Basically the same as pull_request?

Since this seems to fundamentally relate to auto-magic PR support (dynamic pipelines corresponding to PRs), I imagine it's something that'd need to sit alongside and work with the PR event handling inside config repos.

[MPV]

I'm still learning, but from what I've understood when a Merge Queue arrives at (preparing) your PR in the queue, it notifies your CI tool and wants it to perform its checks for the changes. Here's more:

The queue then creates a temporary branch that contains the latest changes from the base branch, the changes from other pull requests already in the queue, and the changes from your pull request.

-- https://github.blog/changelog/2023-02-08-pull-request-merge-queue-public-beta/

[MPV]

So my problem (so far) hasn't been that I'm not getting "check status updates" back from GoCD.

Rather that GoCD receives a webhook but doesn't trigger any PR pipeline job for it.

Ideas welcome.

[chadlwilson]

How is your current GoCD setup configured to respond to regular PRs? Which plugins are you using?

[MPV]

We're using:

• https://github.com/ashwanthkumar/gocd-build-github-pull-requests

[chadlwilson]

Ok, that plugin independently tracks PRs (probably polls the GH API for new PRs every minute or something?). It sounds to me like you want/need that plugin to handle merge group builds, rather than GoCD itself.

I'm not familiar with how that plugin does its magic, but im not sure GoCD responding to webhook events will change any of the way it behaves.

My understanding is that current it basically updates the material it uses on a single pipeline every time a PR is created or changes (to point to a new ref/branch). To handle merge groups I imagine that plugin would also need to use similar magic to reuse the same pipeline to build off the special merge group branch (and thus report status checks).

What I'm trying to say is that the model GoCD itself uses to allow PR support via webhooks and config repo plugins is very different than the approach that this custom git material plugin takes, so anything done within GoCD itself likely won't help you, if depending on that particular plugin.

[MPV]

Thanks for your thoughts I delving into this. I've opened a downstream issue for the plugin, hoping the investigation can continue there. 🤞