Security Advisories Feature Requests & Improvements

[JLLeitschuh]

I've been working with GHSA since the feature was originally released. I've attempted to capture the collection of issues and features I've had over the past several years with GitHub Security Advisories (GHSA):

User Context: I'm primarily using GHSA as a security researcher, however, I've also been on the receiving end of reports (when I worked for Gradle). As such, my perspective more heavily comes from that of the party attempting to report security vulnerabilities, less the party on the receiving end, but I do have both perspectives.

Note to viewers outside of GitHub or GitHub Stars the slack links point to internal slack conversations with the GitHub Stars slack channel and aren't viewable by the public.

'User Pain Priority' is purely subjective based upon my own experience using GHSA. More stars, more painfully needed feature.

Preamble

Vulnerability handling does not end with disclosure. Vulnerabilities evolve after disclosure. They sometimes get worse. New information is discovered about the vulnerability. Threat actors begin abusing these vulnerabilities. This point plays into several of the feature requests below, so I figured that I'd open with this.

Feature Requests

Allow Security Researchers to Open GHSA without Requiring a Maintainer to be Involved

User Story: As a security researcher, I'd like to be able to open a GHSA against any OSS GitHub repository, without needing the maintainers to open the GHSA for me.

User Pain Priority: ⭐ ⭐ ⭐ ⭐ ⭐

Making contact with the maintainer can be one of most complex and annoying parts of vulnerability disclosure. As a security researcher, I've been begging for this feature since GHSA was released.

I understand that there's an argument about attempting to limit maintainer spam. The GitHub team doesn't want to create a private, potentially, information-restricted channel where maintainer harassment can be conducted. These concerns could be potentially mitigated by creating a list of known & trusted security researchers that have the permission to take advantage of this feature.

Allow Repository privileges below "Admin" to open GHSA

User Story: As a repository maintainer with 'write' level permissions on a repository, I should be able to create a GHSA against my repository.

User Pain Priority: ⭐ ⭐

According to the docs you still need to have the 'Admin' level permissions to create a GHSA against a repository.

https://docs.github.com/en/code-security/repository-security-advisories/creating-a-repository-security-advisory

IMHO, this permission is way too high. Having worked for a company that likes to keep the number of accounts with 'Admin' level permissions rather light (as it should be for a good security posture), requiring the 'Admin' permission level for simply creating a GHSA seems like a high bar to clear.

I'd argue that the permission of 'Write' should be enough to grant the permission to open a GHSA. If you have the permission to merge a PR, you should be able to open a GHSA and kick off a discussion with a researcher or your team about a sensitive vulnerability easily.

Discoverability of an individual GH users open/disclosed/closed GHSA

User Story: As a GitHub User who has been a part of several GHSA (either as a reporter or as a maintainer), I should be able to easily discover these GHSA across all organizations and repositories in a single GitHub UI.

User Pain Priority: ⭐ ⭐ ⭐ ⭐

As a security researcher, or maintainer, who has been involved with several GitHub Security Advisories, there is no way to "discover" these advisories in a simplified list or search via the GitHub API. User interfaces already exist for finding all open/closed issues and pull requests, however the same does not exist for GitHub Security Advisories.

• Pull Requests - https://github.com/pulls?q=is%3Apr+is%3Aopen+author%3A%40me+archived%3Afalse+sort%3Aupdated-desc
• Issues - https://github.com/issues?q=is%3Aissue+is%3Aopen+author%3A%40me+archived%3Afalse+sort%3Aupdated-desc

This can make it easy to lose open/in-progress GitHub Security Advisories. The only way to reliably make these advisories 'pop' to the users attention is via the notification feed, either finding the advisory in your feed, or by sending a message so others are reminded about the GHSA.

Personally, I use a hacky notification search to keep track of all my GitHub Security Advisories, but this is imperfect and doesn't contain advisories for notifications (idk why some are missing but they are): is:repository-advisory is:done is:read is:unread

A user interface to find all of the existing GitHub Security Advisories your user account has been a part of (ie. open/closed/disclosed) is sorely needed by the security research community.

Slack Conversation: https://github-partners.slack.com/archives/C01MF7QQK3P/p1614987770032300?thread_ts=1614987743.032200&cid=C01MF7QQK3P

GHSA Comments are Locked for EVERYONE After Disclosure

User Story: As a maintainer or reporter, I should be able to continue a discussion about a vulnerability after it has been published.

User Pain Priority: ⭐ ⭐ ⭐

See above note about 'vulnerability handling does not end with disclosure'.

When a GHSA is disclosed, no further discussion can happen on that GHSA amongst the maintainers/reporters. The rationale behind this has never been fully communicated to me, and it's baffling that this hasn't been fixed.

If GHSA truly want to be the location where maintainers disclose vulnerabilities, they should also be the way that the maintainers can keep their users updated as new information is disclosed about these vulnerabilities.

Keeping a dialogue/conversation channel open with the researcher through this is important. Sometimes the researcher finds new information that needs to be shared privately.

Currently, because GHSA are locked, to achieve this, the researcher must re-engage with the maintainers again instead of using the existing communication channel that had already existed prior to disclosure. From experience, this has been a cause for severe annoyance on my behalf as getting in touch with the maintainers via a secure channel is sometimes a significant amount of work. Having to spend this time again, is very annoying.

Ideally, additional comments should support the ability to either be private (exclusively to the the GHSA Collaborators & Publishers), or to make these additional comments public.

Slack Discussion: https://github-partners.slack.com/archives/C013VPD24BW/p1611951569123500

Discussion/Comment Support on GHSA

User Story: As a GitHub user, a GHSA should also allow me to comment on the GHSA to provide new/updated information or ask questions/clarification about the vulnerability.

User Pain Priority: ⭐ ⭐

See above note about 'vulnerability handling does not end with disclosure'.

New information about vulnerabilities can come from a variety of sources. Additionally, the discussion around a vulnerability can evolve and potentially uncover additional details or impacted conditions that were not originally understood by group fixing/disclosing the vulnerability.

A perfect example of this came regarding my disclosure of a vulnerability in Google Guava. When performing the initial disclosure, we assumed that the vulnerability only impacted unix like systems, but then the community brought it to our attention that the ramifications of the vulnerability could also adversely impact the Android ecosystem. This lead to a bunch of research by the community to capture this information and further refine the advice provided.

Google Guava Disclosure Issue Thread: google/guava#4011

Slack Discussion: https://github-partners.slack.com/archives/C013VPD24BW/p1611951569123500

GHSA Should Display Change History

User Story: As a visitor to the GitHub site looking at a GHSA, I should be able to easily see the edit history on that GHSA.

User Pain Priority: ⭐

As stated above, after a vulnerability is disclosed, it may evolve. New information may need to be communicated about the vulnerability.

ISO/IEC 29147:2018 Information technology — Security techniques — Vulnerability disclosure states the following:

9.5.12 Revision history
This section should contain the date when the advisory was first published. It can contain a modification history if the advisory is subsequently updated.

GitHub is fundamentally built on Git and diffs are commonly rendered in places like the wiki's and even issue comments (you can view edit history). This history should also be viewable on advisories as well. Being able to see how an advisory has changed over time, with new information being disclosed is valuable to the defender community.

Additionally, the ability to subscribe to information changes on a GHSA could be very valuable as well.

Slack Discussion

• https://github-partners.slack.com/archives/C013VPD24BW/p1613509427397700?thread_ts=1611951569.123500&cid=C013VPD24BW
• https://github-partners.slack.com/archives/C013VPD24BW/p1597868226043000

Display Inline Code Blocks Consistently across GHSA

User Story: As a visitor to the GitHub site looking at a GHSA, I should be able to see inline code blocks no matter what variant of the GHSA I am viewing (ie. advisory feed, from repository, or from external repository referencing another repository).

User Pain Priority: ⭐ ⭐ ⭐

These two GHSA are from the same source but render differently:

• Source GHSA: https://github.com/eclipse/jetty.project/security/advisories/GHSA-g3wg-6mcf-8jj6#:~:text=directory%0AtmpDir.mkdirs()%3B-,jetty.project/jetty%2Dwebapp/src/main/java/org/eclipse/jetty/webapp/WebInfConfiguration.java,-Lines%20511%20to
  
• GitHub 'Advisories' version: https://github.com/advisories/GHSA-g3wg-6mcf-8jj6#:~:text=eclipse/jetty.project%401b59672/jetty%2Dwebapp/src/main/java/org/eclipse/jetty/webapp/WebInfConfiguration.java%23L511%2DL518
  

Additionally, when GHSA are cross referencing another repository, the inline-code blocks don't render (for example):

• GHSA-f4jh-ww96-9h9j

Inline code block links should all render the same across GHSA publication locations.

Slack Discussion: https://github-partners.slack.com/archives/C013VPD24BW/p1642787947206400

[jhutchings1]

Thanks for this feedback thread, @JLLeitschuh !

[mkurz]

When a user watches only "Security alerts" for a repository, shouldn't that user get a notification from GitHub as soon a Security Advisory is published for that repo? Right now no notification will be send at all.

[RaphaelKimmig]

I'd really love to see that as well. Some projects only publish their security advisories on github and I'd love to be notified here when they do.

[LiroyvH]

Thank you for opening this. I notably agree that "User Story: As a repository maintainer with 'write' level permissions on a repository, I should be able to create a GHSA against my repository." is a major problem. In fact, the damn for Admin level permissions on a repository before being able to open a GHSA is a critical blocker towards effectively using the feature and severely limits the amount of people you can reasonably give this access to.

Currently, GitHub has an all-or-nothing approach to the Security Advisories. This means that if you want multiple people to be able to open GHSA's, you must give them all Admin access to the repository. That's quite the extreme and technically introduces a security vulnerability by having to assign deep permissions to achieve something that ought to be very simple. It should indeed be moved to the "write" group, at the very least the "maintain" group, but ideally you can simply per repository/team assign the ability to open GHSA's without requiring further permissions.

Moreover, there is the ability to assign "security managers" in organisations, but that role is again an all-or-nothing role. It demands that you give the user you assign this role to to get read access to ALL of the repositories; including those of other projects owned by the same organisation or other private repositories the user should not have access to.

Currently, this all-or-nothing approach is far too aggressive and leads to an unworkable situation.
Ideally, GitHub allows everyone to make custom permission profiles without having to drop $3K/year/seat, but in the interim this Security Advisories permission requirements really need to be revised.

[KateCatlin]

Hi @JLLeitschuh et al,

First of all, really appreciate the feedback. For the last couple of quarters, we've been doubling down on user interviews, synthesis of feedback, and setting aside time to knock out UX issues large and small. In order to prioritize, we need every data point we can get, and this is an excellent and thoroughly written source of data.

I'm going to walk through each of the issues I see voiced above one by one and let you know GitHub's game plan on each.

Allow Security Researchers to Open GHSA without Requiring a Maintainer to be Involved

I see you've proposed a solution to this, but it feels like the real need you're voicing is resolving this problem: "Making contact with the maintainer can be one of most complex and annoying parts of vulnerability disclosure."

If so, this is something we are working on addressing with some upcoming new features. User research and initial design are happening for that now.

Allow Repository privileges below "Admin" to open GHSA

This is an issue that's coming up in many of my user conversations and is something that requires change. I'm not committed to exactly what that change is at the moment, but I've got a process started for it and will be exploring many options.

Discoverability of an individual GH users open/disclosed/closed GHSA

This is definitely an interesting idea as we explore how to make our platform the primary place for researchers to operate. I'll document this ask so it's in our system.

GHSA Comments are Locked for EVERYONE After Disclosure

@JLLeitschuh is the need on this one resolved by community contributions? In that case, you'd open a contribution for change, the maintainer is alerted if interested, and you can have an open back-and-forth on the PR about this issue. That said, it will change on the global advisory level instead of the repo-level, but that's the one most people view statistically.

Let me know your thoughts on whether this one is partially or fully resolved.

GHSA Should Display Change History

This one I believe is resolve, at least at the global advisory level. If the advisory has changed at all (by our doing or a community contribution), you can click the link below the community contribution link to see all changes:

Display Inline Code Blocks Consistently across GHSA

Creating a ticket to look into this one, feels like something we could fix. Thanks for raising, this was not on my radar.

When a user watches only "Security alerts" for a repository, shouldn't that user get a notification from GitHub as soon a Security Advisory is published for that repo? Right now no notification will be send at all.

This one is from @mkurz, I'll create an issue for this and circle back if there are problems with fixing it. That aligns with my expectation of that feature (as a new-ish PM here) as well!

Thanks again to everyone for chiming in, please keep the feedback coming– it's how we grow.

Kate Catin
Senior Product Manager
Advisory Database Team

[StayPirate]

Hi @KateCatlin, do you have any update to share?

[KateCatlin]

Hey all! Sorry for the delay. Here's an update on each of the issues mentioned above...

Allow Security Researchers to Open GHSA without Requiring a Maintainer to be Involved

You can now accomplish this through Private Vulnerability Reporting! However, the maintainer's involvement is still required to publish.
Unfortunately, not all repositories have turned on private vulnerability reporting yet, but we are already at >16k repositories with it enabled, 2 months after launch and are planning more marketing to promote it further.

Allow Repository privileges below "Admin" to open GHSA

Again available via private vulnerability reporting, see above!

Discoverability of an individual GH users open/disclosed/closed GHSA

No update here, wishing for spare cycles for this but haven't found them yet.

GHSA Comments are Locked for EVERYONE After Disclosure

No plans for this one, though comment below if this specifically is causing problems for you.

GHSA Should Display Change History

No further plans for this one either, though we are adding more details about where it was published, when it was updated, etc. to the timeline at the bottom of each global advisory.

Display Inline Code Blocks Consistently across GHSA

No update here.

When a user watches only "Security alerts" for a repository, shouldn't that user get a notification from GitHub as soon a Security Advisory is published for that repo? Right now no notification will be send at all.

This broken feature led to a discussion about alerting on repo advisories overall. Thus, we are working right now on a new API that will allow programmatic tracking of advisories on a per-repository basis, along with some other repository-level advisory features.

Am I correct in assuming an API-level approach would be more helpful than a UI button fo this?

[mkurz]

Am I correct in assuming an API-level approach would be more helpful than a UI button fo this?

Actually not really. Please look at the screenshot I posted above. There already is the UI "button" (actually checkbox) to subscribe to security alerts, which right now is useless. What will happen with that checkbox? Will it still be there?
Actually I would love if you keep that checkbox for one reason: Right now we have a security mailing list (google groups) where people can sign up to get notified about security problems in your project. So this mailing list ist used for nothing else, so there are like 3 to 4 posts per year were we publish security related announcements.
So now we move more and more to GitHub, meaning we had a Discourse forum in the past, but using GitHub discussions now. Same with CI, we used TravisCI and moved to GitHub actions last year. We would love to host everything on just one plattform (GitHub) which makes maintance much more easier and avoids user need to sign up to different services.
Now to replace the security mailing list we need a way that people (also non-devs which usually don't have a GitHub account) get notified about security alerts. So if the UI checkbox that already exists would work like we would expect it to (notifies you when a new advisory gets published) we would just tell poeple to sign up to GitHub and subscribe to that checkbox, done.

So as you can see IMHO there is a need for a UI button/checkbox which just notifies you when a security advisory gets published.

Actually it's not just that use case, personally I would also use that checkbox for other projects, where I am not interested in watching releases nor other development, but security alerts only, so I am aware when I should upgrade that library in our framework.

Thanks for listening!

[StayPirate]

Actually it's not just that use case, personally I would also use that checkbox for other projects, where I am not interested in watching releases nor other development, but security alerts only, so I am aware when I should upgrade that library in our framework.

That specific use-case also apply for me, because I do use GH notifications 👍🏼. So, please fix and keep the Security Alert "button/checkbox".

Thus, we are working right now on a new API that will allow programmatic tracking of advisories on a per-repository basis

Nevertheless, I think that a dedicated API endpoint will also be very useful for different use-cases. For instance, I use a tool which scrape the internet looking for new known vulnerabilities for a curated list of software. Having such an API endpoint will definitely help to integrate GHSAs to my (and probably other) tool.

Moreover, if that endpoint is intended to be used with unauthenticated requests (no user token required) and will return a JSON-feed formatted data, then it will become pretty easy for people who don't have a GH account to follow that stream via their preferred RSS reader.

Now to replace the security mailing list we need a way that people (also non-devs which usually don't have a GitHub account) get notified about security alerts. So if the UI checkbox that already exists would work like we would expect it to (notifies you when a new advisory gets published) we would just tell poeple to sign up to GitHub and subscribe to that checkbox, done.

If the above will turn to be true, you could use a simple tool like rss2email to keep offering SA via your ML.

[mkurz]

If the above will turn to be true, you could use a simple tool like rss2email to keep offering SA via your ML.

I don't want to maintain tools myself (anymore). I just want users to sign up to GitHub and subscribe. Easy.

[StayPirate]

Hi @KateCatlin, glad to see how active you and your team are among the GH community :).
I'd like to share my feedback on GHSA with you. Like many others I'm a big consumer of RSS/Atom/Json feeds, and GH is amazing for that: I can just add .atom to many URLs to easily get a feed for that page.

For instance:

• to follow a user activities: https://github.com/staypirate.atom
• software releases: https://github.com/StayPirate/sieve-susede/releases.atom
• tags: https://github.com/StayPirate/sieve-susede/tags.atom
• commits: https://github.com/StayPirate/sieve-susede/commits/master.atom
• etc.

Unfortunately, that's not true for Security Advisories. And IMO this is quite an important resource that should be easily monitored in multiple ways. Let's take the Nextcloud project as an example, they stop maintaining their own SA RSS-feed to switch to GHSA instead (https://github.com/nextcloud/security-advisories/security/advisories), but now I can no longer ingest these data into my scripts anymore. Everyday more and more open source project start using GHSA, which is great, but the risk is that security analysts may lose visibility on SA.
It would be amazing if I could just append the .atom extension to the URL in order to get the Atom feed, like: https://github.com/nextcloud/security-advisories/security/advisories.atom.

I work in the security team of a widely used downstream Linux distribution, and being able to get these information in a non-human readable format is very important for us, RSS/Atom/Json would be a good fit. I'm sure many other users will benefit from this as well.
There is a related discussion at #7517 (comment).

[KateCatlin]

Hey @StayPirate! Thanks for reaching out. This was a great read and highly constructive feedback. Can we set up a time to chat? I haven't dug into the RSS/atom feed user journeys yet and I'd like to learn more. We'd be happy to share a gift card as a thank you for your time: https://calendly.com/security-advisories-ux-calls/25min

[dwisiswant0]

Hi, why my security advisory credit didn't update? It just showed 1 credit, but when I click-thru, it has 2 there.

[KateCatlin]

Hi @dwisiswant0, can you link me to where you're seeing this?

[dwisiswant0]

Right in the highlight of my profile, it says "1", when I click to view my credits — it shows more than one.

[KateCatlin]

Looking into it...

[lieser]

Feature Requests: pull request created in the temporary private fork for the GHSA should still be visible after it is merged and published.

We recently worked for the first time with the GHSA and there surprised that we could no longer view the merged and closed pull request created in the temporary private fork.
I think it is unfortunate that the PR description and the discussions are lost. Especially since we planned to continue to work on the closed PR after the GHSA is published.

[KateCatlin]

Hey @lieser can you tell me more about why this is important to you? Was there something specific in the PR discussion that you wanted to go back to and reassess?

[lieser]

The biggest problem here was that this loose of information came as a surprise for us. If we had known it before we published the GHSA we could have prepared for it.
So IMO the first and most important thing here is to properly make people aware of this. If there is already a warning about this somewhere and we just overlooked it, maybe consider making it more visible.

Was there something specific in the PR discussion that you wanted to go back to and reassess?

For the PR in the GHSA that got merged:
Especially the PR description, but sometimes also the comments and discussions can help better understand why something was changed. So keeping them can help others better understand the change, but it can also be useful for oneself if e.g. a year later one has to again understand why a change was done in a certain way.

For the PR in the GHSA that we closed:
There we started the work on fixing this not just for the latest release (this was done in the merged PR), but also for the next major version still under development.
The PR hat a bigger description and we also already did some code review with discussions there, but the PR was not yet finished. But we also did not want the work here to delay releasing a fix for the latest release version.
So we closed the PR, published the GHSA, and then wanted to continue the work on the closed PR. Either by recreating it from the information in the closed PR (by coping the description and open discussions) or by simply reopen it if possible.

In both cases we could have worked around it if we know about that all this information is lost. Although not the best experience IMO it still would have been OK.
And in our case we also got lucky because I still had the closed PR in an open an cached tab there I copied the description and discussions out of after a coworker noticed he could no longer see the closed PR. But if not we would have had to rewrite the longer PR description and hope that we still remember the open discussion.

[ThiefMaster]

IMHO it would be nice if there was an option to migrate the PR to the public repo after publishing the advisory. My project's changelog references the PRs for all changes, but in case of a security fix I have no public PR which I could reference there...

[lieser]

Feature Requests: optionally allow workflows to run in the private fork for the GHSA.

Although probably a good idea to disable the workflows by default, it would be nice if one can still run the workflows if explicitly requested.
Then we there recently working on a GHSA, for us it would have been fine if the runners provided by GitHub would have seen the fixed code before the GHSA got published. And it would have been helpful to easily run all the test and code analyzes we have in the CI.

The best thing would be if each job in a workflow could be independently marked as trusted to run in the the GHSA.

[KateCatlin]

Coming [waves hands vaguely] soon :)

The best thing would be if each job in a workflow could be independently marked as trusted to run in the the GHSA.

Can you tell me more about this part? Why do you want that? And would you still use Actions in GHSAs without it?

[lieser]

Why do you want [to exclude certain jobs from running in the GHSA]?

There may be some jobs that also publish the code somewhere public and are run for all branches not just master. E.g. some external service to visualize the coverage data. This are jobs most would probably not want to run for GHSAs, even if they want to enable the workflows in general.

I could imagine that it is already possible to write some exclude rule that would achieve this. So maybe just writing a helpful example in the documentation for GHSA on how one could exclude a certain job would already be enough here.

I think both the following would work:

1. Allow enabling all jobs in general for the GHSA somewhere and then needing to somehow excluding certain jobs from running.
2. Having to explicitly enable each job by marking it as trusted.

(1) is probably less work in most cases. The advantage of (2) would be to make it a little harder to enable problematic jobs by accident in the GHSA. But don't now if that is worth the additional effort.

And would you still use Actions in GHSAs without it?

Yes I would still use them, and try to come up with a way to ensure certain jobs are not run.

[feanil]

Feature Request: Be able to give more people access to all advisories in a repo.

Right now only the org security team or repo admin can see all the security advisories on a repo.

Either allow those with write access to see advisories similar to what's happening with dependabot alerts. Or be able to grant permission to advisories to specific people or teams like you currently do with access to alerts:

[KateCatlin]

Hey @feanil this is an active conversation for us! We're very interested in updating our permissions model. Can we take some time to chat and learn more about your use case? You can book time with me here: https://calendly.com/security-advisories-ux-calls/30min?month=2023-01

[ScriptAutomate]

Related feedback, to extend the discussions that have happened here:

• GitHub Discussion: Extend APIs related to GitHub Security Advisories and their Temporary Private Forks
• GitHub Discussion: Is there a way to run GitHub Actions against a temporary private fork created from a Security Advisory?
• GitHub Discussion: Deleting a temporary private fork of a repository after addressing a security advisory

[gtjoseph]

The Asterisk team just did its first security release using Security Advisories back in December and it was neither fun nor pretty. The reasons have been mentioned before but let me give you a bit of history so you know why they're important.

Asterisk has been around for 24 years so we've got lots of history, not just in terms of commits but also baggage. We moved our entire delivery pipeline from self hosted Gerrit/Jenkins to Github only this past May. We're currently maintaining 4 active release branches some of which receive new features but all of which receive bug fixes and security fixes (it's one of those "baggage" things we can't change). This means that a typical bug fix starts its life in the master branch and gets cherry-picked to release branches. Since Github doesn't have the ability to cherry-pick and we don't want to burden our contributors or staff with having to manage multiple PRs for the same change, we have workflows that pre-test a PR against all release branches by testing for a clean cherry-pick and running our over 600 test functional testsuite. When the PR is approved, it's merged into the master branch and automatically cherry-picked to the other branches. Can you tell where I'm going with this? :)

OK so fast forward to mid-December. We had 4 security advisories, each with their own private fork, to go into 4 releases. Now bear in mind, our normal release process consists of filling out a workflow input form consisting of 1 string field for the new version and a few checkboxes. It took us all day to just get to the point where we could fill out that form. Why?...

• No workflow in private forks. Not being able to check that a PR will even cherry-pick cleanly to all branches, let alone pass the functional tests before making it public, makes the private forks as currently implemented useless for us.
• Not being able to "copy" a PR from the private fork to the parent repo is also a bit of a pain.
• No easy way to update the private fork from the parent repo.

Here's what we had to do...

• Locally create a new clone of the private forks' parent repo.
• Use gh pr checkout to checkout the PR's commit from the 4 private forks into separate branches in the clone.
• Manually test each commit branch against the release branches.
• Use gh pr create in each commit branch to create 4 new PRs in the parent repo's master branch.
• Let the existing PR workflow do its thing and cherry-pick to the release branches.
• When everything merged, attempt to publish the advisories, which failed because even though the commits actually merged into parent repo, you still have to press the "Merge" button before the "Publish" button becomes active.
• Pressed the "Merge" button for the first advisory and was dismayed to find that it created a no-op merge commit which had to be deleted (we don't use merge commits).
• Discovered that if you delete the private fork, the "Publish" button becomes active without needing to "merge".
• Deleted the other 3 private forks and published the advisories.
• Finally, kicked off the release workflows.

In hindsight, we probably could have created 1 PR that had the 4 advisory commits in it instead of 1 PR for each commit but we didn't think of it at the time.

What we're going to have to do going forward with the Security Advisory feature in its present state....

• Create a manually kicked off workflow that, given a ghsa_id, creates a new repo with that name, makes it private, adds the public repo as a remote, fetches the branches, and finally kicks off a workflow in the private repo that adds the advisory reporter as a collaborator and adds a comment to the advisory with the name of the private repo.
• Create a workflow for the private repo that periodically updates the local branches from the parent repo.
• Use the normal PR workflow in the private repo to test any PRs, then, when approved, cherry-pick them to the appropriate release branches (still in the private repo).
• Create a workflow to be manually kicked off in the private repo that, at release time, will take the commits that were previously cherry-picked and push them directly to the release branches in the public repo, bypassing PRs and workflows.

Not fun.

[ThiefMaster]

Even on a smaller project with way less automation I found not being able to use CI very annoying.

Not being able to specify the target branch and merge type (squash/rebase/mergecommit) wasn't great either. Since in my case it was a very minor fix I ended up discarding the private fork again and just creating a normal PR in the end...

[KateCatlin]

Thank you all for all this feedback! We have been wanting to get CI running on private forks for quite some time and I apologize that we haven't gotten to it yet. I will circle back here to chat with both of you about it for user experience design when we're ready to pick this work up!

[gtjoseph]

@KateCatlin Any progress on this issue? Just for reference, hare's an example of what we have to go through to create a usable private "fork"...

CopyRepoToPrivate.yml.txt

[KateCatlin]

Thanks for reaching back out @gtjoseph. I know this is frustrating to hear, but this work is gated behind a large and highly-important project that we're focusing all resources on for now, and it will be some time before we're able to get to this CI support work. It is still on our minds and our roadmap though ❤️

[edemaine]

@gtjoseph Thanks for outlining the workarounds you made. I just went through 4 security advisories with 4 private forks, too, on the KaTeX project. I ran into similar issues because we have CI that automatically releases upon push to main, and I wanted all four security patches to be in a single release. Normally I'd merge PRs into a temporary release branch, but this doesn't seem to be possible: the private-fork PRs could target a different branch, but only branches in the temporary fork, not any newly created branches in the parent repository. 😦

Inspired by your workaround, I did the following slightly different steps:

• git remote add and git fetch for each private fork
• git merge --squash the private-fork branches corresponding to the PRs (--squash because that's our merge policy, also not an option with the private-fork GitHub interface sadly)
• git push these commits all at once to main (triggering release)
• Close each PR (didn't have to fully delete the fork, as you did)
• Publish the report

I also find it particularly weird that the private forks get deleted when publishing the report. As a result, all private discussions and review around the PRs is lost. This seems like a major flaw in the current system. I can understand wanting to archive the private forks, but why do they need to be deleted? Overall it would seem much cleaner to have private PRs on the main repository...