Testing a New Stalebot in GitHub's Community Discussions

[queenofcorgis]

TL;DR - What is happening?

We are working to improve the quality of content across the GitHub Community Discussions. For the next 30 days we are going to perform a test using a custom action to label Question discussions that have not had any activity for 60 days as stale. The action will apply the label stale and comment to inform the original poster of the stale discussion. Sample draft copy is included at the bottom of this announcement.

📢 Feedback is requested on this initial test. Please comment on this post and let us know what you think!

What Discussions will be impacted?

• The action will label discussions that have been labeled Questions and have not had and activity in the past 60 days as stale.
• The action will post the sample copy (included below) as a comment on the discussion to alert the original poster, and request they take action to resolve the question. That automated reply should prompt users to return to add more information, select a reply as the solution, or close the question as out of date. I'll add some example copy we can use below.
• This will only be applied to discussions in the Copilot, Projects and Issues, and Accessibility categories as a part of this initial test.
• This action will not close the discussions automatically.

What is a Stalebot?

You can use GitHub Actions to comment on or close issues that have been inactive for a certain period of time.

Here is more info on this from the GitHub Docs: Closing inactive issues.

We're working with GitHub engineers to test this action in discussions!

Why are only limited categories impacted?

1. We have a hypothesis that using the action to close the loop on open and stale questions in the Community Discussions will improve the quality of content overall, but we want to test assumption before we risk disrupting the entire community discussions experience.
2. The initial test includes categories with high engagement, dedicated community managers, and a high level of involvement from GitHub engineers.
3. We are starting with an initial test with Discussions that are 60 days stale. We want to see how the age of posts impacts overall engagement and resolution rates. We track how long a discussion is open without a response and without an answer so we'll be able to track data on this test and use that to inform our next steps.
4. We want to hear from you! This is an experiment to benefit our users and the quality of content. If it doesn't work, we can roll it back. If it does, we can move forward and apply this across all categories.

What will happen after the trial period?

• The team will have an ongoing evaluation of how the stalebot impacts the response and resolution rates on exiting discussions.
• If we see positive indicators based on data that will be one indicator that this worked
• We will evaluate the feedback on this discussion. If you love it or hate it, we want to know! Please comment with ways you think we can improve the experience.
• If we deploy this broadly, we'll provide users with a hot-to guide here in the Community Discussions so you can apply a similar action to your own repo discussions 🚀

Example Stalebot Copy:

🕒 **Stale Discussion Alert** 🕒

This Discussion has been labeled as stale by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as "out of date" at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This stale notification will only apply to Discussions with the `Question` label. To learn more, see our recent stalebot announcement.

Thank you for helping bring this Discussion to a resolution! 💬

[ferdnyc]

@queenofcorgis Since you're showing the sample copy, presumably that's meant to invite input, so I have a suggestion:

Note: This stale notification will only apply to Discussions with the Question label. To learn more, see our recent stalebot announcement.

...The message could just link to the "recent stalebot announcement" (that'd be this discussion, #70478), instead of sending us on a scavenger hunt.

[Marcono1234]

Thanks for being so considerate by only limiting it to the "Question" category and not closing discussions directly.

The stale action (and other similar actions) are known to be quite disruptive in some cases, see actions/stale#719, when they close issues where the maintainer has simply not responded yet. Or in this case here, acting on questions (possibly even well written ones) where no one ever commented on.

As you mentioned, many of the questions here are answered by GitHub staff or at least someone from the community. But some also seem to have no answer at all (discussions with label stale and only a single comment). In that case only action 1, "Close as Out of Date", would be relevant.

Do you think that is a problem, and will you try to address that?

[jsoref]

@queenofcorgis sadly, this is in the announcements category, because I'd really like to see the action mark this discussion as stale.

I'd like to hear if you in fact ran this action and what the results were.

I personally hate the stale bot. (That I send PRs to it is not an endorsement, it's awful software for a lousy purpose.)

There appear to be two constructive pieces of feedback here:

1. If you're running an experiment, you should link to the announcement about the experiment: https://github.com/orgs/community/discussions/70478#discussioncomment-7338056
2. A critique (similar to mine) of the feature: https://github.com/orgs/community/discussions/70478#discussioncomment-7440236

There's also a PR relating to this discussion:
#69778

One problem with bots is that they make a mess of sorting (by last modified) by bubbling up stale items. Sure, it might breathe be life into a thing, but, for a system built around searching, that isn't worth doing. If someone's searching for a thing, they'll find or not find it based on their search and the text of the item.

A better approach would be to improve the UX flow so that authors are encouraged to yes/no answers as they arrive.

[jricciardi]

Hi @jsoref! I can weigh in a little bit here now that we have made some updates and have more data:

With the exception of a few edge case frustrations, we are seeing a lot of overall user acceptance for the stalebot, especially after reducing the "stale" language in favor of "inactive" and "dormant" language.

Additionally, and the numbers change daily, about 5-8% of Discussions are brought to resolution after prompting from the bot; about 1-2% are discovered to actually be product feedback that was incorrectly labeled as a question (this is a clear source of confusion and frustration for some users and something we're paying close attention to); and otherwise most Discussions never get new responses or activity.

A better approach would be to improve the UX flow so that authors are encouraged to yes/no answers as they arrive.

This is great feedback! If we find evidence this type of flow would improve outcomes for the Community, we may escalate this idea.

[jsoref]

I've tripped on the question category thing. The bot didn't explain how to recategorize my feedback.

[ant-techs]

Hi How can I talk to the creator/Dev of Zello as I have a proposition for him, if you could get in touch at [redacted] I would be grateful to talk with you, Best regards, Anton.

[queenofcorgis]

Hi @ant-techs ,

I've removed your private information from your comment as posting identifiable information is not allowed, per our Code of Conduct.

If you're asking how to use the piece of software that you found in a repository on GitHub, the best way to contact the maintainers of that software is to:

• Check the README for instructions on how to operate the software and pointers to documentation or troubleshooting info.
• Check the SUPPORT file, if one exists, for instructions on how to best contact the maintainers for support.
• Check the CONTRIBUTING guide, if one exists. Sometimes if there isn't a SUPPORT file, the CONTRIBUTING guide will give instructions on how to contact the maintainers for support.

All of these documents can be found, if they exist, in the repository where you found the software itself.

[ant-techs]

[ferdnyc]

@ant-techs

Then you should probably change your GitHub password, because 7 days ago your account posted a message to this discussion that started with, "Hi How can I talk to the creator/Dev of Zello as I have a proposition for him..."

[timmc-edx]

A bit of feedback: I'm here because the stalebot just replied to a question I've had open for some months about the GitHub API. It suggests three courses of action, all of which are frustrating to read:

• "Close as Out of Date": Nope, I still have the problem.
• "Provide More Information": Already gave all the info I have.
• "Mark a Reply as Answer": No one has answered, so there's no reply to mark.

The only useful action I could take would be something like "Do $X to get a GitHub employee to look at your question", but that's not one of the suggestions. :-) So the bot is just reminding me that there's a gap in GH's API that I can't do anything about, and there's no one to contact, and no one has responded... which doesn't feel great.

Anyway, I don't have a specific thing I think you should do differently (with regards to the bot at least) but I did want to share that the net effect was very annoying.

[jsoref]

I agree w/ your point about dynamic messages, surprisingly, they're using Ruby and not stale-bot, so they absolutely could check for things (they'd want to exclude comments from the author and from github-actions):

https://github.com/community/community/blob/main/.github/actions/comment

[jricciardi]

Hello @timmc-edx, we appreciate the feedback on this new Action we're testing out! That's a fair call out that we could have a smarter Action that checks to see if there are existing comments before suggesting 3️⃣. We will definitely consider this — especially if we see additional similar feedback.

One thing this Action does do is it bumps your Discussion to the top of the 'recent activity' for the given category, which can help get more eyes on your question and help question-labeled Discussions get answers from other Community members. Additionally, if this does indeed appear to be a gap in the API, you may bit better off submitting a product feedback Discussion in the APIs and Webhooks category, providing feedback detailing how your expectations of the API aren't being met so other users can upvote & generate more demand for a product solution. Other community members might not think to look through question Discussions for product feedback to add their support and upvotes.

I hope you're able to get an answer to your question soon! And thank you for flagging this experience with a detailed comment — again, the feedback is appreciated.

[timmc-edx]

Interesting that it bumps the activity -- that might be useful.

I wonder if one thing the bot could do in a no-replies situation is point to a document on how to ask better questions/get better answers. There are already documents like that floating around (be specific, give a summary up front, etc.). But given what you mention here and in the other thread, it might be worth prominently noting (there or in the message itself) that miscategorized posts are a big source of low response, and encouraging posters to review their choice of category.

[timmc-edx]

In my case, it started out as a "how do I do this" question. But now that I've gotten 4 votes and 0 replies over ~90 days, it's likely that this is actually an "I can't do this" product feedback. Not sure how that might fit into the Discussions workflow.

[jsoref]

I guess you could open a product feedback item and then answer your question by linking to the product feedback and then mark your link comment as the answer. -- If that's the desired path, then the bot should suggest it.