Should we look at github stars as an acceptance criteria?

[phanirithvij]

There are many PRs which add projects with no end users for that project yet. (basically 0 stars and using awesome-go as a way to advertise the project)

• I feel like we should set some sort of base limit for the number of users who use a project. For e.g. say project should have at least 10 stars given the first commit was 5 months ago. (arbitrary)
• Is the number of people using the project as a library the alternative? for eg. minio
• Should the project have existed for atleast a certain amount of time, say for eg. 5 months? (arbitrary)

Potential issues/questions that arise with using stars as a metric:

• Github stars can be bought online in bulk.

• Gitlab, etc. other hosted git projects will have very litte amount of "stars" but they will be used by many people who don't star the project.

• A star simply indicates that it's a bookmark and nothing else these days, but a bookmark nontheless is a person choosing to bookmark it so it is one step closer to being awesome than a project which no one bookmarked yet.
  - Unless of course botted stars in which case manual rejection of the project can be done when reviewing or removal of the project can be done once it is proved beyond reasonable doubt that it has fraudulent activity.

• I am aware of the discovery aspect of awesome-go and that in turn leading to "stars" to the projects listed in awesome-go.
  - But this is not a place where we can keep on piling up many toy projects which in turn bloat the readme file and make it so that it won't even open in the browser anymore.

• I know this is controvertial but I'd say it's worthy of discussion.

• If we need a limit what should it be?
  - There is a stable v1.0.0 requirement for a project to be added to awesome-go and this doesn't help much when the very first release the projects makes is marked v1.0.0.

Should personal toy projects be added to awesome-go? (these are the most projects which correlate to low amount of "stars")

I would appreciate if all the maintainers and anyone who comes across this post contribute to this discussion.
Feel free to shoot down any points anyone made with logic and facts and please be civil and follow the code of conduct.

All the above are my opinions and mine alone.

cc: maintainers @avelino @appleboy @cassiobotaro @panjf2000 @ceriath @jtemporal @felipeweb @kirillDanshin @joeybloggs @matrixik @dmitshur @dukex

PS: feel free to remove yourselves from the maintainer list if you don't want to be pinged, for any such discussions from now on.

[avelino]

I think it's always good to talk about this subject again, especially after a few years of discussing the issue below

#1931

I'm wary of using stars as a "vanity metric", there may be projects with no stars or few stars that are great projects, that add value to a user who is looking for a solution to their current problem.

My feeling about limiting the number of stars to a minimum is that we're starting to filter out projects that have the potential to solve real problems because we're looking at "vanity metrics".

[phanirithvij]

How would we go about reviewing these hidden gem projects? Many of the submissions do fall under 80% test coverage but soon after they are accepted to awesome-go they stop updating the project altogether, This #5136 is how unmanagable it has become.

[avelino]

that's sad, this issue made me rethink awesome-go a lot
I don't have an answer as to how to control test coverage without our own system controlling it and forcing all projects to use it, but we can't afford to maintain such a system nowadays.

what would be your suggestion?

[phanirithvij]

I could think of one rule which I think would not miss gems like the projects you mentioned. That is, 20+ stars and project should be atleast 5 months old. (Not sure how reasonable it is because it's kinda arbitrary.)
But I'd say if 20 people are interested in an un-advertised project in 5 months then it's worth it to review it. (Also as I mentioned above gitlab, etc where "stars" are hard to come by, I suggest proceeding with normal review process ignoring this star metric)
Also pkg.go.dev "imported by" would be a very good metric too if it is a library project. So that should have say 5+ importers as well for not reviewing projects that just started and want to be in awesome-go.

We'll say that project is too young and has low amount of interest and thus the project should be submitted again later based on the project's initial commit date.
In case we don't get the submission again because of various reasons then I suggest this,
We'll have a labels for "closed due to project age" and "closed due to low adoption" and review those projects again bi-monthly and look at the growth of those projects and review those.

Ofcourse code coverage should be looked at still and if unavailable and the project has significant no. of users the tests should atleast pass.
If project has no tests then it's hard to consider it for approval unless it's too famous (for e.g. projects falling under "software built in go" but author has no time to add tests). Maybe it's too useful a tool.

As for the current list and that issue, those need to be slowly reviewed one by one as we normally have done till now.

[avelino]

what worries me most about awesome-go is the amount of manual work and arbitrary decisions we have to make based on our prior knowledge

• this project is OK, even without testing
• this project is OK, even without ci
• this project is OK, ... etc

We need to improve and automate the automatic review to the point where it automatically closes the PR if it doesn't pass the basic points on our "quality" list - of course we need to automatically comment on why the PR is being closed, for example:

project has less than XX stars, we only accept project with at least XX stars - see our quality list here

this applies to all the rules we set as "minimum quality".

By automating, we can minimize the number of PRs we need to review. It makes me "desperate" to look at the number of PRs we have open today - I know we won't be able to look at them all.

[panjf2000]

I absolutely second the idea that we should raise the bar for incoming repositories to be adopted into awesome-go. After all, awesome-go is positioned as a curated list. It's unlikely for me to go through the source code in depth for every repository when reviewing the PR. And I doubt other maintainers have the time for that either. Therefore, we can all use some quantitative metrics to help review incoming repositories and GitHub Stars would be a solid candidate IMO, aside from the potential zombie stars.

Numerous GitHub Stars don't necessarily make an open-source project high-quality, but it could be a credible indicator. If we don't have much time to read all the source code of incoming repositories before we can tell if they are awesome enough to be in awesome-go, then let the community tell, I believe that people give stars to an open-source repository for a reason. Open-source repositories with GitHub Stars show that they have been proven.

The bottom line is that we need to include some quantitative metrics in our review process if we still care about the term "curated list" and don't want awesome-go to be the Times Square Billboards. Even if it's not GitHub Stars, there has to be something else.

[ceriath]

I do agree something needs to change here. There are a lot of dead projects as mentioned above and a lot of open PRs noone has the time to review manually.

So yes, we need to change the criteria and i agree that "awesome-go" should be a list of good projects and not all projects.

Instead of thinking about a list of requirements that all need to be met, maybe we could think of criteria that would validate a project, but make them not all a requirement. So let's say we find 5 criteria, but meeting 3 of them are enough to get on the list. Having X stars could be one of them, but it would not exclude hidden gems, when they meet 3 other criteria.

What do you think about that?

[phanirithvij]

This is the current process I follow for reviewing, I usually ignore low code coverage if it has 5000+ stars and is actively maintained. I usually ignore star count if it has proper docs, 80% coverage and is actively maintained, this is where the problem lies I think. Just because a project has good docs and code coverage doesn't mean in the long run it will be used by many people or it's a hidden gem. There are many such toy projects in the readme.