Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate releases #401

Closed
wants to merge 54 commits into from
Closed

Automate releases #401

wants to merge 54 commits into from

Conversation

tie
Copy link
Contributor

@tie tie commented May 14, 2021

This PR automates publishing new releases.

To publish a new release, just git push release. The new version is automatically inferred based on API changes.
Alternatively, it’s possible to override this behavior (e.g. if we have breaking changes in cmd/genji but not in Go API) by pushing to release-v0.13.0 branch, or manually dispatching a workflow with version inputs.

The workflow then creates version bump commits for Genji’s submodules and tags them. In the end it creates a draft release with changelog (additionally using CHANGELOG.md if it exists). When GitHub release is published, another CI workflow kicks in that builds and uploads binaries as release assets.

Additionally, version bump commits are only reachable from tags so it should be safe to dispatch the workflow on main branch.

That said, this should also eliminate the need to manually manage unreleased versions in go.mod.

workflow

Build matrix
Build matrix

Closes #269

Copy link
Contributor Author

@tie tie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, adding a few typo fixes as review suggestions so that I can apply them from UI.

.github/workflows/release.yaml Outdated Show resolved Hide resolved
.github/workflows/release-assets.yaml Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented May 14, 2021

Codecov Report

Merging #401 (36ce470) into main (91580d6) will decrease coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #401      +/-   ##
==========================================
- Coverage   67.82%   67.82%   -0.01%     
==========================================
  Files          75       75              
  Lines        7777     7786       +9     
==========================================
+ Hits         5275     5281       +6     
- Misses       1819     1821       +2     
- Partials      683      684       +1     
Impacted Files Coverage Δ
document/scan.go 56.46% <0.00%> (+0.41%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91580d6...36ce470. Read the comment docs.

tie added 7 commits May 14, 2021 15:52
Also fixes go install since we had replace directives in version bump
commit. Those are now in a separate commit that moves module back to
development mode.

And also makes versions a bit more robust by passing them as env vars.
This allows us to properly escape them in shell commands.
Go get, uh, gets confused if go.mod contains non-existent version.
That’s the case with current go.mod file.
@tie
Copy link
Contributor Author

tie commented May 14, 2021

Testing release.yaml workflow now by changing the import path and creating a new release in a separate repository.

find . \( -type d -name .git -prune \) -o -type f -print0 | xargs -0 sed -i 's:genjidb/genji:tie/genji-release-test:g'
go mod tidy
(cd engine/badgerengine && go mod tidy)
(cd cmd/genji && go mod tidy)

@tie tie marked this pull request as ready for review May 14, 2021 19:36
@tie tie marked this pull request as draft May 14, 2021 22:03
@tie
Copy link
Contributor Author

tie commented May 14, 2021

Looks like gorelease silently ignores replace directive. I’ll add a rollback strategy and run gorelease for badgerengine in a separate job after tagging root module.

@tie tie marked this pull request as ready for review May 15, 2021 02:11
@tie
Copy link
Contributor Author

tie commented May 15, 2021

@asdine should be ready for review.

@asdine
Copy link
Collaborator

asdine commented May 15, 2021

Thanks @tie ! I will review ASAP

Copy link
Collaborator

@asdine asdine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tie ! Sorry for the delay

I don't know much about Github Actions so I don't have much to say about the manifests and I trust your expertise. (cc @jhchabran perhaps you know more than me, if you want to have a look)
@tie could you add a README explaining how to create a release? Otherwise I'm sure I'll forget the minute I merge the PR 😆

@gedw99
Copy link

gedw99 commented May 23, 2021

This looks good !

would you be able to add the tinygo wasm target into the matrix ?

@jhchabran
Copy link
Collaborator

jhchabran commented May 24, 2021

@tie That's a lot of good work you poured into making this work! And those things are not fun to debug either, thank you so much for taking the time to make it work.


I experimented with the process and it works really nicely. AFAIU, with this PR the development process becomes the following:

  1. Classic PR and merge onto the main branch, as usual
  2. When it's time to release, git push origin main:release to trigger the release workflow
  3. This opens a PR from release-vX.Y.Z to release where we can review the updates to all go.mod (and go.sum files).
  4. We turn the release draft that was automatically created, which triggers the release assets workflow
  5. The build matrix runs and attaches the assets
  6. Drop the release branch?

Additionally, version bump commits are only reachable from tags so it should be safe to dispatch the workflow on main branch.

So that means, we'd be stuck with https://github.com/genjidb/genji/pull/401/files#diff-f94dd260d5f8f1b19caefcdfefe9ad1ab0cdfc040cd655cc82367b6950da9a0dR9 "forever" unless we rebase the release branch onto `main.

➡️ I don't think that's an issue, but I just wanted to be 100% sure I understand how we should handle the release branch. Could you confirm or correct me here?


Short feedback on minor things:

➡️ Optionally, what about zipping the binaries assets from the build matrix before uploading them on GH?

➡️ Could you squash the commits before we merge this PR? There are 54 of them and most of them are internal iterarations that do not bring any value to be kept as is in the commit history. Don't bother, there's the squash and merge button.

PS: thanks for taking the time to write this comment, it saved me some time 😊 #401 (comment) !

@jhchabran
Copy link
Collaborator

@tie Zipping things aside which is not important, could you just confirm what is the expected flow with the release branch?

@jhchabran
Copy link
Collaborator

jhchabran commented Jul 13, 2021

After discussing with @asdine, we have decided to not merge this pull-request. This decision has nothing to do with the quality of your contribution, which is absolutely great!

➡️ What motivates our decision is that at this stage, Genji is still small and we aren't releasing often enough to justify adding an automated release mechanism and having to own that particular code.

I'll keep a bookmark to this PR because it's a great example of what can be achieved with Github actions. I hope this come out as not too disappointing due to how much energy was invested here.

@jhchabran jhchabran closed this Jul 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automatically add go binaries to the release
5 participants