Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GHA pipeline rewrite for ease and speed #1551

Merged
merged 71 commits into from
Nov 17, 2023
Merged

GHA pipeline rewrite for ease and speed #1551

Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
943ca6a
Renmove unnecessary/old workflow files
gpmayorga Sep 15, 2023
9ea2c6f
Add PR common checks and wasm build(s)
gpmayorga Sep 15, 2023
298c307
Modify docs buld
gpmayorga Sep 15, 2023
06dc4ef
modify benchmark
gpmayorga Sep 15, 2023
77948d5
Modify CI script (simpler)
gpmayorga Sep 15, 2023
949dbeb
Add docker build
gpmayorga Sep 15, 2023
934145d
Add prep action for common Ubuntu steps.
gpmayorga Sep 15, 2023
3641a11
separate sccache gcloud action
gpmayorga Sep 15, 2023
98ca8f3
fix dockertag pattern
gpmayorga Sep 15, 2023
c2bc2cf
delete old benchmark check
gpmayorga Sep 15, 2023
79eff04
Trick the wasm publish for this branch
gpmayorga Sep 15, 2023
bb5713e
new benchmark pipeline
gpmayorga Sep 15, 2023
c6be052
additional cache options
gpmayorga Sep 15, 2023
309afdb
exclude runtime integration tests from dockerfile
gpmayorga Sep 15, 2023
c2b8fc7
fix buildwasm
gpmayorga Sep 15, 2023
53a21a0
fix docker build & fix benchmarks
gpmayorga Sep 15, 2023
f79e94b
remove old benchmark job
gpmayorga Sep 15, 2023
97fa94e
debug benchmark cache
gpmayorga Sep 15, 2023
6a4c523
fix wasm location upload
gpmayorga Sep 15, 2023
12292b0
better benchmark PR
gpmayorga Sep 15, 2023
9486c1e
small fixes
gpmayorga Sep 15, 2023
ed7ec02
fix typo on build-wasm
gpmayorga Sep 15, 2023
5200f38
better Docker build and management
gpmayorga Sep 15, 2023
d467440
small fixes
gpmayorga Sep 15, 2023
0bfbf7a
cleanup and concurrency review
gpmayorga Sep 15, 2023
9fddac4
simplify docker and fix bench-check cache
gpmayorga Sep 15, 2023
9ca1e9a
inject sccache in the docker image
gpmayorga Sep 16, 2023
61cbe48
fix benchmark write access for PRs
gpmayorga Sep 16, 2023
d72014a
test gha cache for docker
gpmayorga Sep 16, 2023
c9264d8
remove sccache from dockerfile
gpmayorga Sep 16, 2023
e23b574
adjust machine size and fix quotation in Dockfile
gpmayorga Sep 16, 2023
8087d93
try fix docker var quoting
gpmayorga Sep 16, 2023
3b1d5b3
pin all github action versions
gpmayorga Sep 16, 2023
06ec8e1
fix srtool build
gpmayorga Sep 16, 2023
0271fdb
Upload srtool report from wasm build
gpmayorga Sep 16, 2023
ec97693
add code coverage to wasm build
gpmayorga Sep 16, 2023
d4512b0
Merge branch 'main' into ci-rewrite-n-cache
gpmayorga Sep 16, 2023
8ed4eda
fix typo
gpmayorga Sep 16, 2023
b0b406c
add nix build (fully commented)
gpmayorga Sep 16, 2023
64d1822
Update build-wasm.yml
gpmayorga Sep 18, 2023
7868914
change build wasm docker run
gpmayorga Sep 18, 2023
fae3288
Move the code coverage up to see erros
gpmayorga Sep 18, 2023
c47131e
check if wasm builds
gpmayorga Sep 18, 2023
30a8468
try and force the report generation
gpmayorga Sep 18, 2023
b967a3a
Merge branch 'main' into ci-rewrite-n-cache
gpmayorga Sep 30, 2023
31660dd
review the PR with @wischli
gpmayorga Nov 7, 2023
8a9f774
Merge remote-tracking branch 'origin/main' into ci-rewrite-n-cache
gpmayorga Nov 7, 2023
eed117c
Try cargo install --locked
gpmayorga Nov 8, 2023
d03c379
Use toolchain from toml file
gpmayorga Nov 8, 2023
d51ee77
small fixes
gpmayorga Nov 8, 2023
31d1884
comment code coverage reports
gpmayorga Nov 8, 2023
7d486b7
trigger wasm build only on tag push
gpmayorga Nov 8, 2023
44012ad
Rename benchmarks job
gpmayorga Nov 8, 2023
2ce5707
try and fix the docker tag logic
gpmayorga Nov 8, 2023
780adbf
small fixes
gpmayorga Nov 8, 2023
f0f25af
more minor fixes
gpmayorga Nov 8, 2023
0d414a8
fix cache warning
gpmayorga Nov 8, 2023
2ce04c8
srtool has no latest tag on the repo, using 1.69
gpmayorga Nov 8, 2023
3eda725
remove production environment restrictions
gpmayorga Nov 8, 2023
aeaaedd
Fix GITHUB_ENV and bump srtool version
gpmayorga Nov 9, 2023
5a5d394
Merge remote-tracking branch 'origin/main' into ci-rewrite-n-cache
wischli Nov 13, 2023
8879d86
chore: update srtool version to latest
wischli Nov 13, 2023
61fcacb
fix: remove unused rust toolchain from setup
wischli Nov 13, 2023
9dfdcfb
fix: toolchain version docker build
wischli Nov 13, 2023
43e5d7a
fix: remove unused thread_local
wischli Nov 13, 2023
5177624
Update build-wasm.yml
gpmayorga Nov 14, 2023
8398d7a
Final review of CI changes
gpmayorga Nov 16, 2023
13e2085
review of sanity-checks
gpmayorga Nov 17, 2023
3e9b865
Final review of CI PR with @wischli
gpmayorga Nov 17, 2023
bfc67a5
Use Rust docker image for building the binary
gpmayorga Nov 17, 2023
2fac7af
Fix review comments
gpmayorga Nov 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions .github/workflows/sanity-checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
on:
# push:
# branches: [main, 'release-v**']
# branches: [main]
pull_request:
# paths:
# - 'src/'
# - '.github/workflows/sanity-checks.yml'
name: Sanity checks
concurrency:
group: 'tests-${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
Expand All @@ -15,7 +12,7 @@ permissions:
jobs:
test-n-lint:
name: ${{ matrix.target }}
runs-on: ubuntu-latest-4-cores
runs-on: ubuntu-latest-8-cores
strategy:
matrix:
target: [test-general, test-integration,
Expand All @@ -40,10 +37,10 @@ jobs:

benchmark-check:
name: bench-check-${{ matrix.runtime }}
runs-on: ubuntu-latest #r-4-cores
runs-on: ubuntu-latest #-4-cores
strategy:
matrix:
runtime: [development, altair, centrifuge]
runtime: [altair, centrifuge]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not development?

Copy link
Contributor

@wischli wischli Nov 17, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we didnt need to check dev benchmarks to save time. I know this kind of contradicts my argument of keeping benchmarks in Dev. However, in the past releases I merged Centrifuge benchmarks to Dev because there was no runtime benchmark pipeline. I think thats feasible because this way we mimic mainnet.

How about we iterate over that when we build the next feature, which will probably first only exist in Dev runtime?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this are PR checks, let's keep it minimal, this won't actually publish or set benchmarks anywhere, it's just to check that the benchmarks run
Do we really need all 3 to run?

steps:
- name: Check out code
uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab #3.5.2
Expand Down
Loading