Proposal: CI Cleanup

[tmdzk]

CI Cleanup

Problem

We are currently using Github Action as our main CI component, GHA UI is definitely not the best and It can be messy, It's very important to have a clean and efficient CI as an open source project, when external contributor open a PR you can see 102 jobs being attached to your pull request. On top of that the Github Action UI can be overwhelming. The proposal is about cleaning all jobs, decommissioning all the unrelevant/unused jobs.

Solution

Multiple things can be done:

• Move all related jobs under the same workflow to avoid having 25 workflows for 1 component.
• Bring back the magma CI dashboard that had all the useful information
• Decommission all the job that are not used anymore
• Merge jobs that could be run at the same time

Non-goals

None

[lucasgonze]

A security problem with centralizing workflows is that it becomes hard to reason about the security properties of permissions needed for each one. However there could be a smaller number of workflows, where they are organized by permissions.

Decommissioning and merging is a great idea.

I have no opinion on the dashboard.

[jordanvrtanoski]

I am fully aligned with the proposal, @panyogesh can you please evaluate it as well. If ok, let's have budget for it and let's bring it on TSC for vote.
The one workflow will be organized in multiple steps, let's verify if the security can be controlled on the steps

[panyogesh]

+1, Proposal looks fine.

[lucasgonze]

@tmdzk Are you asking for budget for this, or is it within your current funding?

[tmdzk]

It's within current funding

[lucasgonze]

@tmdzk Can you provide an update on the state of this?

Update later: Tim says he is at 25%.