Can you share tasks efficiently across multiple git repos?

[fredrikaverpil]

Let's say you have a bunch of git repos in your organization. In each, you can add a Taskfile.yml with commands for the individual project. But let's say that for all these repos there is a common task that ideally would be defined in one central place (not in each of all the repos).

Is it possible to set up git repo for such shared tasks and somehow have them all available when running task --list along with any project-specific tasks in the current project directory?

I saw there are global tasks, but this assumes you place the shared Taskfile.yml on each developer machine which is not ideal. But this could perhaps work if this local/global Taskfile.yml includes a update-shared-tasks task which pulls down all tasks from the central git repo. It doesn't seem like it would automatically bring in any tasks from a Taskfile.yml in the current working directory as well though.

Any input or ideas on this problem is much appreciated!

EDIT: it looks like something like this is work in progress over here?

[andreynering]

Hi @fredrikaverpil,

Yes, remote Taskfiles is what you're looking for. For now it is an "experiment", which means you need to enable it with an ENV and it is subject to breaking changes until launched for real.

Given you're interested, it'd be awesome if you gave it try and added your feedback on #1317.

https://taskfile.dev/experiments/remote-taskfiles/

[andreynering]

@pd93 I actually think that supporting ENVs like TASK_GITHUB_TOKEN and TASK_GITLAB_TOKEN, etc would be a great idea.

It'd allow us to support private repos in a simple way without having to implement Git support necessarily. And it's secure, because each user would have to set this token on their own machine.

What do you think?

[pd93]

I'm not against supporting environment variables. It's definitely easier/quicker to implement than git/ssh. Though also less flexible. I think either way, we will have to support git/ssh in the long run. For example, some private repos have https disabled entirely. Do we want different env vars per upstream or a single, generic one and domain detection?

[fredrikaverpil]

It'd allow us to support private repos in a simple way without having to implement Git support necessarily.

I would say there's likely a demand for ssh too. But starting out by reading a $TOKEN environment variable would be quicker to implement, like you say, and enable some people.

Depending on how this is implemented, git's insteadOf could perhaps be used to enable ssh use. But I'm not really sure if it would cover the true ssh story.

# warning, this changes the local git config

# use https for git
git config --global url."https://github.com/".insteadOf [email protected]:
git config --global url."https://".insteadOf git://

# use git for https
git config --global url."[email protected]:".insteadOf https://github.com/
git config --global url."git://".insteadOf https://

And it's secure, because each user would have to set this token on their own machine.

You also have the use case where the Taskfile is included from a private repo and the tasks need to run in e.g. GitHub Actions. In this case, I'm not really sure what the best way would be for Task (and the includes declaration) to understand the secret.

[andreynering]

I think both makes sense, indeed. I'd go for ENVs over HTTPS first, and consider adding support for SSH afterwards.

And just a reminder: Go has a native Git implementation, which is probably the way to go here, to avoid depending on local tools. Needs to check if they have proper support for the SSH keys the user has alredy setup, of course.

https://github.com/go-git/go-git

[fredrikaverpil]

Here's my full use case:

Personal tasks

• I'm using ~/Taskfile.yml for personal needs: https://github.com/fredrikaverpil/dotfiles/blob/main/Taskfile.yml
• I'm cloning down a private github repo of mine to local disk, in which I store sensitive tasks.
• This use case does not need any authentication for the includes directive as I can include from a local path.

Tasks at my company

• Tasks are defined in private git repos.
• Some git repos might benefit from sharing tasks. In this case, authentication is required for the includes directive.
• Tasks needs to run locally, which means e.g. defining the $GITHUB_TOKEN in the includes directive.
• Tasks needs to run in GitHub Actions. In this case, GitHub Actions needs to somehow pass secrets ({{ secrets.MY_SECRET }}) to the task command such that the includes directive can pick up on it and use it.
• Git repos may be including tasks from other repos in a nested fashion. Nesting seems to work out of the box, but again here authentication is required for the includes directive.