Implement CI_JOB_TOKEN, so private includes can be accessed

[cs35-owncloud]

Hello,

sorry if I created this issue as a bug as it may be more of a feature request.
I've read the documentation, mostly the quirks section and this commit

Minimal .gitlab-ci.yml illustrating the issue

---
include:
  - project: 'ansible_hosting/ansible_config'
    file: '/ci/gitlab-ci-templates/.gitlab-ci.yml'
    ref: master

In the variables I specified a gitlab token:

~/.gitlab-ci-local/variables.yml

---
global:
  ANSIBLE_HOSTING_DEV_RO_USER: ro_token
  ANSIBLE_HOSTING_DEV_RO_TOKEN: xxxx

I'm using the above .gitlab-ci.yml on several projects to avoid rewriting the core ci in every projects.

Expected behavior
This should fetch the /ci/gitlab-ci-templates/.gitlab-ci.yml of the project ansible_hosting/ansible_config
It seems that gitlab-ci-local find the correct URL (probably because my current project come from the same gitlab URL) but it cannot use the token and prompts for a password:

[email protected]'s password: 

Is it possible to use this feature with gitlab's token ?

I also tried this version:

Minimal .gitlab-ci.yml illustrating the issue

---
include:
  - remote: 'https://gitlab.xxx.com/api/v4/projects/xxxx/repository/files/ci%2Fgitlab-ci-templates%2F.gitlab-ci%2Eyml/raw'

It doesn't work either and this time I get this error:

Remote include could not be fetched https://gitlab.xxx.com/api/v4/projects/xxx/repository/files/ci%2Fgitlab-ci-templates%2F.gitlab-ci%2Eyml/raw AxiosError: Request failed with status code 404

If I try to download the file with curl and the above token, it works.

curl  --location --header "PRIVATE-TOKEN: xxx" https://gitlab.xxx.com/api/v4/projects/xxx/repository/files/ci%2Fgitlab-ci-templates%2F.gitlab-ci%2Eyml/raw

If I create a public snippet on my gitlab, the include remote will work but it's not really a suitable solution for me.
Is there any other way so I can grab my ci file remotely with gitlab-ci-local ?
Thanks.

Host information
Ubuntu
gitlab-ci-local 4.38.1

Containerd binary
Docker version 24.0.5, build ced0996l.

Thanks !

[firecow]

I wonder how Gitlab CI knows that the remote include, which is basicly just a curl request needs a gitlab private-token.

What if the url was not gitlab.<company>.com, but pkg.example.com

They must be doing some sort of matching to find that the URL contains subdomain.domain matching the gitlab instance.

[cs35-owncloud]

Thanks for your fast answer, in my case I have a self hosted gitlab.
I guess it's checking the remote (git remote -v) and simply take the same domain to fetch.
That's a good question about the private-token 🤔
Maybe since it's in the same "context" and going through a runner that's already "logged in" it's bypassing the auth.
I will try to reach a ci file from an unauthorized project.

[cs35-owncloud]

OK that's weird.
So I created a project in a specific namespace with a private visibility.
In my other project on a different namespace, I defined the .gitlab-ci.yml to include the .gitlab-ci.yml from the first project.
And it works, the other project's ci got injected and started to run.

[firecow]

Well, if your user has access to both projects, I'm guessing the pipeline is generating a tmp token impersonating your access permissions, so you need to include a "ci-module" that your user don't have access to.

[cs35-owncloud]

OK I checked and it's because my user has admin access, I would have expected that the permissions were about the projects and not the users when you run the pipeline.

[bcouetil]

Indeed the access (or not) to remote include/trigger depends exclusively on the person launching the pipeline 😉

[firecow]

We need to implement a similar token feature that requires users to put a CI_JOB_TOKEN in their gcl variables.

This token will then be used by include fetching if git remote "matches" include url.

[cs35-owncloud]

It would be the best :)
Getting the base gitlab URL in this case and clone it would be the dream:

---
include:
  - project: 'ansible_hosting/ansible_config'
    file: '/ci/gitlab-ci-templates/.gitlab-ci.yml'
    ref: master