Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposed fix for ACR caching while providing guidance for GitLab users. #1628

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions binderhub/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,13 +204,19 @@ def _parse_www_authenticate_header(self, header):
) from None

async def _get_token(self, client, token_url, service, scope):
# specify required query parameters
query_params = {
"scope": scope,
}

# add optional query parameters
if service is not None:
query_params["service"] = service

auth_req = httpclient.HTTPRequest(
url_concat(
token_url,
{
"scope": scope,
"service": service,
},
query_params,
),
auth_username=self.username,
auth_password=self.password,
Expand Down Expand Up @@ -263,8 +269,8 @@ async def get_image_manifest(self, image, tag):
token = await self._get_token(
client,
self.token_url,
scope=f"repository:{image}:pull",
service="container_registry",
None, # service may be pre-set in self.token_url as per Helm chart docs for specific CRs
f"repository:{image}:pull",
)
req = httpclient.HTTPRequest(
url,
Expand Down
55 changes: 53 additions & 2 deletions binderhub/tests/test_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,15 @@ async def test_get_token():
test_handle = {"username": username, "password": password}
app = Application(
[
(r"/token", MockTokenHandler, {"test_handle": test_handle}),
(
r"/token",
MockTokenHandler,
{
"test_handle": test_handle,
"service": "service.1",
"scope": "scope.2",
},
),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change needed?

]
)
ip = "127.0.0.1"
Expand All @@ -196,6 +204,49 @@ async def test_get_token():
assert token == test_handle["token"]


async def test_get_token_with_config_supplied_service():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this is a copy of test_get_token apart from the arguments to registry._get_token(). Can you use @pytest.mark.parametrize instead to reduce duplication?

# regression coverage for https://github.com/jupyterhub/binderhub/issues/1620

username = "user"
password = "pass"
test_handle = {"username": username, "password": password}
app = Application(
[
(
r"/token",
MockTokenHandler,
{
"test_handle": test_handle,
"service": "service.1",
"scope": "scope.2",
},
),
]
)
ip = "127.0.0.1"
port = randint(10000, 65535)
app.listen(port, ip)

registry = DockerRegistry(
url="https://example.org", username=username, password=password
)

assert registry.url == "https://example.org"
assert registry.auth_config_url == "https://example.org"
# token_url should be unset, since it should be determined by the caller from a
# WWW-Authenticate header
assert registry.token_url == ""
assert registry.username == username
assert registry.password == password
token = await registry._get_token(
httpclient.AsyncHTTPClient(),
f"http://{ip}:{port}/token?service=service.1",
None,
"scope.2",
)
assert token == test_handle["token"]


@pytest.mark.parametrize("token_url_known", [True, False])
async def test_get_image_manifest(tmpdir, token_url_known):
username = "asdf"
Expand Down Expand Up @@ -230,7 +281,7 @@ async def test_get_image_manifest(tmpdir, token_url_known):
f,
)
if token_url_known:
token_url = url + "/token"
token_url = url + "/token?service=container_registry"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed? If it's testing another scenario can you use @pytest.mark.parametrize to test with and without this extra parameter?

else:
token_url = ""
registry = DockerRegistry(
Expand Down
28 changes: 28 additions & 0 deletions docs/source/zero-to-binderhub/setup-binderhub.rst
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,34 @@ where:
If this is not provided, you may find BinderHub rebuilds images every launch instead of pulling them from the ACR.
Suggestions for `<project-name>` could be `ACR_NAME` or the name you give your BinderHub.

If you are using an authenticated (private) GitLab Container Registry
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you want your BinderHub to push and pull images from an authenticated (private) GitLab Container Registry, then your `config.yaml` file will look as follows::

config:
BinderHub:
use_registry: true
image_prefix: registry.gitlab.com/<gitlab-user>/<project-name>/<prefix>-
DockerRegistry:
token_url: "https://gitlab.com/jwt/auth?service=container_registry"
url: "https://registry.gitlab.com"
username: <deploy-token-username>
password: <deploy-token-pw>
registry:
url: https://registry.gitlab.com
username: <deploy-token-username>
password: <deploy-token-pw>

where:

* `<gitlab-user>` is your gitlab username,
* `<deploy-token-username>` and `<deploy-token-pw>` are valid GitLab credentials. (Passwords may be moved to `secret.yaml`.)
* `<project-name>` is an arbitrary name that is required due to BinderHub assuming that `image_prefix` contains an extra level for the project name.
See `this issue <https://github.com/jupyterhub/binderhub/issues/800>`_ for futher discussion.
If this is not provided, you may find BinderHub rebuilds images every launch instead of pulling them from the GitLab container registry.
Suggestions for `<project-name>` could be the name you give your BinderHub.

If you are using OVH Container Registry
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down