Better handling of authentication on docker registries

[philpep]

Hi there,

I spend some time debugging binderhub to make it work with my own docker registry, because I needed to add a "scope" to my "token_url".

I think the token_url parameter and specific handling of docker hub, gcr.io and azure (and others in the future) could be dropped from https://github.com/jupyterhub/binderhub/blob/master/binderhub/registry.py#L189

Documentation is here: https://docs.docker.com/registry/spec/auth/token/

The idea is to make a request for manifest, catch for a 401 and look at the "www-authenticate" header which contains the authentication url and parameter to get a working token.
Then replay the first request using the token.

Example with docker registry:

# first request (eventually with basic auth if needed)
% http -ph https://registry.hub.docker.com/v2/library/nginx/manifests/latest
HTTP/1.1 401 Unauthorized
Content-Type: application/json
Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:library/nginx:pull"

# just  follow www-authenticate (eventually with basic auth if needed)
% http -phb 'https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/nginx:pull'
HTTP/1.1 200 OK
Content-Type: application/json

{"expires_in": 300, "issued_at": "2019-10-31T15:52:24.857363835Z", "token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsIng1YyI6Wy..."}


# replay first request with bearer auth token
http -ph https://registry.hub.docker.com/v2/library/nginx/manifests/latest 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsIng1YyI6Wy...'
HTTP/1.1 200 OK
Content-Type: application/vnd.docker.distribution.manifest.v1+prettyjws

I implemented this in another (golang) tool and it work well with docker hub, gcr and registry:2 (with cesanta/docker_auth:1).

If you need help to implement this, I can help. Let me known what you think about dropping "token_url" parameter in flavor of this implementation.

CC @dylex

[betatim]

Simplifying the code would be nice.

Related issues to "docker registry" topics that we should keep an eye on are #986 and #920 just so we are all going in the same direction.