Soft-deprecate /metrics endpoint

[yuvipanda]

/metrics is where notebook server publishes prometheus metrics.
Currently, if nbresuse is installed, it's impossible to get
prometheus metrics out of running notebooks.

Just removing /metrics caused issues with third-party
code (#36) that depended
on the JSON API from the endpoint, and so was reverted.

Instead, we 'soft' deprecate that endpoint.

1. Re-add /api/metrics/v1 endpoint, with exact same semantics as
   /metrics. This is enabled unconditionally.
2. /metrics is also enabled by default, but can be turned off with
   a notebook config. This allows admins who need prometheus metrics
   to turn it on.
3. The bundled notebook extension will read from /api/metrics/v1
   endpoint, so for those users things will 'just work'.

The path to removing /metrics is:

1. Release this as 0.4.0 even though there aren't any breaking changes.
   This makes it easier for code using legacy endpoints to simply pin
   to nbresuse<0.4.0
2. Find third party users relying on /metrics, and simply change them
   to use /api/metrics/v1 - should be a fairly simple change. They can
   then be pin to nbresuse>=0.4.0
3. When we're sure enough packages have switched, we can release a 0.5.0
   where /metrics is disabled by default, but enablable with a config.
4. Eventually, we can simply remove /metrics and the associated config.

Ref #36

[jtpio]

Thanks!

The path to removing /metrics looks good and similar to what was summarized in the roadmap, with one more step of deprecation and backward compatibility with /metrics: #58

[jtpio]

Should this be part of a separate release commit?

[yuvipanda]

It is a separate commit, just in the same PR. Should probably be ok?

[jtpio]

Thinking that we might want to add a couple more changes between this PR and the release.

For example moving the lab extension here to the nbresuse repo: jupyterlab/jupyterlab#9363

[yuvipanda]

@jtpio hmm, ideally I'd like to do a quick release here, and then maybe do a 0.5 with other changes - so only change in 0.4 is the deprecation. Since existing JupyterLab installations will work with 0.4 anyway, it should be probably ok?

[jtpio]

Sounds good 👍

So to summarize:

• 0.4.0 with the soft-deprecated endpoint. Users in lab 2.x will still see the indicator because /metrics is still available
• if the status bar item is removed from core for lab 3.0, then we release nbresuse 0.5.0 with this change: Add a JupyterLab extension for the memory usage status bar item #69. With 0.5.0 removing the /metrics endpoint (or enablable with a config) so users with the lab 2.x + nbresuse combination don't see the indicator twice in the status bar

[jtpio]

Thanks a lot Yuvi for this!

[yuvipanda]

Thanks for reviewing & engaging, @jtpio! Would you feel comfortable merging this PR? :D

[jtpio]

Sure. Was holding a bit more in case you had some input of the summary in #68 (comment)

[jtpio]

Would you like to proceed with the release? Otherwise I can do it later today / tomorrow.

[yuvipanda]

@jtpio nope, would be great if you could do the release!