Support for listen-metrics-urls configuration

[heilerich]

/area monitoring
/kind enhancement

What would you like to be added: We would like to have the ability to pass down the listen-metrics-urls option to etcd. Maybe through an additional field in .spec.listenMetricsURLs in the Etcd CR? There is already the .spec.metrics field to set the metrics level of etcd.

Why is this needed: Currently metrics are served on the client port. If TLS / authentication is enabled for the client port, a client certificate is required to scrape metrics. This means that an (external) monitoring system is effectively granted full database access as well (with all its potential k8s secrets etc.). The ability to enable the dedicated metrics port would allow users to improve security by reducing the blast radius of a compromised monitoring system.

We would be willing to provide a PR if this feature would be welcome.

[renormalize]

Thanks for your interest @heilerich!

In general the proposed enhancement does sound good to me.
Introducing a top level spec field .spec.listerMetricsURLs does not sound like the best way to go about this, though.

It's slightly unfortunate that .spec.metrics specifies the metrics level currently, instead of it being a section where all metrics related configuration lived; a .spec.metrics.listenMetricsURLs would have been ideal.

Unsure if we'd want to repurpose the .spec.metrics field. Thoughts, @gardener/etcd-druid-maintainers?

You're also more than welcome to contribute this feature through a PR once the maintainers reach a consensus on this.

[shreyas-s-rao]

@heilerich thanks a lot for proposing this change. It makes sense to limit access to the DB and only specific actors have TLS creds for the etcd.

You're right @renormalize , the current API is a bit tricky to use for this. Currently spec.etcd.metrics just points to the metrics level, and should have ideally been spec.etcd.metrics.level, so that this new field could be spec.etcd.metrics.listenURLs. I feel currently the best place for this would be spec.etcd.listenMetricsURLs.

@heilerich please let me know if this suits you, and do feel free to raise a PR for the same :)

[unmarshall]

@shreyas-s-rao, @renormalize can you please check if enable TLS for metrics endpoint via --listen-metrics-urls then it will also impact health endpoint. In K8S TLS is not supported for health endpoints.
Please check: etcd-io/etcd#18477