Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add micrometer.prometheus-proxy.connected-metrics-metadata property #68

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Mateo00
Copy link

@Mateo00 Mateo00 commented Mar 29, 2023

Add micrometer.prometheus-proxy.connected-metrics-metadata property to enable or disable publishing Prometheus metrics metadata (lines "# HELP ..." and lines "# TYPE ...") in the payload returned by the /metrics/connected endpoint. The default value is True.

Disabling metrics metadata reduces the amount of data sent on each scrape.

In addition, some clients such as Telegarf expect the "# HELP ..." lines to be unique.
Currently, collecting metrics reported by the /metrics/connected endpoint fails with a Telegarf agent:
parsing metrics failed: reading text format failed: text format parsing error in line XX: second HELP line for metric name "XXXXX"

Disabling metrics metadata allows to work around this problem and scrapping the /metrics/connected endpoint with a Telegraf agent works well.

@pivotal-cla
Copy link

@Mateo00 Please sign the Contributor License Agreement!

Click here to manually synchronize the status of this Pull Request.

See the FAQ for frequently asked questions.

@pivotal-cla
Copy link

@Mateo00 Thank you for signing the Contributor License Agreement!

@Mateo00
Copy link
Author

Mateo00 commented Apr 20, 2023

Hi @shakuzen,
Is the PR compliant or is there a missing step on my end for it to be considered ?

@shakuzen
Copy link
Member

I think I've understood the problem. Losing the HELP lines seems somewhat alright, but losing the TYPE lines seems potentially more problematic. Where are you eventually storing the metrics after Telegraf scrapes them, and is there no issue for you losing all type info?

@Mateo00
Copy link
Author

Mateo00 commented May 15, 2023

After Telegraf scrapes, the metrics are stored in a VictoriaMetrics database. A Grafana connected to this database allows us to visualize the collected metrics.
In our tests, we didn't find any problems when the TYPE lines are deleted. All metrics are saved in the database and they display correctly in Grafana dashboards.

In the Prometheus documentation it's mentioned that HELP and TYPE lines are optional.
https://prometheus.io/docs/instrumenting/exposition_formats/#grouping-and-sorting :
Grouping and sorting
All lines for a given metric must be provided as one single group, with the optional HELP and TYPE lines first (in no particular order). Beyond that, reproducible sorting in repeated expositions is preferred but not required, i.e. do not sort if the computational cost is prohibitive.
Each line must have a unique combination of a metric name and labels. Otherwise, the ingestion behavior is undefined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants