What happened? How can we reproduce this?
While building a Grafana dashboard using the Program Overhead metrics,
I noticed that tetragon_overhead_program_seconds_total seems to store
nanoseconds rather than seconds, despite the _seconds_total naming
convention.
Looking at the example from the official docs:
tetragon_overhead_program_runs_total = 15894
tetragon_overhead_program_seconds_total = 1.03908217e+08
1.039e8 / 15894 = 6537 → if seconds: 1.8 hours/call (impossible) if nanoseconds: 6.5 µs/call is OK
This is also consistent with the source code in
pkg/metrics/overhead/collector.go, where RunTime (which comes from
the kernel's run_time_ns) is passed directly to the metric without
dividing by 1e9.
Is this intentional? And is the correct unit actually seconds and I'm
misreading something?
Tetragon Version
tetragon:v1.5.0
Kernel Version
6.1.0-40-amd64
Kubernetes Version
Client Version: v1.30.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.13
Bugtool
No response
Relevant log output
Anything else?
No response
What happened? How can we reproduce this?
While building a Grafana dashboard using the Program Overhead metrics,
I noticed that tetragon_overhead_program_seconds_total seems to store
nanoseconds rather than seconds, despite the _seconds_total naming
convention.
Looking at the example from the official docs:
tetragon_overhead_program_runs_total = 15894
tetragon_overhead_program_seconds_total = 1.03908217e+08
1.039e8 / 15894 = 6537 → if seconds: 1.8 hours/call (impossible) if nanoseconds: 6.5 µs/call is OK
This is also consistent with the source code in
pkg/metrics/overhead/collector.go, where RunTime (which comes from
the kernel's run_time_ns) is passed directly to the metric without
dividing by 1e9.
Is this intentional? And is the correct unit actually seconds and I'm
misreading something?
Tetragon Version
tetragon:v1.5.0
Kernel Version
6.1.0-40-amd64
Kubernetes Version
Client Version: v1.30.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.13
Bugtool
No response
Relevant log output
Anything else?
No response