Skip to content

Is tetragon_overhead_program_seconds_total intentionally storing nanoseconds? #4829

@danilovid

Description

@danilovid

What happened? How can we reproduce this?

While building a Grafana dashboard using the Program Overhead metrics,
I noticed that tetragon_overhead_program_seconds_total seems to store
nanoseconds rather than seconds, despite the _seconds_total naming
convention.

Looking at the example from the official docs:

tetragon_overhead_program_runs_total = 15894
tetragon_overhead_program_seconds_total = 1.03908217e+08

1.039e8 / 15894 = 6537 → if seconds: 1.8 hours/call (impossible) if nanoseconds: 6.5 µs/call is OK

This is also consistent with the source code in
pkg/metrics/overhead/collector.go, where RunTime (which comes from
the kernel's run_time_ns) is passed directly to the metric without
dividing by 1e9.

Is this intentional? And is the correct unit actually seconds and I'm
misreading something?

Tetragon Version

tetragon:v1.5.0

Kernel Version

6.1.0-40-amd64

Kubernetes Version

Client Version: v1.30.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.13

Bugtool

No response

Relevant log output

Anything else?

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions