Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add telemetry #1981

Merged
merged 1 commit into from
Nov 29, 2024
Merged

feat: add telemetry #1981

merged 1 commit into from
Nov 29, 2024

Conversation

snakster
Copy link
Contributor

@snakster snakster commented Nov 28, 2024

What this PR does / why we need it:

This PR adds telemetry to collect metrics about which commands are used most actively. We already have some general insights from the checkpoint API, but this gives more detailed data.

Documentation will be updated accordingly.

Special notes for your reviewer:

Does this PR introduce a user-facing change?

- telemetry options have been added

@snakster snakster requested a review from a team as a code owner November 28, 2024 10:05
Copy link

github-actions bot commented Nov 28, 2024

2024/11/29 13:56:42 Terraform detected version: 1.9.8
toolsetPath: /tmp/cmd-terramate-test-572205335
=== RUN   TestInteropCloudSyncPreview
=== RUN   TestInteropCloudSyncPreview/preview:_basic-drift
    interoperability_test.go:46: using GITHUB_EVENT_FILE=/home/runner/work/_temp/_github_workflow/event.json
=== RUN   TestInteropCloudSyncPreview/preview:_basic-drift-uppercase-id
    interoperability_test.go:46: using GITHUB_EVENT_FILE=/home/runner/work/_temp/_github_workflow/event.json
--- PASS: TestInteropCloudSyncPreview (14.47s)
    --- PASS: TestInteropCloudSyncPreview/preview:_basic-drift (7.65s)
    --- PASS: TestInteropCloudSyncPreview/preview:_basic-drift-uppercase-id (6.82s)
=== RUN   TestInteropSyncDeployment
=== RUN   TestInteropSyncDeployment/deployment:_empty
=== RUN   TestInteropSyncDeployment/deployment:_empty-uppercase-id
--- PASS: TestInteropSyncDeployment (46.57s)
    --- PASS: TestInteropSyncDeployment/deployment:_empty (23.82s)
    --- PASS: TestInteropSyncDeployment/deployment:_empty-uppercase-id (22.76s)
=== RUN   TestInteropDrift
=== RUN   TestInteropDrift/drift:_basic-drift
=== RUN   TestInteropDrift/drift:_basic-drift-uppercase-id
--- PASS: TestInteropDrift (73.39s)
    --- PASS: TestInteropDrift/drift:_basic-drift (36.39s)
    --- PASS: TestInteropDrift/drift:_basic-drift-uppercase-id (37.00s)
PASS
ok  	github.com/terramate-io/terramate/e2etests/cloud/interop	194.503s

@snakster snakster force-pushed the snk-add-telemetry branch 2 times, most recently from 21ac914 to f8f728a Compare November 28, 2024 10:18
Copy link

github-actions bot commented Nov 28, 2024

metric: time/op
ChangeDetection-4: old 4.04ms ± 8%: new 4.08ms ± 4%: delta: 0.00%
ChangeDetectionTFAndTG-4: old 13.2ms ± 6%: new 13.4ms ± 5%: delta: 0.00%
CloudReadLines-4: old 1.00ms ± 7%: new 1.01ms ± 5%: delta: 0.00%
CloudReadLine-4: old 7.41ms ± 2%: new 7.42ms ± 2%: delta: 0.00%
ListFiles-4: old 86.3µs ± 0%: new 87.0µs ± 1%: delta: 0.83%
Generate-4: old 1.54s ± 4%: new 1.55s ± 3%: delta: 0.00%
GenerateRegex-4: old 1.06s ± 4%: new 1.06s ± 2%: delta: 0.00%
TokensForExpressionComplex-4: old 1.25ms ± 1%: new 1.26ms ± 1%: delta: 0.51%
TokensForExpressionPlainStringNoNewline-4: old 789ns ± 1%: new 790ns ± 2%: delta: 0.00%
TokensForExpressionStringWith100Newlines-4: old 23.8µs ± 1%: new 23.8µs ± 2%: delta: 0.00%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 1.45ms ± 0%: new 1.45ms ± 1%: delta: 0.23%
TokensForExpression-4: old 1.27ms ± 1%: new 1.26ms ± 1%: delta: 0.00%
PartialEvalComplex-4: old 553µs ± 2%: new 552µs ± 3%: delta: 0.00%
PartialEvalSmallString-4: old 3.94µs ± 1%: new 3.92µs ± 1%: delta: -0.66%
PartialEvalHugeString-4: old 1.92ms ± 1%: new 1.93ms ± 1%: delta: 0.37%
PartialEvalHugeInterpolatedString-4: old 5.00ms ± 1%: new 4.93ms ± 1%: delta: -1.37%
PartialEvalObject-4: old 27.1µs ± 2%: new 26.6µs ± 1%: delta: -1.64%
TmAllTrueLiteralList-4: old 616µs ± 1%: new 617µs ± 0%: delta: 0.00%
TmAllTrueFuncall-4: old 20.2µs ± 1%: new 20.3µs ± 1%: delta: 0.38%
TmAnyTrueLiteralList-4: old 4.87ms ± 1%: new 4.85ms ± 0%: delta: 0.00%
TmAnyTrueFuncall-4: old 20.6µs ± 1%: new 20.8µs ± 2%: delta: 0.71%
TmTernary-4: old 2.52µs ± 1%: new 2.50µs ± 1%: delta: -0.60%
TmTryUnknownFunc-4: old 2.27µs ± 0%: new 2.26µs ± 1%: delta: -0.29%
TmTryUnknownVariable-4: old 2.21µs ± 0%: new 2.17µs ± 0%: delta: -1.82%
TmTryUnknownObjectKey-4: old 2.52µs ± 2%: new 2.49µs ± 1%: delta: -1.06%
metric: alloc/op
ChangeDetection-4: old 353kB ± 0%: new 353kB ± 0%: delta: 0.00%
ChangeDetectionTFAndTG-4: old 4.81MB ± 0%: new 4.80MB ± 0%: delta: 0.00%
CloudReadLines-4: old 3.12MB ± 0%: new 3.12MB ± 0%: delta: 0.00%
CloudReadLine-4: old 3.37MB ± 0%: new 3.37MB ± 0%: delta: 0.00%
ListFiles-4: old 27.6kB ± 0%: new 27.6kB ± 0%: delta: 0.00%
Generate-4: old 2.24GB ± 0%: new 2.24GB ± 0%: delta: 0.00%
GenerateRegex-4: old 926MB ± 0%: new 926MB ± 0%: delta: 0.00%
TokensForExpressionComplex-4: old 394kB ± 0%: new 394kB ± 0%: delta: 0.00%
TokensForExpressionPlainStringNoNewline-4: old 512B ± 0%: new 512B ± 0%: delta: 0.00%
TokensForExpressionStringWith100Newlines-4: old 12.5kB ± 0%: new 12.5kB ± 0%: delta: 0.00%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 395kB ± 0%: new 395kB ± 0%: delta: 0.00%
TokensForExpression-4: old 394kB ± 0%: new 394kB ± 0%: delta: 0.00%
PartialEvalComplex-4: old 361kB ± 0%: new 361kB ± 0%: delta: -0.00%
PartialEvalSmallString-4: old 1.95kB ± 0%: new 1.95kB ± 0%: delta: 0.00%
PartialEvalHugeString-4: old 196kB ± 0%: new 196kB ± 0%: delta: 0.00%
PartialEvalHugeInterpolatedString-4: old 4.31MB ± 0%: new 4.31MB ± 0%: delta: -0.00%
PartialEvalObject-4: old 19.4kB ± 0%: new 19.4kB ± 0%: delta: 0.00%
TmAllTrueLiteralList-4: old 321kB ± 0%: new 321kB ± 0%: delta: -0.00%
TmAllTrueFuncall-4: old 10.6kB ± 0%: new 10.6kB ± 0%: delta: 0.00%
TmAnyTrueLiteralList-4: old 2.09MB ± 0%: new 2.09MB ± 0%: delta: 0.00%
TmAnyTrueFuncall-4: old 10.7kB ± 0%: new 10.7kB ± 0%: delta: 0.00%
TmTernary-4: old 1.18kB ± 0%: new 1.18kB ± 0%: delta: 0.00%
TmTryUnknownFunc-4: old 784B ± 0%: new 784B ± 0%: delta: 0.00%
TmTryUnknownVariable-4: old 768B ± 0%: new 768B ± 0%: delta: 0.00%
TmTryUnknownObjectKey-4: old 952B ± 0%: new 952B ± 0%: delta: 0.00%
metric: allocs/op
ChangeDetection-4: old 2.39k ± 0%: new 2.39k ± 0%: delta: 0.00%
ChangeDetectionTFAndTG-4: old 33.6k ± 0%: new 33.6k ± 0%: delta: 0.00%
CloudReadLines-4: old 5.54k ± 0%: new 5.54k ± 0%: delta: 0.00%
CloudReadLine-4: old 60.0k ± 0%: new 60.0k ± 0%: delta: 0.00%
ListFiles-4: old 335 ± 0%: new 335 ± 0%: delta: 0.00%
Generate-4: old 25.7M ± 0%: new 25.7M ± 0%: delta: 0.00%
GenerateRegex-4: old 18.4M ± 0%: new 18.4M ± 0%: delta: 0.00%
TokensForExpressionComplex-4: old 4.83k ± 0%: new 4.83k ± 0%: delta: 0.00%
TokensForExpressionPlainStringNoNewline-4: old 20.0 ± 0%: new 20.0 ± 0%: delta: 0.00%
TokensForExpressionStringWith100Newlines-4: old 227 ± 0%: new 227 ± 0%: delta: 0.00%
TokensForExpressionObjectWith100KeysWithNumberValues-4: old 3.19k ± 0%: new 3.19k ± 0%: delta: 0.00%
TokensForExpression-4: old 4.83k ± 0%: new 4.83k ± 0%: delta: 0.00%
PartialEvalComplex-4: old 3.78k ± 0%: new 3.78k ± 0%: delta: 0.00%
PartialEvalSmallString-4: old 26.0 ± 0%: new 26.0 ± 0%: delta: 0.00%
PartialEvalHugeString-4: old 38.0 ± 0%: new 38.0 ± 0%: delta: 0.00%
PartialEvalHugeInterpolatedString-4: old 26.1k ± 0%: new 26.1k ± 0%: delta: 0.00%
PartialEvalObject-4: old 183 ± 0%: new 183 ± 0%: delta: 0.00%
TmAllTrueLiteralList-4: old 5.94k ± 0%: new 5.94k ± 0%: delta: 0.00%
TmAllTrueFuncall-4: old 275 ± 0%: new 275 ± 0%: delta: 0.00%
TmAnyTrueLiteralList-4: old 59.6k ± 0%: new 59.6k ± 0%: delta: 0.00%
TmAnyTrueFuncall-4: old 277 ± 0%: new 277 ± 0%: delta: 0.00%
TmTernary-4: old 27.0 ± 0%: new 27.0 ± 0%: delta: 0.00%
TmTryUnknownFunc-4: old 21.0 ± 0%: new 21.0 ± 0%: delta: 0.00%
TmTryUnknownVariable-4: old 20.0 ± 0%: new 20.0 ± 0%: delta: 0.00%
TmTryUnknownObjectKey-4: old 23.0 ± 0%: new 23.0 ± 0%: delta: 0.00%

@snakster snakster force-pushed the snk-add-telemetry branch 2 times, most recently from f8f85f1 to e37f69d Compare November 28, 2024 11:09
@snakster snakster marked this pull request as draft November 28, 2024 11:16
@snakster snakster force-pushed the snk-add-telemetry branch 3 times, most recently from f21a1e1 to ef2117c Compare November 28, 2024 12:34
Copy link

github-actions bot commented Nov 28, 2024

Preview of ubuntu-focal/go1.21 tests in 1f69272

🔍 View Details on Terramate Cloud

.
cmd/terramate/cli
cmd/terramate/cli/cliconfig
cmd/terramate/cli/telemetry
generate
generate/genfile
generate/genhcl
hcl

Copy link

github-actions bot commented Nov 28, 2024

Preview of macos-ventura/go1.21 tests in 1f69272

🔍 View Details on Terramate Cloud

.
cmd/terramate/cli
cmd/terramate/cli/cliconfig
cmd/terramate/cli/telemetry
generate
generate/genfile
generate/genhcl
hcl

@snakster snakster marked this pull request as ready for review November 28, 2024 12:49
Copy link
Contributor

@i4ki i4ki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but please address the comments in the public types and values.

cmd/terramate/cli/telemetry/telemetry.go Show resolved Hide resolved
cmd/terramate/cli/telemetry/telemetry.go Outdated Show resolved Hide resolved
cmd/terramate/cli/telemetry/endpoint.go Outdated Show resolved Hide resolved
cmd/terramate/cli/telemetry/endpoint.go Outdated Show resolved Hide resolved
// Copyright 2023 Terramate GmbH
// SPDX-License-Identifier: MPL-2.0

//go:build localhostEndpoints
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you use environment variables?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using the same approach as the checkpoint API here. It could be more convenient for debugging, but beyond that there is no reason to make the endpoint address configurable, so I'd just go with the existing scheme.

cmd/terramate/cli/telemetry/record.go Outdated Show resolved Hide resolved
@snakster snakster force-pushed the snk-add-telemetry branch 4 times, most recently from a1dbd15 to fca5a8d Compare November 29, 2024 09:37
@snakster snakster requested a review from i4ki November 29, 2024 09:38
Copy link
Contributor

@i4ki i4ki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Well done!!
🚀

@snakster snakster merged commit 504a7f3 into main Nov 29, 2024
7 checks passed
@snakster snakster deleted the snk-add-telemetry branch November 29, 2024 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants