feat: add observability to anchoring worker #1199
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## feat/no-ipfs #1199 +/- ##
===============================================
Coverage ? 81.70%
===============================================
Files ? 46
Lines ? 1847
Branches ? 291
===============================================
Hits ? 1509
Misses ? 338
Partials ? 0 ☔ View full report in Codecov by Sentry. |
| @@ -302,45 +302,65 @@ export class AnchorService { | |||
| private async _anchorCandidates(candidates: Candidate[]): Promise<Partial<AnchorSummary>> { | |||
| logger.imp(`Creating Merkle tree from ${candidates.length} selected streams`) | |||
| const span = Metrics.startSpan('anchor_candidates') | |||
|
|
|||
| const buildMerkleTreeSpan = Metrics.startSpan('build_merkle_tree') | |||
There was a problem hiding this comment.
Does startSpan create timing metrics automatically? As in, how long a given span takes on avg, etc.
There was a problem hiding this comment.
A span represents a single operation within a specific trace. Traces are used to monitor the execution paths in a system. startSpan() will handle timing by reporting start and endTime.
It does not allow us to do aggregations, like counts or gauges. So the answer to your question is : no https://opentelemetry.io/docs/concepts/signals/traces/#spans
There was a problem hiding this comment.
In that case, can we please add metrics for the time spent in each of those spans?
Counter for how many time a span is taken, a summary for how long each took, etc.
There was a problem hiding this comment.
I am wondering if this is necessary after we data-dog profiling enabled on CAS @christianlavoie @smrz2001
[Replace Me With Meaningful Name] - #[Issue]
Description
Added spans to see how long each transaction while anchoring candidates takes in the worker.
This will help us identify bottlenecks and improve scalability.
How Has This Been Tested?
Describe the tests that you ran to verify your changes. Provide instructions for reproduction.
Definition of Done
Before submitting this PR, please make sure:
References:
Please list relevant documentation (e.g. tech specs, articles, related work etc.) relevant to this change, and note if the documentation has been updated.