-
Notifications
You must be signed in to change notification settings - Fork 422
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Metrics SDK improvements #1740
Milestone
Comments
This was referenced Aug 24, 2024
Compared the benchmark to emit four attributes with OpenTelemetry .NET and we are comparable to .NET now: Machine InformationOS: Ubuntu 22.04.4 LTS (5.15.153.1-microsoft-standard-WSL2) Hardware: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz, 16vCPUs, RAM: 64.0 GB
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Opening a parent issue to track Metrics SDK improvements for Stable release.
Background
The primary function of the Metrics SDK is to accept a number
<T>
along with a slice of KeyValue pairs<T>, &[KeyValue]
, aggregating these measurements in memory and exporting the aggregated values to Readers/Exporters as needed. Our main goals are ensuring correctness, thread-safety, memory-efficiency and high performance, particularly on the "hot path" where measurement reporting occurs, as this demands the utmost efficiency. Correctness/thread-safety/memory-efficiency requires extensive testing via unit tests and stress testing.Performance issues
AttributeSet
. We can avoid this in many cases by using a thread-local Vec, which would reduce memory allocations, but still requires copy. Copy as well can be avoided by carefully designing temp data structures to hold references only.RwLock
and applying interior mutability could lessen this issue, though sharding may be necessary for further scalability improvements as demonstrated here.Some issues like calculating hash outside of lock, special casing 0-attributes etc. were addressed already. Also, a lot of ideas were discussed in the past (Community meetings, PRs, issues). I have attempted prototyping several of them here: https://github.com/cijothomas/metrics-mini/tree/main/metrics/src. A lot of the issues from 1,2,3, part of 4 has been addressed in the prototype, giving huge performance improvements. I plan to incorporate them to this repo soon.
It is unlikely that we fix all performance issues for 1.0, but the goal is to ensure that the fixes can be continued even after 1.0 without any breaking changes. This requires trimming off unnecessary public APIs, and also to avoid exposing any internals to readers/exporters.
Correctness issues:
Most of the correctness issues can fixed via better test coverage. One thing to note is that "Views" feature expands the testing matrix significantly due to its capability to alter aggregations/attributes and produce multiple metrics streams from a single measurement. This is the main reason to remove "Views" from the scope of 1st stable release.
The text was updated successfully, but these errors were encountered: