Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Add more telemetry metrics and insight for usage analytics #2017

Open
1 of 26 tasks
akphi opened this issue Mar 16, 2023 · 0 comments
Open
1 of 26 tasks
Labels
Application: Query Issues related to Legend Query application Application: Studio Issues related to Legend Studio application Application: Taxonomy Issues related to Legend Taxonomy application Component: UX/Design Issues related to UI/UX features and improvements, style system, and design principle Studio Core Team Opened by a member of the Studio core team Type: Feature Request Type: Mega-thread Tracker for multiple related issues
Milestone

Comments

@akphi
Copy link
Contributor

akphi commented Mar 16, 2023

Similar requests

  • I have searched and found no existing similar requests

How are you using Legend?

Legend Studio, Legend Query, Legend Taxonomy, Legend Pure IDE

What problems are you trying to solve?

We have laid out the foundation for telemetry across all the apps, we now should systematically find metrics that worths capturing, they divide roughly into 2 camps:

  • Usage metrics: what actions do users take, what do they use (potentially, for deeper study, how do they use that feature, where do they use it from, etc.)
  • Performance metrics: For long-running actions, what is the timing, what is the load, etc.

I want the team to work on this thread together to exhaustively find all the interesting metrics to capture, also potentially, we can discuss popular capturing techniques to generalize that into some method inside of ApplicationStore.TelemetryService

And most importantly, after we capture those metrics, what is the best way to present those metrics in dashboard to provide value and insights into users' behavior

Describe the solution you would like to see implemented

  • Think about another kind of endpoint that does the absolute, call a bunch of APIs that pulls stats like: total number of queries, dataspaces, projects, etc. and create cron job to push these data to analytics service
  • Include generic object to indicate users' workspace: SDLC coordinates, GAV, Query ID, etc.
  • Metrics How can we track the number of times people flip between text/form mode - check for windows visibility
    • Leaving/entering text-mode
    • Service test runner
    • Mapping test runner
    • Service registration
    • For compilation / graph load / execution
    • Check entitlement
    • Mapping algorithm
    • Loading model
    • Pushing change
    • Updating workspace
    • Updating project configuration
    • Database builder
    • Sample data generator: metrics for launch, with format info
    • Query Builder: Preview data
    • Query Builder: Typeahead
  • THINK Consider the following aspects of metrics:
    • For performance metrics, what kind of timings should we capture? For multi-staged timings, what do we do?
    • How can we justify that a feature is popular, should we based it off the percentage of unique user using it?
    • Think about how we can determine feature to deprecate
    • Think about how we can make use of application session telemetry to study users' behavior per session, such as how frequently they do action a, b, c, etc.
    • Think about the popularity of a feature, percentage of users that use the feature - just to see how popular the actions are

Describe alternatives you have considered

No response

Documentation, Design, Adoption, Migration Strategy

No response

Contribution

  • I would like to work on this feature
@akphi akphi added Type: Feature Request Application: Studio Issues related to Legend Studio application Application: Query Issues related to Legend Query application Type: Mega-thread Tracker for multiple related issues Application: Taxonomy Issues related to Legend Taxonomy application Component: UX/Design Issues related to UI/UX features and improvements, style system, and design principle labels Mar 16, 2023
@akphi akphi modified the milestones: On Deck, 10.0.0 Mar 16, 2023
@akphi akphi self-assigned this Mar 16, 2023
@github-actions github-actions bot added the Studio Core Team Opened by a member of the Studio core team label Mar 16, 2023
@github-actions github-actions bot modified the milestones: 10.0.0, 11.0.0 Jul 20, 2023
@akphi akphi removed the help wanted label Oct 23, 2023
@github-actions github-actions bot modified the milestones: 11.0.0, 12.0.0 Dec 21, 2023
@github-actions github-actions bot modified the milestones: 12.0.0, 13.0.0 Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Application: Query Issues related to Legend Query application Application: Studio Issues related to Legend Studio application Application: Taxonomy Issues related to Legend Taxonomy application Component: UX/Design Issues related to UI/UX features and improvements, style system, and design principle Studio Core Team Opened by a member of the Studio core team Type: Feature Request Type: Mega-thread Tracker for multiple related issues
Projects
None yet
Development

No branches or pull requests

5 participants