[Lens][Annotations] Annotation group should behave by reference

[thomasneirynck]

Describe the feature:

When adding an annotation group to one or more visualizations, this group should behave "by reference". When changes are made to the group, these changes should reflect on all charts.

Currently, a group behaves "by value". A user can add a group to a chart, but it "copies" over all the annotaitons. When this group is edited, these changes will only reflect on that single chart.

Describe a specific use case for the feature:

This is an important feature because it allows annotations to be kept "in sync" on multiple charts on a dashboards. ie. different charts should be able to show the same annotations.

[elasticmachine]

Pinging @elastic/kibana-visualizations (Team:Visualizations)

[dej611]

I think @thomasneirynck found out some underline problem with the current Annotation sync flow.

Current state

Currently an annotation group can behave in 3 ways:

• by value
  • the annotation layer lives only inside the visualization where it's defined
• by reference
  • have a SO on its own, be loaded from the library and be updated by either its own editor or by explicit click save on an annotation layer within the visualization where it's used
  • it seems inline editing has an underline bug here for this where the other by-reference instances in a dashboard are not correctly synced
• hybrid approach
  • this is a sort of multi-authoring experience. The annotation can start as by-reference but it can be "changed" without propagating to the original reference.
  • this is the default behaviour in Lens, which is leading to confusion to authors like @thomasneirynck

The problem

The main problem with the hybrid approach seems to be related to its implementation, which is designed to be a multi-author experience while lacking the foundations to resolve the type of conflicts that this kind of user experience requires.

This error can be described with the following story:

• [t0] User A creates a by reference annotation A1 with Event 1
• [t1] User B creates a visualization importing the annotation A1 as layer
• [t2] User B deletes Event 1 from the layer and adds an Event 2 to it without saving upstream
• [t3] User A edits the annotation A1 and adds a new Event 3 to it making it persistent
• [t4] User B saves its by-value changes upstream, making the annotation by-reference again

After t4 the edits made by User B now overrides completely what User A did.
Additionally if any shared Event would have changed in the by-reference annotation this would have been completely reset by the user B overwrite action.
Like the example above, there are many more scenarios where things can get out of sync very easily.

Some ideas

Idea 1

First idea that comes to mind is to actually go all-in with a full multi-author experience (a-la google doc) with a data structure and sync engine able to handle this type of challenges.
Like any multi-author experiences it must be well investigate to identify potential risks and limits (i.e. the offline sync problem).
Implementation cost would likely be high in this case.

Idea 2

On the complete opposite side we could stop the hybrid approach once for all.
This can be achieved in different ways, each one with a different gradient of friction from the user experience prospective:

1. not allow edit on by-reference annotation layers within visualization

• provide a way to direct the user to the annotations editor (not great UX but safe)

2. allow edit on by-reference annotation layers and directly sync each change upstream

• UX similar to the by-reference visualizations in a dashboard (still not great UX but safe)

3. allow edit the by-reference annotation layer and prompt the user about syncing upstream the changes to not lose them

• probably the UX with less friction of the pack

Probably the idea space here has not bee fully explored, so feedback is welcomed.

cc @andreadelrio @MichaelMarcialis @ghudgins

[ghudgins]

I don't think we need to be too worried about simulataneous edits of annotation groups from different users in different visualizations. Is there an "Idea 1" where the last saver wins? (thus keeping the implementation cost down?)

[MichaelMarcialis]

hybrid approach

• this is a sort of multi-authoring experience. The annotation can start as by-reference but it can be "changed" without propagating to the original reference.
• this is the default behaviour in Lens, which is leading to confusion to authors like @thomasneirynck

@thomasneirynck: Is this hybrid approach the issue you were encountering? For example, you loaded a by-reference annotation into a visualization, made a change to it, but didn't save the annotation changes to the library (leaving it in a dirty/changed state)? If so, did you assume the annotation changes would be auto-saved and applied to all other visualizations when saving the parent visualization?

• it seems inline editing has an underline bug here for this where the other by-reference instances in a dashboard are not correctly synced

@thomasneirynck: Or was this your issue, where there was a bug in the inline editing experience that was causing your saved by-reference annotations not to be correctly synced?

If your issue was the hybrid approach, and it's one that we can confidently say confuses our users, then we can certainly consider alternatives. The most obvious approach is likely forcing users to save any dirty/changed by-reference annotations when attempting to save the parent visualization. It's a roadblock we we hoping to avoid with the hybrid approach, but I imagine it would be considered acceptable to avoid confusion.

However, if your issue was the inline editing bug that @dej611 mentioned, then I'd hesitate to make any changes beyond fixing the bug.