Increase UI reactiveness on folders manual upserts #11496
Conversation
| true => Refresh::True, | ||
| false => Refresh::False, |
There was a problem hiding this comment.
@flvndvd any insight on the impact of this?
There was a problem hiding this comment.
I'm wondering about the intended use case here, is the goal to run this every time someone upserts a document from the UI?
If so, Elasticsearch generally isn't designed for this approach. The refresh operation does come with some performance overhead (which is why the default is set to 30 seconds).
Perhaps we could explore a middle ground? We could try reducing the current refresh interval to a lower threshold that still meets your needs while being mindful of performance.
One other consideration, this implementation might allow multiple refresh operations to start simultaneously, which could potentially impact performance.
There was a problem hiding this comment.
Code looks good, but a bit wary of letting people use force_refresh in the code. Force refresh causes heavy I/O, segments merging overhead, and memory pressure as far as I can read.
Maybe it would be better to try with a simple change of refresh_interval to 5s first?
awaiting your opinion @flvndvd
core/bin/core_api.rs
Outdated
| @@ -1840,6 +1840,8 @@ struct DataSourcesDocumentsUpsertPayload { | |||
| title: String, | |||
| mime_type: String, | |||
| provider_visibility: Option<ProviderVisibility>, | |||
| // Force an ES refresh upon indexation to make the node searchable immediately. | |||
| force_refresh: Option<bool>, | |||
There was a problem hiding this comment.
force_search_index_refresh is more explicit I think
This reverts commit 13f2e51.
This reverts commit bdd4951.
This reverts commit 9289101.
This reverts commit d83bb58.
This reverts commit e4c0efd.
This reverts commit e28f062.
This reverts commit 8a932d5.
This reverts commit bb6506f.
This reverts commit 772fcc5.
This reverts commit e3657eb.
This reverts commit 3ba3e28.
aubin-tchoi
changed the title
Description
refresh_intervalon the ES index (onlycore.data_sources_nodes_4, no need forcore.data_sources_1) from 30s to 5s.Tests
Risk
Deploy Plan
20250320_reduce_refresh_interval.http.