You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We see some Lucene indices taking many seconds (occasionally minutes) to abort merges during rollback, doing a lot of now-pointless IO, with the merge thread spending all its time within a call to one of the various checkIntegrity methods that reads a file from beginning to end. For instance:
Also here, although this is using a org.elasticsearch.index.codec.postings.ES812PostingsReader it doesn't look to be doing anything different from e.g. org.apache.lucene.codecs.lucene99.Lucene99PostingsReader#checkIntegrity:
The data in these cases is in rather cold storage so we would expect it to take quite some time (possibly minutes) to complete this end-to-end read. That's ok, we don't need such merges to complete especially quickly, but it is rather troublesome that it takes so long to react to the abort signal in these situations. Is there something we can do to abort this read more promptly? For instance, could we add an abort-sensitive wrapper to the DataInput that's reading the data?
Version and environment details
Lucene 9.10 embedded in Elasticsearch (here the main branch, currently targetting 8.15.0-SNAPSHOT) but this behaviour does not seem to be at all new.
The text was updated successfully, but these errors were encountered:
Description
We see some Lucene indices taking many seconds (occasionally minutes) to abort merges during rollback, doing a lot of now-pointless IO, with the merge thread spending all its time within a call to one of the various
checkIntegrity
methods that reads a file from beginning to end. For instance:Also here, although this is using a
org.elasticsearch.index.codec.postings.ES812PostingsReader
it doesn't look to be doing anything different from e.g.org.apache.lucene.codecs.lucene99.Lucene99PostingsReader#checkIntegrity
:The data in these cases is in rather cold storage so we would expect it to take quite some time (possibly minutes) to complete this end-to-end read. That's ok, we don't need such merges to complete especially quickly, but it is rather troublesome that it takes so long to react to the abort signal in these situations. Is there something we can do to abort this read more promptly? For instance, could we add an abort-sensitive wrapper to the
DataInput
that's reading the data?Version and environment details
Lucene 9.10 embedded in Elasticsearch (here the
main
branch, currently targetting8.15.0-SNAPSHOT
) but this behaviour does not seem to be at all new.The text was updated successfully, but these errors were encountered: