New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
esutil BulkIndexer Body ReadSeaker is incompatible with JSONReader #592
Comments
Hi @project0 Thanks for raising this, that change was more to align the existing API with the behavior of the indexer rather than a performance optimization. I do have received parallel requests for a different version of the bulk indexer which I think fits more the use case you are asking for. Would you be interested in that ? Let me know what you think! |
@Anaethelion my use case is pretty simple. I am just receiving a stream of event data (currently via http requests, hopefully eventually via kafka, etc..) and need to ingest into elasticsearch. |
I like that the library provides some convenient way to do bulk indexing data.
I think with v8 the
BulkIndexerItem.Body
has been changed fromReader
toReadSeeker
interface for better performance.This makes it very hard to use the library in a efficient way, even the provided JSONReader is now incompatible.
Setting the body becomes pretty hard with native packages using the
Reader
interface.Now we have to buffer/cache the data a lot of times, just so it gets wrapped into the
ReadSeeker
interface, i am in doubt if this makes it really more efficient.However, this is not working anymore:
instead, i need to do it manually:
The text was updated successfully, but these errors were encountered: