-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for streaming large files #128
Comments
Hi there, yep some work on this was started in #40, but that has eventually been abandoned. I'd be happy to review and accept PR's that implement some kind of streaming option or support for large files. However, I'm not sure how the replace algorithm will work correctly if the file contents are read in chunks (for example, the regex may not find a match when a chunk is split exactly at a match). |
Hello @adamreisnz |
Hi @Noeldesarrollo , please fork the repository and then open a pull request, and I'll be able to have a look. |
PR: #139 |
Hello Adam,
Thanks for the great library, it's really useful.
I'd like to mention one thing about its usage: when trying to perform replacements in big files (like hundreds of megabytes) then OutOfMemory errors could easily pop up, because the lib loads the file content fully into the memory before, actually, making the replacement. I'm not quite sure whether this was done by design, but it, probably, makes sense to at least mention this fact in README.
An alternative solution, which is memory-friendly (uses streams), could be something like https://github.com/eugeneware/replacestream
The text was updated successfully, but these errors were encountered: