-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question/discussion] Why S3Path's smart_open usage defaults to compression='disable'? #93
Comments
Hi @Kache, thank you for opening the discussion. Now, in the past year we did added a custom features that relates to our specific implementations and we did add some features that specific for s3. |
Ah, that makes sense. Thanks. With that understanding, then my suggestion is to expose: S3Path('/mybucket/mypath/file.csv.gz').smart_open()
# equivalent to
import smart_open
smart_open.open(S3Path('/mybucket/mypath/file.csv.gz')) Could have the method passthrough arguments to I don't feel too strongly about it though, up to you. (Close issue if you'd like) |
Hi @Kache sounds good It will take me sometime to work on it. |
Seeing as how S3Path leverages smart_open, I was surprised that:
Did not "autodetect" compression from file extension, as smart_open does by default:
As it turns out, S3Path sets
compression='disable'
: https://github.com/liormizr/s3path/blob/master/s3path.py#L388Any particular reason?
The text was updated successfully, but these errors were encountered: