Fix infinite loop in directory discovery #53
Conversation
|
Hi again. I see this PR was approved really quick but has remain unmerged for almost two months. Please, let me know if there is anything else I should do for this fix to be integrated in |
|
@manu-crealytics sorry for not following up on your contribution. It seems that the CI build on this PR never completed. Would you be so kind as to re-trigger CI by pushing an empty commit to your branch and check for the status of the CI. Once this is green we can proceed with merging your PR. |
manuteleco
force-pushed
the
fix_dir_blob_creation
branch
from
5eb431f
to
3789f49
Compare
|
Sorry, this time it was me not responding promptly. I've force-pushed the same commit, hoping that's enough to automatically trigger a build. Still, it doesn't seem to do anything. When I visit https://app.travis-ci.com/github/Othoz/gcsfs/builds, I see a banner on top and with red background that reads:
If the problem is that the current plan in Travis doesn't allow any additional user (me) to trigger builds, the obvious fix could be changing/upgrading the plan. But, as short term solution, it would be totally fine by me if you decide to close my PRs (this one and #54) and re-open them yourself. Here I am assuming that your user account is allowed to trigger Travis builds, while mine isn't. Let me know what you think and whether there is anything else I can do. Thanks. |
The `fix_storage` method enters an infinite loop when there are other
directories that have `root_path` as prefix.
For example, if the contents of the bucket look like this:
```
gs://mybucket/abc/file.txt
gs://mybucket/abc123/file.txt
```
this code results in an infinite loop:
```python
from fs import open_fs
open_fs("gs://mybucket/abc?strict=False").fix_storage()
```
In order to avoid this, we need to ensure `root_path` has a trailing
"/", when listing blobs, which prevents "abc123/file.txt" from being
listed as child of "abc".
Additionally, `Bucket.list_blobs` was deprecated in favor of
`Client.list_blobs` (see [reference documentation][1]).
[1]: https://cloud.google.com/python/docs/reference/storage/1.44.0/buckets#listblobsmaxresultsnone-pagetokennone-prefixnone-delimiternone-startoffsetnone-endoffsetnone-includetrailingdelimiternone-versionsnone-projectionnoacl-fieldsnone-clientnone-timeout60-retrygoogleapicoreretryretry-object
manuteleco
force-pushed
the
fix_dir_blob_creation
branch
from
3789f49
to
1809e10
Compare
|
Not sure if that's at all relevant, but I don't see this banner anymore:
Did anything change on your side? Are PRs expected to trigger builds again? I've applied a How should we proceed? |
The
fix_storagemethod enters an infinite loop when there are other directories that haveroot_pathas prefix.For example, if the contents of the bucket look like this:
this code results in an infinite loop:
In order to avoid this, we need to ensure
root_pathhas a trailing "/", when listing blobs, which prevents "abc123/file.txt" from being listed as child of "abc".Additionally,
Bucket.list_blobswas deprecated in favor ofClient.list_blobs(see reference documentation).Grüße aus Berlin