Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate if opening a store with icechunk 0.1.0-alpha15 corrupts a store written with alpha12 #78

Open
abarciauskas-bgse opened this issue Feb 5, 2025 · 0 comments
Assignees

Comments

@abarciauskas-bgse
Copy link
Contributor

abarciauskas-bgse commented Feb 5, 2025

@sharkinsspatial @maxrjones - I am just tagging you on this as an FYI since I asked you to review earth-mover/icechunk-nasa#4

I noticed upon testing today that the MUR SST icechunk store I created and previously demonstrated as working in https://github.com/earth-mover/icechunk-nasa/blob/main/notebooks/read_virtual.ipynb has been corrupted in someway. It's unclear when and how (I had an inconclusive slack conversation with Seba about this), and my only guess is that opening the store with a later version than it was created with somehow corrupts the data. Seba did confirm that the on disk format changes between pre-stable release versions (such as alpha12 and alpha15)

You can see the traceback for the error IcechunkError: store error: unsuccessful session operation: Repository error: error contacting storage error getting object from object store service error`` in this gist: https://gist.github.com/abarciauskas-bgse/ddfb19f02f3018df191c9ec8ade67765.

I'm not sure it's worth the time, given I think icechunk is nearing a stable release, but it would be nice to investigate in a toy example if my guess is correct and that reading with a later version corrupts the store.

@abarciauskas-bgse abarciauskas-bgse self-assigned this Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant