Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor error handling on read requests #750

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

passaro
Copy link
Contributor

@passaro passaro commented Feb 13, 2024

Description of change

Minor refactor to simplify how errors returned from Prefetch::read() are handled. It implements the error conversion in the fs::error module, to align it to errors on the write path.

Preliminary work for #644

Does this change impact existing behavior?

No change in behavior.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

Copy link
Contributor

@sauraank sauraank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I had one question. Why are we directly returning Bytes from prefetcher rather than converting them from ChecksummedBytes at the fs level?

@vladem vladem self-requested a review February 14, 2024 16:14
vladem
vladem previously approved these changes Feb 14, 2024
@passaro
Copy link
Contributor Author

passaro commented Feb 15, 2024

LGTM. I had one question. Why are we directly returning Bytes from prefetcher rather than converting them from ChecksummedBytes at the fs level?

  • the prefetcher can also return a PrefetchReadError::Integrity (in case the integrity checks failed at some other point in the pipeline), so it is simpler to merge the two and return an already validated Bytes
  • Before the change, we were immediately validating the ChecksummedBytes and extracting the Bytes anyway, so this does not actually moves when the validation happens.

@jamesbornholt
Copy link
Member

I kind of think we might prefer to keep the prefetcher returning ChecksummedBytes. Today we just convert immediately, but there's nothing stopping future us from inserting more work between the prefetcher and the FUSE response, and the type at least makes that decision clear.

@passaro
Copy link
Contributor Author

passaro commented Mar 11, 2024

@jamesbornholt, @sauraank, I reverted the change to the return type of read. It brings back 2 routes to get a form of integrity error, but overall I agree we want to propagate our "checked" type as much as possible, even if right now we immediately extract the buffer.

@jamesbornholt
Copy link
Member

Ok, but now I'm not sure why we need this change — does moving the error parsing logic out into a From impl buy us anything? I reverted this after checking out #644 and it didn't seem to break anything.

@passaro
Copy link
Contributor Author

passaro commented Mar 13, 2024

Ok, but now I'm not sure why we need this change — does moving the error parsing logic out into a From impl buy us anything? I reverted this after checking out #644 and it didn't seem to break anything.

Fair question. You're right, this is not really tied to #644 anymore, but I do see moving the PrefetchReadError mapping logic to the error module, as done for UploadWriteError, as a marginal improvement.

@passaro passaro temporarily deployed to PR integration tests May 3, 2024 19:56 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 3, 2024 19:56 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 3, 2024 19:56 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 3, 2024 19:56 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 3, 2024 19:56 — with GitHub Actions Inactive
Copy link
Contributor

@dannycjones dannycjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants