Skip to content

Update docs on using a buffered stream #538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 1, 2025
Merged

Update docs on using a buffered stream #538

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions csharp/Arrow/ArrowReaderProperties.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ public void SetReadDictionary(int columnIndex, bool readDictionary)
/// When enabled, the Arrow reader will pre-buffer necessary regions
/// of the file in-memory. This is intended to improve performance on
/// high-latency filesystems (e.g. Amazon S3).
/// This is enabled by default.
/// </summary>
public bool PreBuffer
{
Expand Down
18 changes: 14 additions & 4 deletions csharp/ReaderProperties.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,17 @@ public void Dispose()
}

/// <summary>
/// Whether the buffered stream is enabled for reading.
/// Whether a buffered stream is used for reading.
///
/// This limits the size of reads from the underlying file to limit memory usage in resource
/// constrained environments.
/// Using a buffered stream is disabled by default.
/// Note that this has no effect when reading as Arrow data and <see cref="ParquetSharp.Arrow.ArrowReaderProperties.PreBuffer" /> is enabled.
/// </summary>
public bool IsBufferedStreamEnabled => ExceptionInfo.Return<bool>(Handle, ReaderProperties_Is_Buffered_Stream_Enabled);

/// <summary>
/// The size of the buffer (in bytes) used for reading.
/// The size of the buffer (in bytes) used for the buffered stream. This has no effect when the buffered stream is disabled.
/// </summary>
public long BufferSize
{
Expand Down Expand Up @@ -64,7 +69,12 @@ public FileDecryptionProperties? FileDecryptionProperties
}

/// <summary>
/// Enable buffered stream for reading.
/// Enable using a buffered stream for reading.
///
/// This limits the size of reads from the underlying file to limit memory usage in resource
/// constrained environments.
/// The size of the buffer can be controlled with the <see cref="BufferSize" /> parameter.
/// Note that this has no effect when reading as Arrow data and <see cref="ParquetSharp.Arrow.ArrowReaderProperties.PreBuffer" /> is enabled.
/// </summary>
public void EnableBufferedStream()
{
Expand All @@ -73,7 +83,7 @@ public void EnableBufferedStream()
}

/// <summary>
/// Disable buffered stream for reading.
/// Disable using a buffered stream for reading.
/// </summary>
public void DisableBufferedStream()
{
Expand Down