feat: try to get content length with zero copy #81
Conversation
Signed-off-by: Ben Tam <[email protected]>
|
@chripo |
utils.go
Outdated
| case *strings.Reader: | ||
| contentLength = int64(reader.Len()) | ||
| case io.Seeker: | ||
| pos, err := reader.Seek(0, io.SeekEnd) |
There was a problem hiding this comment.
this is probably slightly incorrect as the reader could be at a position other than zero:
I think you need to:
startPos, err = reader.Seek(0, io.SeekCurrent)to determine the current positiontotalLength, err = reader.Seek(0, io.SeekEnd)to determine the length entire streamreader.Seek(0, startPos)to restore the original starting position.return totalLength - startPos- this is how many bytes are remaining
utils.go
Outdated
| @@ -111,3 +111,29 @@ func (l *limitedReadCloser) Read(buf []byte) (int, error) { | |||
| func (l *limitedReadCloser) Close() error { | |||
| return l.rc.Close() | |||
| } | |||
|
|
|||
| func GetContentLength(reader io.Reader) (int64, error) { | |||
| contentLength := int64(0) | |||
There was a problem hiding this comment.
zero is a valid length for some transfers, this should probably return -1 to indicate the length is unknown or return additional bool
Signed-off-by: Ben Tam <[email protected]>
|
@jkowalski |
|
The branch for bytes.Reader and strings.Reader are removed because they have implemented io.Seeker |
|
thanks for the collaboration! |
|
I think you still need to modify For the testing I would recommend trying the following case:
For each implementation try different lengths: Try the length of zero and make sure it is explicitly passed as Content-Length: 0 when length Is known and not set at all if length is unknown Try some non-zero determinate and non-determinate lengths and make sure Content-Length is set accordingly. bonus points if you can also try a very large stream of 1GB without a length to make sure it's never loaded into memory and streamed all the way to the server. |
|
Thank for your idea, I will implement it in these days |
No description provided.