Bugfix/gzip

[jaboja]

• I've fixed the gzip responses, which seemed to get wrong when the chunked transfer encoding was also on. It may however still need some work, to make the gzip chunked, and not just avoid doing a chunked response when it's gzipped.
• I've also fixed the MIME type for GET responses for directories, as the httpd/unix-directory makes no sense with a HTML listing of a directory.
• And as an alternative to those listings, I've added support for index.html files. It requires defining the name(s) of the index file(s), however, and by default is disabled.

[andrewleech]

Fantastic cleanup & consolidation in data_to_bytes_iterator() and send_body_encoded(). I've read through them and to me it does look like you've correctly captured the slightly different usages of this code nicely with a bunch of great simplifications along the way!

I don't have any way to really test this myself other than the unit tests run here in Actions - which fail but they fail in the same way as the master branch :-) so it's equivalent as far as that goes!

[andrewleech]

What inspired this change?
I'm not sure what the webdav spec suggests, however a google of webdav mime "httpd/unix-directory" gets a lot of hits; it seems to be pretty common?

[jaboja]

This is not a WebDAV response, but a standard HTTP GET/HEAD response returning the list of files as a HTML. Serving the HTML file as httpd/unix-directory makes no sense. It breaks the webbrowser pointed at that URL, while WebDAV clients would nevertheless use the WebDAV method PROPFIND and not the GET/HEAD.

[andrewleech]

Ah ok, that makes sense. I didn't actually realise the server would serve "regular" http requests as well as webdav ones.

[andrewleech]

I gather this index file handing is quite non-standard for webdav servers?

Does this mean that if it's enabled and you try to view a folder that's got an index style file in it, you get served that file rather than the directory listing?

by default is disabled

Looking at the code, is an end user expected to edit this python file to add filenames to index_files = () above?
Or have to create a wrapper script that imports this, injects the names, then start the server programatically?
For this to be included I think it'd need at least some docs, if not a command line argument to enable?

[jaboja]

1. WebDAV allows that behavior:
   http://www.webdav.org/specs/rfc4918.html#rfc.section.9.4

2. I intended it to be extended via inheritance, like this:

class CustomFilesystemHandler(FilesystemHandler):
    mimecheck = True
    index_files = ("index.html",)

However I see it would be better if I write some docs for that feature.

[andrewleech]

This would be safer as:

Suggested change

	yield buf if isinstance(buf, six.binary_type) else str(buf).encode('utf-8')
	yield buf if isinstance(buf, six.binary_type) else bytes(buf, 'utf-8')

Using str(buf) could cause an undesired output if buf happens to be something that doesn't implicitely convert to the str cleanly.

[jaboja]

Maybe we should just throw exception in such case? This should be str or bytes anyway.

[andrewleech]

Well I don't think it's an expected error, throwing an exception is ugly useful if you're going to catch and deal with it somewhere.

Without tracing the code through I could see it potentially being a bytearray, or a future change might want to change it to one...

This was more of a best practices suggestion, if you want to convert the object to a bytes it's better to convert in one step, rather than convert to str first and then to bytes.