Documentation

[mrx23dot]

readme.md doesn't mention

• what happens when encoding arbitrary byte array (eg. utf8 with transmission errors)
• will unishox2_decompress_simple still be able to recover original byte array?
• Can decompress_simple take arbitrary/random input without crashing? (or input needs to be checksum checked)
• Is it guaranteed that len(compressed data) <= len(original)?
• API doesn't describe the return values in code.

[siara-cc]

Replying here for now:

• what happens when encoding arbitrary byte array (eg. utf8 with transmission errors)
  It will consider them as binary characters and encode them, but compression ratio will reduce.

• will unishox2_decompress_simple still be able to recover original byte array?
  It will go as per the rule book and the decompressed out may have some unreadable parts

• Can decompress_simple take arbitrary/random input without crashing? (or input needs to be checksum checked)
  The C Library won't write past the length of the buffer passed, so the input needs to checksum checked only if perfection is needed. The Javascript library would throw a runtime exception and the buffer will have whatever was decoded.

• Is it guaranteed that len(compressed data) <= len(original)?
  Yes, in most real life scenarios. But if you expect weird sequences in input or garbled input, please allocate extra safety buffer for compression and decompression.

• API doesn't describe the return values in code.
  I will add them. But for C API, both compress and decompress return the number of bytes. For JS API, the compress functions return number of bytes of compressed output and the decompress function may return the output string or number of bytes of decompressed output based on choice specified in input.

[mrx23dot]

Thanks for the clarification!
So far it works great.

[siara-cc]

• The C Library won't write past the length of the buffer passed,

This applies only if the library is compiled with -DUNISHOX_API_WITH_OUTPUT_LEN=1 and unishox2_(de)compress() is used with an additional parameter of provided buffer length. Otherwise it is always assumed that the caller provides enough buffer.
This is not the default to make it faster.
I missed mentioning this earlier.