You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The tsv_join wdl task is intended to accept gzipped tsv inputs to simplify handling of nextmeta files in their native state. But it doesn't work:
File "/opt/viral-ngs/source/file_utils.py", line 208, in tsv_join
header.extend(reader.fieldnames)
File "/opt/miniconda/envs/viral-ngs-env/lib/python3.7/csv.py", line 98, in fieldnames
self._fieldnames = next(self.reader)
File "/opt/miniconda/envs/viral-ngs-env/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
2021/01/27 03:06:54 Starting delocalization.
Probably because the util.file.open_or_gzopen in viral-core might not be opening in text mode properly, etc...
Current workaround is to just pass decompressed tsvs in.
Tangentially, while fixing this, we might want to consider just moving the relatively small python bits into the wdl task itself and using a smaller docker container (like python:slim instead of viral-core) for reduced spinup times on an otherwise simple task.
The text was updated successfully, but these errors were encountered:
The tsv_join wdl task is intended to accept gzipped tsv inputs to simplify handling of nextmeta files in their native state. But it doesn't work:
Probably because the
util.file.open_or_gzopen
in viral-core might not be opening in text mode properly, etc...Current workaround is to just pass decompressed tsvs in.
Tangentially, while fixing this, we might want to consider just moving the relatively small python bits into the wdl task itself and using a smaller docker container (like
python:slim
instead ofviral-core
) for reduced spinup times on an otherwise simple task.The text was updated successfully, but these errors were encountered: