fix/prevent falco main module to process an empty fastq and fail#5769
Conversation
…s-an-empty-fastq-file
…process-an-empty-fastq-file
…s-an-empty-fastq-file
…process-an-empty-fastq-file
jfy133
left a comment
There was a problem hiding this comment.
I'm not sure I like this. The process should just silently not execute when it gets an empty FASTQ file.
An empty FASTQ file implies something has gone wrong, so the user should be told why the module is not executing.
I would rather leave as is, and pipeline developers should handle this within the pipeline in the input channel to the falco process, such as a map that uses empty() or w/s on the fastq file
Otherwise could put in the script setup block (with prefix variables etc) an error check snd actually error out if it fails
I don't think stuff like the when block should be modified because it is a standard part of the template and would make it harder to modify en masse
I see your concern, it's a solid argument. I was trying to prevent falco from running when an empty fastq.tar.gz is among the {reads}, but this should be done at the pipeline level. I was doing it in the pipeline, but PR: 5720 refactoring of the subworkflow lead me to think of applying it here. I will close this PR then, and look for alternative routes. Thank you very much for your valuable review and observation. |
|
I see. Yes, in the case I do feel it makes more sense at a subworkflow level! |
|
Does it currently crash with a clear error message? |
PR checklist
Closes #5612
Replaces: #5638
To be merge on top-off of PR: #5720
@k1sauce What about doing the filter directly on the falco module main.nf. This way the subworkflow/bcl_demultiplex/main.nf does not have to handle this issue nor the workflow/demultiplex.nf
I have used a file size filter, to avoid inserting in the falco list of ${reads} those files that have less than 20bytes.
Rationale: An empty text file compressed in tar.gz is usually 20 bytes in size.
edited:
(task.ext.when == null || task.ext.when) && reads.every { it.size() > 20 }Closes #5638
versions.ymlfile.labelnf-core modules test <MODULE> --profile dockernf-core modules test <MODULE> --profile singularitynf-core modules test <MODULE> --profile condanf-core subworkflows test <SUBWORKFLOW> --profile dockernf-core subworkflows test <SUBWORKFLOW> --profile singularitynf-core subworkflows test <SUBWORKFLOW> --profile conda