Fix/prevent falco to process empty fastq files#5731
Closed
glichtenstein wants to merge 44 commits intonf-core:5612-bcl-demultiplexfrom
Closed
Fix/prevent falco to process empty fastq files#5731glichtenstein wants to merge 44 commits intonf-core:5612-bcl-demultiplexfrom
glichtenstein wants to merge 44 commits intonf-core:5612-bcl-demultiplexfrom
Conversation
* port to nf-test * update tests to include versions
* Add missing tximport outputs * Update snapshot for tximeta/tximport
Replace multiple ifs with a single for The loop checks if files exist before moving them to temp/
* port to nf-test * update snap * update tests to include versions * port to nf-test * update snap * update tests to include versions * change snapshot to read the dict * check file md5sum * Apply suggestions from code review * update snap --------- Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io>
* port to nf-test * include versions in test * port to nf-test * include versions in test * change to check tables * change tests to assert whole output and update snap * generalize assertions
* add index * add bwameme_mem * Update modules/nf-core/bwameme/index/tests/main.nf.test Co-authored-by: Annick Renevey <47788523+rannick@users.noreply.github.com> --------- Co-authored-by: Annick Renevey <47788523+rannick@users.noreply.github.com>
* use conda-forge instead of anaconda * Update modules/nf-core/homer/maketagdirectory/environment.yml Co-authored-by: Phil Ewels <phil.ewels@seqera.io> * nf-test homer/findpeaks * nf-test homer/maketagdirectory * nf-test homer/pos2bed * fix perl version? * Update modules/nf-core/gawk/environment.yml Co-authored-by: Phil Ewels <phil.ewels@seqera.io> * fix gawk --------- Co-authored-by: Phil Ewels <phil.ewels@seqera.io>
The new container version no longer requires overriding the entrypoint.
* Added stub for sratools * Added command echo for fasterqdump
* Version bumped bwa/index and bwa/aln * Removed bam md5sum from snapshot
* Bump bwa mem version * Most recent htslib
* add mmseqs_contig_taxonomy * apply prettier * code linting * changed the test dataset to bacteroides_fragilis instead of influenzae * give more memory to mmseqs-taxa * memory increased to 16.GB * increase the process mem requirements * fix memory * add reviewer suggestion * change back memory * fix snap * fix snap
…-core#5598) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…f-core#5636) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* add rrnatransctipts * add util for extracting ribosomal transcripts * fix tests * fix tests * mixup with ucsc gtftogenepred * mixup with ucsc gtftogenepred * mixup with ucsc gtftogenepred * make output optional * fill meta * reset gtftogenepred * use grep instead of template * fix * fix test so if grep does not find anything, the exit code is still 0 --------- Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>
…ow (nf-core#5628) * Return primary bam and index file in fastq_align_bamcmp_bwa subworkflow and update snapshots. * Add new tests and update snapshots. * Remove opt out for conda for fastq_align_bamcmp_bwa * Update snapshots * Restore opt out for conda for fastq_align_bamcmp_bwa --------- Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>
* Author: @MartaSantanaSilva Initial pull for module grabix/check Addition of one test (total: 1) * Update modules/nf-core/grabix/check/main.nf Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * Update modules/nf-core/grabix/check/main.nf Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * Update modules/nf-core/grabix/check/main.nf Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * Update modules/nf-core/grabix/check/meta.yml Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * new module: grabix Fixes nf-core#5181 * Apply suggestions from code review * Update modules/nf-core/grabix/check/meta.yml * fix conda declaration * update snapshot * prettier --------- Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io> Co-authored-by: maxulysse <max.u.garcia@gmail.com>
Parabricks doesn't require stageInMode to be 'copy' Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io>
* chore(deps): update biocontainers/glimpse-bio docker tag to v2.0.1 * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * The channel was renamed in nf-core#5117 * The Conda yml files need to match the container definitions * Typo ! There was a missing s --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Louis LE NEZET <58640615+LouisLeNezet@users.noreply.github.com> Co-authored-by: Matthieu Muffato <mm49@sanger.ac.uk>
…core#5653) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Bump csvtk new version * Add nf-test * Get rid of pytest * Bump version in singularity images * Update tests/config/pytest_modules.yml --------- Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com>
* Stub and nf-test for fastk/fastk * Added chmod +r to add read permissions for docker * Added a+r
* Remove leftover of previous PR * Bump version 0.30.0 of csvtk * Ad nf-test for csvtk/split * Remove csvtk/split from pytest_modules.yml * Make lint happy * Update path on tests * Update modules/nf-core/csvtk/split/tests/main.nf.test * Update modules/nf-core/csvtk/split/tests/main.nf.test
…f-core#5662) * Remove cp operation from bclconvert Replaces cp operation with native Nextflow output path handling. Fixes nf-core#5644 * Update modules/nf-core/bclconvert/main.nf
* working on the main.nf and meta.yml initially * working on the main.nf and meta.yml initially * Adding the updates to meta.yml * Adding the latest code changes that add a working version of the tests * Fixing the version mismatch in the docker and singularity containers * Removing the trailing whitespaces * Bump parabricks 4.3.0-1 + remove stageInMode (nf-core#5646) Parabricks doesn't require stageInMode to be 'copy' Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * glimpse2 version bump - abandoned (nf-core#4236) * chore(deps): update biocontainers/glimpse-bio docker tag to v2.0.1 * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * The channel was renamed in nf-core#5117 * The Conda yml files need to match the container definitions * Typo ! There was a missing s --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Louis LE NEZET <58640615+LouisLeNezet@users.noreply.github.com> Co-authored-by: Matthieu Muffato <mm49@sanger.ac.uk> * Update pre-commit hook renovatebot/pre-commit-hooks to v37.371.1 (nf-core#5653) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Bump csvtk version to CSVTK/JOIN and add nf-test (nf-core#5642) * Bump csvtk new version * Add nf-test * Get rid of pytest * Bump version in singularity images * Update tests/config/pytest_modules.yml --------- Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> * Added task.memory to meryl modules and .meryl ext for outputs (nf-core#5655) * Stub and nf-test for fastk/fastk (nf-core#5631) * Stub and nf-test for fastk/fastk * Added chmod +r to add read permissions for docker * Added a+r * Bump csvtk version to CSVTK/SPLIT and add nf-test (nf-core#5656) * Remove leftover of previous PR * Bump version 0.30.0 of csvtk * Ad nf-test for csvtk/split * Remove csvtk/split from pytest_modules.yml * Make lint happy * Update path on tests * Update modules/nf-core/csvtk/split/tests/main.nf.test * Update modules/nf-core/csvtk/split/tests/main.nf.test * Update modules/nf-core/viennarna/rnafold/tests/main.nf.test Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> * Adding the corrected version of meta.yml file --------- Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> Co-authored-by: Marcel Ribeiro-Dantas <marcel.ribeirodantas@seqera.io> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Louis LE NEZET <58640615+LouisLeNezet@users.noreply.github.com> Co-authored-by: Matthieu Muffato <mm49@sanger.ac.uk> Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com> Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> Co-authored-by: Usman Rashid <usman@smme.edu.pk>
* Add nf-test to chromap/index * Updated snapshot and removed old tests * Fix chromap versions and add tags * [automated] Fix linting with Prettier * Check genome.index exists instead of match Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com> * snapshot fix * Updated snapshot * Add version to snapshot * Change version to versions in snapshot --------- Co-authored-by: nf-core-bot <core@nf-co.re> Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>
* Added nf-test for gatk4/indexfeaturefile * Deleted pytest gatk4/indexfeaturefile tests
* Add tests for Salsa * Update test file * Delete unused test file * Resolve linting error * Remove unused params section * Update modules/nf-core/salsa2/tests/main.nf.test Co-authored-by: Sateesh_Peri <33637490+sateeshperi@users.noreply.github.com> * Use snapshot and match methods * Update snapeshot, compare files * Remove pytests --------- Co-authored-by: James Kaptuch <jkaptuch@ambrygen.com> Co-authored-by: Sateesh_Peri <33637490+sateeshperi@users.noreply.github.com>
* biscuit/align nf-test and snapshot * Update modules/nf-core/biscuit/align/tests/main.nf.test * remove bam from snapshots * remove bam,bai from snapshots --------- Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>
* Adding the partially working version of RNAcofold code * Adding the rnacofold tests and main * Removing comments from the env.yml file * Update modules/nf-core/viennarna/rnacofold/main.nf Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> * Update modules/nf-core/viennarna/rnacofold/tests/main.nf.test Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> * Adding the meta to track sample information --------- Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com>
* Added nf-test for gatk4/intervallisttools * Update modules/nf-core/gatk4/intervallisttools/tests/main.nf.test Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com> * Added nf-test stub for gatk4/intervallisttools * Updated snap nf-test stub for gatk4/intervallisttools --------- Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>
* working on the main.nf and meta.yml initially * working on the main.nf and meta.yml initially * Adding the updates to meta.yml * Adding the latest code changes that add a working version of the tests * Fixing the version mismatch in the docker and singularity containers * Removing the trailing whitespaces * Bump parabricks 4.3.0-1 + remove stageInMode (nf-core#5646) Parabricks doesn't require stageInMode to be 'copy' Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * glimpse2 version bump - abandoned (nf-core#4236) * chore(deps): update biocontainers/glimpse-bio docker tag to v2.0.1 * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * Update main.nf * The channel was renamed in nf-core#5117 * The Conda yml files need to match the container definitions * Typo ! There was a missing s --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Louis LE NEZET <58640615+LouisLeNezet@users.noreply.github.com> Co-authored-by: Matthieu Muffato <mm49@sanger.ac.uk> * Update pre-commit hook renovatebot/pre-commit-hooks to v37.371.1 (nf-core#5653) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Bump csvtk version to CSVTK/JOIN and add nf-test (nf-core#5642) * Bump csvtk new version * Add nf-test * Get rid of pytest * Bump version in singularity images * Update tests/config/pytest_modules.yml --------- Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> * Added task.memory to meryl modules and .meryl ext for outputs (nf-core#5655) * Stub and nf-test for fastk/fastk (nf-core#5631) * Stub and nf-test for fastk/fastk * Added chmod +r to add read permissions for docker * Added a+r * Bump csvtk version to CSVTK/SPLIT and add nf-test (nf-core#5656) * Remove leftover of previous PR * Bump version 0.30.0 of csvtk * Ad nf-test for csvtk/split * Remove csvtk/split from pytest_modules.yml * Make lint happy * Update path on tests * Update modules/nf-core/csvtk/split/tests/main.nf.test * Update modules/nf-core/csvtk/split/tests/main.nf.test * Update modules/nf-core/viennarna/rnafold/tests/main.nf.test Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> * Adding the working version of ViennaRNA/RNALfold module * Adding the corrected version of meta.yml file * Removing trailing whitespaces in all files * Removing the type with backtick * Removing the whitespace from rnafold main.nf * Removing the viennarna/rnafold and removing the comments and --- from env.yml * Update modules/nf-core/viennarna/rnalfold/main.nf Co-authored-by: Florian Wuennemann <flowuenne@gmail.com> * Update modules/nf-core/viennarna/rnalfold/main.nf Co-authored-by: Florian Wuennemann <flowuenne@gmail.com> * Update modules/nf-core/viennarna/rnalfold/main.nf Co-authored-by: Florian Wuennemann <flowuenne@gmail.com> * Update modules/nf-core/viennarna/rnalfold/tests/main.nf.test Co-authored-by: Florian Wuennemann <flowuenne@gmail.com> * Update modules/nf-core/viennarna/rnalfold/tests/main.nf.test Co-authored-by: Florian Wuennemann <flowuenne@gmail.com> * Update modules/nf-core/viennarna/rnalfold/tests/nf-test.config Co-authored-by: Florian Wuennemann <flowuenne@gmail.com> * Deleting nf-test.config for viennarna/rnalfold as per suggested * Adding the correct test assertions and snapshots --------- Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> Co-authored-by: Marcel Ribeiro-Dantas <marcel.ribeirodantas@seqera.io> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Louis LE NEZET <58640615+LouisLeNezet@users.noreply.github.com> Co-authored-by: Matthieu Muffato <mm49@sanger.ac.uk> Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com> Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> Co-authored-by: Usman Rashid <usman@smme.edu.pk> Co-authored-by: Florian Wuennemann <flowuenne@gmail.com>
* Add checkm/lineagewf nf tests. * Remove files contain timestamp from snapshot * Remove pytest files * Delete checkm_lineagewf entry in pytest_modules.yml * Apply suggestions from code review Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com> --------- Co-authored-by: Wei-An Chen <wchen@VEDANTABIO.COM> Co-authored-by: Sateesh_Peri <33637490+sateeshperi@users.noreply.github.com> Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io> Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com>
* added module 10xbamtofastq * remove extra test file * Update main.nf.test * Update main.nf * Rename module and fix tests --------- Co-authored-by: Harshil Patel <drpatelhh@gmail.com> Co-authored-by: Harshil Patel <drpatelh@users.noreply.github.com>
* Added nf-test stub for gatk4/variantfiltration * Update for lint * Update for lint
* Add nf-test to chromap/chromap * Add nf-test to chromap/chromap (tags added) * Update modules/nf-core/chromap/chromap/tests/main.nf.test * Update modules/nf-core/chromap/chromap/tests/tags.yml * Update pytest_modules.yml * Update main.nf.test * Update main.nf.test.snap * Update main.nf.test * Update main.nf.test.snap --------- Co-authored-by: Maxime U Garcia <maxime.garcia@seqera.io> Co-authored-by: Maxime U Garcia <max.u.garcia@gmail.com>
… for merqury/hapmers (nf-core#5657) * Migrated to nf-test * Moved to merqury/merqury * Fixed test issues * Moved pngs to name check * Added version warn and trio mode test
* Swap ngscheckmate/ncm to nftest * Update main.nf.test
* Initial work towards a module for RTN TNI Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Make it a simple PoC Tests are passing. From now on, I can make it more complex and closer to what I want this module to do in the end. * Add two more steps in the module * Prettify file * Use test data, add remaining steps/collect output in TNI Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Fix DOI in meta.yml Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Some ongoing changes Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Rename rtn_tni.R to rtn_tni.r * Fix snapshot mismatch issue Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Add --n_permutations + tests Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Get rid of the TODOs Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Use different dataset (real gene ids) Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Add support to --tfs + tests Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Fix indentation for ECLint check Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> * Remove quotes from environment dependencies for conda Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io> --------- Signed-off-by: Marcel Ribeiro-Dantas <mribeirodantas@seqera.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR checklist
What about doing the filter directly on the falco module main.nf. This way the subworkflow/bcl_demultiplex/main.nf does not have to handle this issue nor the workflow/demultiplex.nf
I have used a file size filter, to avoid inserting in the falco list of ${reads} those files that have less than 20bytes.
Rationale: An empty text file compressed in tar.gz is usually 20 bytes in size.
Closes #5638
versions.ymlfile.labelnf-core modules test <MODULE> --profile dockernf-core modules test <MODULE> --profile singularitynf-core modules test <MODULE> --profile condanf-core subworkflows test <SUBWORKFLOW> --profile dockernf-core subworkflows test <SUBWORKFLOW> --profile singularitynf-core subworkflows test <SUBWORKFLOW> --profile conda