What is the practical difference between `-preview` and `-stub-run`

[trym-b]

Hi

One of the issues we are facing is very long running workflows, which can run for hours or days. So when we are making changes to our nextflow workflow signatures, we would like to get more timely feedback from our automatic nextflow runs in our continuous integration system.

So we wondered if there is some kind of dry-run option for evaluating these signatures and call sites, and we found 2 options: nextflow run -preview and nextflow run -stub-run.

For the -preview flag, this is the description: Run the workflow script skipping the execution of all processes.
For the -stub-run flag, this is the description: Whether to replace scripts with command stubs when executing the run.

When running with either of these options, the total runtime of one of the very time-consuming workflows becomes 18 seconds. So in practice, what is the difference between these options? Will our millage vary if we go for -preview vs -stub-run?

[bentsherman]

It's difficult to develop a true "dry run" for Nextflow because you don't know from the beginning how many tasks will be executed. So instead there are multiple related modes

Preview mode isn't really a dry run, it's more a way to inspect / test the workflow DAG that is constructed at the beginning. It's a good way to test most of your pipeline code without actually running the pipeline, but I suspect it will be fully replaced in time by the inspect and lint commands

The stub run is sort of like a dry run and is probably what you should use to test as much of your pipeline code as possible. It allows you to stub the task script, so the stub run should include everything from a normal run except for the actual command-line tools.

I have also experimented with a so-called "dry run" in #4214 but that is mainly about testing the impact of a code change on a resumed run, i.e. if I make change X to my code how far will the pipeline get before re-executing. But I haven't merged this yet