Skip to content

Commit

Permalink
Add sphinx, myst, and the theme used in pandas docs
Browse files Browse the repository at this point in the history
  • Loading branch information
kinow committed Apr 17, 2022
1 parent 4ecdd01 commit 2be017b
Show file tree
Hide file tree
Showing 119 changed files with 902 additions and 8,772 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ _site
.Rhistory
.RData

_build/
6 changes: 4 additions & 2 deletions _episodes/01-introduction.md → 01-introduction/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
title: "Introduction"
teaching: 0
exercises: 0
questions:
Expand All @@ -14,6 +13,8 @@ keypoints:
- "Descriptions in CWL aid portability between environments"
---

# Introduction

CWL is a way to describe command line tools and connect them together to create
workflows. Because CWL is a specification and not a specific piece of
software, tools and workflows described using CWL are portable across a variety
Expand All @@ -29,4 +30,5 @@ vendors. CWL is well suited for describing large-scale workflows in cluster,
cloud and high performance computing environments where tasks are scheduled in
parallel across many nodes.

{% include links.md %}
```{include} ../_includes/links.md
```
52 changes: 25 additions & 27 deletions _episodes/02-1st-example.md → 02-1st-example/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
title: "First Example"
teaching: 5
exercises: 0
questions:
Expand All @@ -13,35 +12,38 @@ keypoints:
- "Input values are specified in a separate YAML file."
- "The tool description and input files are provided as arguments to a CWL runner."
---

# First Example

The simplest "hello world" program. This accepts one input parameter, writes a message to the terminal or job log, and produces
no permanent output.
CWL documents are written in [JSON][json] or [YAML][yaml], or a mix of the two.
We will use YAML throughout this guide.
If you are not familiar with YAML,
you may find it helpful to refer to
[this quick tutorial for the subset of YAML used in CWL]({{ page.root }}{% link _extras/yaml.md %}).
[this quick tutorial for the subset of YAML used in CWL](/yaml/index.md).

First, create a file called `1st-tool.cwl`, containing the boxed text below. It will help you to use a text editor that can be
specified to produce text in YAML or JSON. Whatever text editor you use, the indents you see should not be created using tabs.

*1st-tool.cwl*
~~~
{% include cwl/02-1st-example/1st-tool.cwl %}
~~~
{: .source}

```{literalinclude} /_includes/cwl/02-1st-example/1st-tool.cwl
:language: yaml
```

Next, create a file called `echo-job.yml`, containing the following boxed text, which will describe the input of a run:

*echo-job.yml*
~~~
{% include cwl/02-1st-example/echo-job.yml %}
~~~
{: .source}

```{literalinclude} /_includes/cwl/02-1st-example/echo-job.yml
:language: yaml
```

Now, invoke `cwl-runner` with the tool wrapper `1st-tool.cwl` and the input object echo-job.yml on the command line. The command
is `cwl-runner 1st-tool.cwl echo-job.yml`. The boxed text below shows this command and the expected output.

~~~
```bash
$ cwl-runner 1st-tool.cwl echo-job.yml
[job 1st-tool.cwl] /tmp/tmpmM5S_1$ echo \
'Hello world!'
Expand All @@ -50,43 +52,39 @@ Hello world!
{}
Final process status is success

~~~
{: .output}
```

The command `cwl-runner 1st-tool.cwl echo-job.yml` is an example of a general form that you will often come across while using
CWL. The general form is `cwl-runner [tool-or-workflow-description] [input-job-settings]`

What's going on here? Let's break down the contents of `1st-tool.cwl`:

~~~
```yaml
cwlVersion: v1.0
class: CommandLineTool
~~~
{: .source}
```
The `cwlVersion` field indicates the version of the CWL spec used by the document. The `class` field indicates this document
describes a command line tool.

~~~
```yaml
baseCommand: echo
~~~
{: .source}
```

The `baseCommand` provides the name of program that will actually run (`echo`). `echo` is a built-in program in the bash and
C shells.

~~~
```yaml
inputs:
message:
type: string
inputBinding:
position: 1
~~~
{: .source}
```

The `inputs` section describes the inputs of the tool.
This is a mapped list of input parameters
(see the [YAML Guide]({{ page.root }}{% link _extras/yaml.md %}#maps) for more about the format)
(see the [YAML Guide](/yaml/index.md) for more about the format)
and each parameter includes an identifier,
a data type,
and optionally an `inputBinding`.
Expand All @@ -95,13 +93,13 @@ on the command line.
In this example,
the `position` field indicates where it should appear on the command line.

~~~
```yaml
outputs: []
~~~
{: .source}
```

This tool has no formal output, so the `outputs` section is an empty list.

[echo]: http://www.linfo.org/echo.html

{% include links.md %}
```{include} ../_includes/links.md
```
33 changes: 15 additions & 18 deletions _episodes/03-input.md → 03-input/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
title: "Essential Input Parameters"
teaching: 10
exercises: 0
questions:
Expand All @@ -14,6 +13,8 @@ keypoints:
appears in the command."
---

# Essential Input Parameters

The `inputs` of a tool is a list of input parameters that control how to
run the tool. Each parameter has an `id` for the name of parameter, and
`type` describing what types of values are valid for that parameter.
Expand All @@ -30,19 +31,17 @@ First, create a file called inp.cwl, containing the following:

*inp.cwl*

~~~
{% include cwl/03-input/inp.cwl %}
~~~
{: .source}
```{literalinclude} /_includes/cwl/03-input/inp.cwl
:language: yaml
```

Create a file called inp-job.yml:

*inp-job.yml*

~~~
{% include cwl/03-input/inp-job.yml %}
~~~
{: .source}
```{literalinclude} /_includes/cwl/03-input/inp-job.yml
:language: yaml
```

Notice that "example_file", as a `File` type, must be provided as an
object with the fields `class: File` and `path`.
Expand All @@ -63,14 +62,15 @@ $ cwl-runner inp.cwl inp-job.yml
{}
Final process status is success
~~~
{: .output}
> ## Where did those `/tmp` paths come from?

```{note}
> <p class="rubric">Where did those `/tmp` paths come from?</p>
>
> The CWL reference runner (cwltool) and other runners create temporary
> directories with symbolic ("soft") links to your input files to ensure that
> the tools aren't accidentally accessing files that were not explicitly
> specified
{: .callout}
```

The field `inputBinding` is optional and indicates whether and how the
input parameter should be appear on the tool's command line. If
Expand All @@ -84,7 +84,6 @@ example_flag:
position: 1
prefix: -f
~~~
{: .source}

Boolean types are treated as a flag. If the input parameter
"example_flag" is "true", then `prefix` will be added to the
Expand All @@ -97,7 +96,6 @@ example_string:
position: 3
prefix: --example-string
~~~
{: .source}

String types appear on the command line as literal values. The `prefix`
is optional, if provided, it appears as a separate argument on the
Expand All @@ -112,7 +110,6 @@ example_int:
prefix: -i
separate: false
~~~
{: .source}

Integer (and floating point) types appear on the command line with
decimal text representation. When the option `separate` is false (the
Expand All @@ -128,7 +125,6 @@ example_file:
separate: false
position: 4
~~~
{: .source}

File types appear on the command line as the path to the file. When the
parameter type ends with a question mark `?` it indicates that the
Expand All @@ -138,7 +134,7 @@ parameter were not provided in the input, nothing would appear on the
command line.

Input files are read-only. If you wish to update an input file, you must
[first copy it to the output directory]({{ page.root }}/15-staging/).
[first copy it to the output directory](/15-staging/index.md).

The value of `position` is used to determine where parameter should
appear on the command line. Positions are relative to one another, not
Expand All @@ -151,4 +147,5 @@ is optional. The default position is 0.
The `baseCommand` field will always appear in the final command line before the parameters.

[touch]: http://www.linfo.org/touch.html
{% include links.md %}
```{include} ../_includes/links.md
```
28 changes: 14 additions & 14 deletions _episodes/04-output.md → 04-output/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
title: "Returning Output Files"
teaching: 10
exercises: 0
questions:
Expand All @@ -13,6 +12,9 @@ keypoints:
output parameter."
- "Wildcards are allowed in the `glob` field."
---

# Returning Output Files

The `outputs` of a tool is a list of output parameters that should be
returned after running the tool. Each parameter has an `id` for the name
of parameter, and `type` describing what types of values are valid for
Expand All @@ -26,26 +28,25 @@ themselves, or come from examining the content of those files.

The following example demonstrates how to return a file that has been extracted from a tar file.

> ## Passing mandatory arguments to the `baseCommand`
```{note}
> <p class="rubric">Passing mandatory arguments to the `baseCommand`</p>
>
> In previous examples, the `baseCommand` was just a string, with any arguments passed as CWL inputs.
> Instead of a single string we can use an _array of strings_. The first element is the command to run, and
> any subsequent elements are mandatory command line arguments
{: .callout}
```

*tar.cwl*

~~~
{% include cwl/04-output/tar.cwl %}
~~~
{: .source}
```{literalinclude} /_includes/cwl/04-output/tar.cwl
:language: yaml
```

*tar-job.yml*

~~~
{% include cwl/04-output/tar-job.yml %}
~~~
{: .source}
```{literalinclude} /_includes/cwl/04-output/tar-job.yml
:language: yaml
```

Next, create a tar file for the example and invoke `cwl-runner` with the tool
wrapper and the input object on the command line:
Expand All @@ -69,7 +70,6 @@ $ cwl-runner tar.cwl tar-job.yml
}
Final process status is success
~~~
{: .output}

The field `outputBinding` describes how to set the value of each
output parameter.
Expand All @@ -81,9 +81,9 @@ outputs:
outputBinding:
glob: hello.txt
~~~
{: .source}

The `glob` field consists of the name of a file in the output directory.
If you don't know name of the file in advance, you can use a wildcard pattern like `glob: '*.txt'`.

{% include links.md %}
```{include} ../_includes/links.md
```
22 changes: 11 additions & 11 deletions _episodes/05-stdout.md → 05-stdout/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
title: "Capturing Standard Output"
teaching: 10
exercises: 0
questions:
Expand All @@ -10,23 +9,24 @@ keypoints:
- "Use the `stdout` field to specify a filename to capture streamed output."
- "The corresponding output parameter must have `type: stdout`."
---

# Capturing Standard Output

To capture a tool's standard output stream, add the `stdout` field with
the name of the file where the output stream should go. Then add `type:
stdout` on the corresponding output parameter.

*stdout.cwl*

~~~
{% include cwl/05-stdout/stdout.cwl %}
~~~
{: .source}
```{literalinclude} /_includes/cwl/05-stdout/stdout.cwl
:language: yaml
```

*echo-job.yml*

~~~
{% include cwl/05-stdout/echo-job.yml %}
~~~
{: .source}
```{literalinclude} /_includes/cwl/05-stdout/echo-job.yml
:language: yaml
```

Now invoke `cwl-runner` providing the tool wrapper and the input object
on the command line:
Expand All @@ -49,5 +49,5 @@ $ cwl-runner stdout.cwl echo-job.yml
Final process status is success
~~~
{: .output}
{% include links.md %}
```{include} ../_includes/links.md
```
Loading

0 comments on commit 2be017b

Please sign in to comment.