Documentation: clarify ideas around usage

[agoose77]

Literary makes some opinionated decisions that might need to be relaxed. See #7 for more.

What are (Literary) literate notebooks?

To understand why one would write entire packages in Jupyter Notebooks, let's identify their benefits:

• REPL for quick prototyping and experimentation.
• Embed rich markup (Markdown) and visualisations
• Magics for improved expressiveness

In Literary, the notebook is generally seen as the "ground truth".

• Documentation is placed adjacent to implementation
• Tests are defined and run in-line.

This makes it easy to show users how the code works without having a separate documentation system, and makes it easier to write docs first.

Nuance of the REPL

Without the REPL, we would only be able to embed rich static content around sections of code. However, Jupyter also has the rich display mechanism for rich-rendering of objects. To use this, we need to leverage a kernel. Although this could be done offline and the results embedded in the notebook, this would strongly undermine the user-friendliness of the paradigm.

Tangle/weave alternatives

Instead of (albeit similar to) tangle and weave, there can be several views of a notebook with certain modifications, e.g.

• Generated code (strip non-exports)
• Documentation (hide exported implementations, generate summary a la https://github.com/fastai/nbdev#a-motivating-example)

Testing

Tests are really just another form of documentation; they ensure that the code is correct, and enforce the API that it must satisfy.

Simple asserts can be placed in-line in the notebook in cells. As they shouldn't be exported, they won't be executed by clients of the module. So how do we rectify this requirement with test-runners like pytest, which provide useful parameterisation features, but require the tests to be encapsulated and called by the runner?

It looks like https://github.com/chmp/ipytest/ might be a good way to resolve this. It should be possible to pass the current module to pytest, but constrain the test functions to those defined in the cell. This might be done by performing an AST search for function nodes defined in the cell, or by inspecting the globals() items that change.

[agoose77]

This is now in the wiki