Automated checks for formatting and styleguide in the documentation

[lb-]

Added as a discussion and not directly a new issue as we do not know if we want this enhancement just yet.

Context

• Earlier in 2022 we introduced prettier into the codebase but intentionally left out .md files.
• The reasoning at the time was that it makes it harder for first time contributors to add to the documentation as they may easily trigger linting errors but not be able to run the code locally (e.g. may be editing in the GitHub UI) or may struggle with getting node tooling running.
• More recently some clean up work has been done and it was found that some common mistakes and non-trivial inconsistency has been introduced as more Markdown documentation has been introduced.

Examples of issues Prettier can fix

• Double backticks (when converting from RST, it is easy to forget that only one backtick is required around inline code snippets, while this does not visibly break things it can cause confusion.
• Incorrect closing of triple backticks and nested backticks code snippets, easy to miss but the formatter fixes these without issues.
• Unintended whitespace - not a critical issue but can cause confusion and odd diffs between changes based on whether the developer was contributing via Github UI or windows/mac code editors.
• Malformed HTML in code snippets - Prettier is not a linter but does pick up and fix up small issues if found.
• EOF line breaks - no more issues where these switch around back and forth based on whether the developer was using a different setting locally.
• Consistency - this is obviously the main win for Prettier, no matter who edits the markdown files the same line breaks, italics modifier, spacing, formatting etc are all enforced which make it easier to switch between files and not have to 'think' about these things.

Examples of problems that this would cause

• Adds friction for new contributors - the build will fail for any change that does not align with the formatter which can be quite confusing for new developers (although, very new developers often ignore the CI failures).
  • Although, it is quite easy for the core team to run the formatter over the files once checking out locally and fixing before merging.
• Prettier is opinionated about things like list items (adding spacing before the list content starts) and heading styles (must use ### and not --- underlines) and we may want this to be more flexible.
  • A mitigation would be to just ignore some sub-set of files such as the release notes in the documentation.

Links

• documentation - markdown formatting #8602
• auto-format core markdown files (e.g. readme) #8603

[zerolab]

I am in favour of keeping things tidy and consistent. It does become a question of when we should add this - now, or after all docs have been converted to Markdown.

As an aside, we should encourage people to use pre-commit - it can help take one of the cons away

[lb-]

I have renamed this discussion to be a bit more broad.

[Stormheg]

Linking #9225 here as an example of a regression fix that could have been prevented if we would have had some kind of formatter / linter in place.
In this particular case it is a code example that no longer conforms to the PEP8 code style, which is not something Prettier can help us with I think.

I'm sure there exist tools / plugins to format / lint code blocks in documentation but I can't seem to find them through a quick google search, so maybe I'm mistaken and no such tools exist... or my google-fu search skills fail me today, the latter being more likely 😝

[zerolab]

Perhaps https://github.com/asottile/blacken-docs could help with that

[lb-]

Another tool that may help a lot, including for enforcing spelling of things, avoiding some text like etc. and even basic grammar checks.

https://textlint.github.io/

[thibaudcolas]

I’d be up for this if it can be set up in a way that people contributing via the GitHub UI can be linked to any CI failures, and see obvious guidance on how to fix any issues there.

In the case of Prettier in particular – this would mean setting it up with a diff of any changes, working around the lack of support for diffing in the --check output: prettier/prettier#6885.

[lb-]

#10597 - PR up, feedback or thoughts welcome.