Commit a8d131b
feat!: format shell directives using shfmt (#295)
* feat!: format shell directives using shfmt
Add shfmt-py as dependency to format Snakemake shell directives.
Mask variables like {input} to prevent shfmt mangling.
Ignore escaped braces like {{ }} during masking.
Extract and format multiline string literals safely.
Add --no-format-shell CLI flag to opt-out.
Shell_formatter internals are split into focused helpers
(_mask_snakemake_vars, _invoke_shfmt, _unmask_snakemake_vars) to give
future backend swaps a clean seam — see
docs/adr/0001-shell-formatter-distribution.md.
Fixes a bug where format_python_string_literal received literals with a
trailing newline from the parser (a NEWLINE token after the closing
triple-quote), causing re.fullmatch to return the string unchanged and
leaving shell blocks silently unformatted in all real snakefmt runs.
Fix: rstrip the literal before regex matching.
Adds a parametrised robustness suite covering all trailing-whitespace
input shapes, and end-to-end behavioural tests through the public
formatter and CLI.
Closes #170
* docs: add ADR-0001, bugfix plan, and update README for shell formatting
- docs/adr/README.md: index for architecture decision records
- docs/adr/0001-shell-formatter-distribution.md: records the decision to
keep shfmt-py rather than forking or building Go bindings, with concrete
revisit triggers
- docs/plans/shell-formatting-trailing-newline-bugfix.md: working plan for
the trailing-newline bug identified and fixed in the previous commit
- README: new Shell Block Formatting section with before/after example,
opt-out instructions, placeholder masking and invalid-shell notes; updated
TOC, --help block, configuration example (sort_directives, format_shell),
pre-commit rev (v0.10.2 -> v1.1.0), and Recent Changes callout
* feat(lua): expose format_shell option in Neovim plugin
Adds format_shell = nil to config defaults. When set to false, --no-format-shell
is passed to the snakefmt binary, allowing users to disable shell block
formatting from their editor config without touching pyproject.toml.
require("snakefmt").setup({ format_shell = false })
nil (default) passes no flag, deferring to snakefmt's own default (on).
* fix: update format_shell_code to unpack 3-tuple from _mask_snakemake_vars
After the UUID nonce refactor, _mask_snakemake_vars returns (masked, tokens,
originals) but format_shell_code still unpacked a 2-tuple. Update the call
site to match the new signature.
* fix: address PR #295 review comments
- C1: unified two-layer brace masking — {{...}} is now masked before
single {var} placeholders, ensuring brace groups, parameter expansion,
brace expansion, awk patterns, and f-string Snakemake vars all survive
shfmt unchanged
- B2: replace textwrap.indent with _indent_preserving_heredocs, which
skips heredoc body and terminator lines so <<EOF terminators remain at
column 0 as bash requires
- T1/C2: add unit tests for _indent_preserving_heredocs, integration
heredoc tests with concrete expected values, and a long-line
no-spurious-wrap test confirming shfmt's flags are syntactic only
- M1: move format_shell from imperative post-construction attribute into
Formatter.__init__ (default False, matching sort_directives); update
setup_formatter and all TestShellBlockFormatting tests to opt in
explicitly with format_shell=True
- S1: consolidate two re.fullmatch calls into a single compiled
_TRIPLE_QUOTE_RE using "{3}|'{3} to avoid backslash escapes in raw
strings that confuse black's parser
- README: add "Brace groups" section documenting the {{...}} preservation
trade-off and the fmt: off escape hatch
* chore(deps): bump shfmt-py to >=4.0.0,<5.0.0
v4.0.0 bundles shfmt v3.13.1 (latest upstream), decouples the package
version from the shfmt version, adds Renovate for automated future
updates, and falls back to a system shfmt on unsupported platforms.
Updates ADR-0001 to record that the maintenance concerns that prompted
the original decision have been addressed.
* fix: chain CalledProcessError when raising InvalidShell from shfmt
Preserves the original exception context so tracebacks show the full
cause (missing binary, permission error, etc.) rather than only the
InvalidShell message.
* fix(format): address PR review comments on shell formatting
Tighten double-brace regex to match non-greedily.
Broaden heredoc start regex to support custom delimiters like !EOF!.
Update README to explain opaque mask design choice accurately.
* chore: lint
* feat: mask heredoc bodies before shfmt to support non-standard terminators
shfmt requires heredoc terminators at column 0 (or leading tabs for
<<-EOF). Snakemake shell blocks sometimes use escape-prefixed terminators
like \n!EOF!, which shfmt rejects as unclosed heredocs.
Pre-process masked code with _mask_heredocs before invoking shfmt:
detect the user-intended terminator permissively (accepting whitespace
and \X escape prefixes before the delimiter word), replace the body and
terminator with a placeholder, let shfmt format the surrounding shell,
then restore the original body and terminator verbatim via _unmask_heredocs.
shfmt does not reformat heredoc body content anyway, so masking has no
effect on formatting quality. Five new tests cover the reviewer's exact
case, body preservation, multiple heredocs, surrounding shell formatting,
and truly unclosed heredocs that should still raise InvalidShell.
Also updates the README with a dedicated "Heredoc handling" section.
* docs(readme): collapse verbose sections into expandable details blocks
* fix: rename shadowed variable
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* fix: preserve original literal when format_python_string_literal finds no triple-quote match
rstrip was applied before the no-match early return, so single-line
strings and other non-triple-quoted literals were returned trimmed
rather than verbatim. Store the original before rstripping and return
it on the no-match path.
---------
BREAKING CHANGE: shell blocks in rules are now formatted by shfmt by
default. Existing Snakefiles may see whitespace and structure changes in
their shell directives. Use `--no-format-shell` or `format_shell = false` in
`pyproject.toml` to opt out.
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>1 parent 6990f79 commit a8d131b
18 files changed
Lines changed: 1778 additions & 44 deletions
File tree
- docs
- adr
- plans
- lua/snakefmt
- snakefmt
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
46 | 47 | | |
47 | 48 | | |
48 | 49 | | |
49 | | - | |
| 50 | + | |
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
| |||
56 | 57 | | |
57 | 58 | | |
58 | 59 | | |
| 60 | + | |
59 | 61 | | |
60 | 62 | | |
61 | 63 | | |
| |||
233 | 235 | | |
234 | 236 | | |
235 | 237 | | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
236 | 241 | | |
237 | 242 | | |
238 | 243 | | |
| |||
246 | 251 | | |
247 | 252 | | |
248 | 253 | | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
254 | | - | |
255 | | - | |
256 | | - | |
257 | | - | |
258 | | - | |
259 | | - | |
260 | | - | |
261 | | - | |
262 | | - | |
263 | | - | |
264 | | - | |
265 | | - | |
266 | | - | |
267 | | - | |
268 | | - | |
269 | | - | |
270 | | - | |
271 | | - | |
272 | | - | |
273 | | - | |
274 | | - | |
275 | | - | |
276 | | - | |
277 | | - | |
278 | | - | |
279 | | - | |
280 | | - | |
281 | | - | |
282 | | - | |
283 | | - | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
284 | 293 | | |
285 | 294 | | |
| 295 | + | |
| 296 | + | |
286 | 297 | | |
287 | 298 | | |
288 | 299 | | |
289 | 300 | | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
290 | 304 | | |
291 | 305 | | |
292 | 306 | | |
| |||
300 | 314 | | |
301 | 315 | | |
302 | 316 | | |
| 317 | + | |
| 318 | + | |
303 | 319 | | |
304 | 320 | | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
305 | 440 | | |
306 | 441 | | |
307 | 442 | | |
| |||
334 | 469 | | |
335 | 470 | | |
336 | 471 | | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
337 | 475 | | |
338 | 476 | | |
339 | 477 | | |
| |||
383 | 521 | | |
384 | 522 | | |
385 | 523 | | |
| 524 | + | |
| 525 | + | |
386 | 526 | | |
387 | 527 | | |
388 | 528 | | |
| |||
405 | 545 | | |
406 | 546 | | |
407 | 547 | | |
| 548 | + | |
| 549 | + | |
408 | 550 | | |
409 | 551 | | |
410 | 552 | | |
| |||
433 | 575 | | |
434 | 576 | | |
435 | 577 | | |
436 | | - | |
| 578 | + | |
437 | 579 | | |
438 | 580 | | |
439 | 581 | | |
| |||
444 | 586 | | |
445 | 587 | | |
446 | 588 | | |
447 | | - | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
448 | 594 | | |
449 | 595 | | |
450 | 596 | | |
| |||
485 | 631 | | |
486 | 632 | | |
487 | 633 | | |
| 634 | + | |
488 | 635 | | |
489 | 636 | | |
490 | 637 | | |
491 | 638 | | |
492 | 639 | | |
| 640 | + | |
| 641 | + | |
493 | 642 | | |
494 | 643 | | |
495 | 644 | | |
| |||
499 | 648 | | |
500 | 649 | | |
501 | 650 | | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
502 | 654 | | |
503 | 655 | | |
504 | 656 | | |
| |||
512 | 664 | | |
513 | 665 | | |
514 | 666 | | |
| 667 | + | |
| 668 | + | |
515 | 669 | | |
516 | 670 | | |
517 | 671 | | |
| |||
524 | 678 | | |
525 | 679 | | |
526 | 680 | | |
527 | | - | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
528 | 685 | | |
529 | 686 | | |
530 | 687 | | |
| |||
539 | 696 | | |
540 | 697 | | |
541 | 698 | | |
| 699 | + | |
| 700 | + | |
542 | 701 | | |
543 | 702 | | |
544 | 703 | | |
| |||
0 commit comments