Skip to content

The verbose reporter is too verbose to be the default. #1442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wlandau opened this issue Mar 3, 2025 · 1 comment
Closed

The verbose reporter is too verbose to be the default. #1442

wlandau opened this issue Mar 3, 2025 · 1 comment
Assignees

Comments

@wlandau
Copy link
Member

wlandau commented Mar 3, 2025

For a large pipeline with fast targets, the verbose reporter prints an overwhelming number of messages. It's difficult to read, and it's even a performance bottleneck:

# _targets.R file
library(targets)
list(
  tar_target(x, seq_len(1e5)),
  tar_target(y, x, pattern = map(x))
)
tar_make()
#> ▶ dispatched target x
#> ● completed target x [0 seconds, 100 bytes]
#> ▶ dispatched branch y_5015dfa5778ec11d
#> ● completed branch y_5015dfa5778ec11d [0 seconds, 47 bytes]
#> ▶ dispatched branch y_5ea9d15a285adf66
#> ● completed branch y_5ea9d15a285adf66 [0.001 seconds, 48 bytes]
#> ▶ dispatched branch y_20a2bb14d64371f8
#> ● completed branch y_20a2bb14d64371f8 [0 seconds, 47 bytes]
#> ▶ dispatched branch y_4cd96100022a8ff3
#> ● completed branch y_4cd96100022a8ff3 [0 seconds, 48 bytes]
#> ▶ dispatched branch y_4540b7c6f3594628
#> ● completed branch y_4540b7c6f3594628 [0.001 seconds, 47 bytes]
#> ▶ dispatched branch y_5ba091e5e95035c4
#> ● completed branch y_5ba091e5e95035c4 [0 seconds, 48 bytes]
#> ▶ dispatched branch y_b946959f8d95e7a5
#> ● completed branch y_b946959f8d95e7a5 [0 seconds, 48 bytes]
#> ▶ dispatched branch y_eedb33cb517a8e92
#> ● completed branch y_eedb33cb517a8e92 [0 seconds, 48 bytes]
#> ▶ dispatched branch y_9077e356c1730e1b
#> ● completed branch y_9077e356c1730e1b [0 seconds, 48 bytes]
#> ▶ dispatched branch y_8b96fd284ce59a59
#> ● completed branch y_8b96fd284ce59a59 [0 seconds, 48 bytes]
#> ▶ dispatched branch y_6e0f3e3742f0ff03
#> ● completed branch y_6e0f3e3742f0ff03 [0 seconds, 48 bytes]
#> ▶ dispatched branch y_aa9d559f1e8c2344
#> ● completed branch y_aa9d559f1e8c2344 [0 seconds, 48 bytes]
#> ▶ dispatched branch y_edba1fe594d3ea2c
#> ● completed branch y_edba1fe594d3ea2c [0.001 seconds, 47 bytes]
#> ▶ dispatched branch y_14a0e5b27f69f42c
#> ● completed branch y_14a0e5b27f69f42c [0 seconds, 48 bytes]
#> ▶ dispatched branch y_956a31fe8f71cc8d
#> ● completed branch y_956a31fe8f71cc8d [0 seconds, 48 bytes]
#> ▶ dispatched branch y_f55a96309b822730
#> ● completed branch y_f55a96309b822730 [0 seconds, 48 bytes]
#> ▶ dispatched branch y_108cb7d92410b8f0
#> ● completed branch y_108cb7d92410b8f0 [0 seconds, 48 bytes]
#> ...

I don't think we need a message for every single dynamic branch. It would be enough to print a small number of strategic messages to indicate how many dynamic branches of `y have completed so far:

tar_make()
#> ▶ dispatched target x
#> ● completed target x [0 seconds, 100 bytes]
#> ▶ y: 1 dispatched, 0 skipped, 0 completed, 10000 total
#> ▶ y: 1000 dispatched, 0 skipped, 0 completed, 10000 total
#> ▶ completed pattern y [10 megabytes]
#> ▶ ended pipeline [...]

It could even be adaptive based on the frequency of activity (similar to messages about skipped targets).

@wlandau
Copy link
Member Author

wlandau commented Apr 1, 2025

I created a new "balanced" reporter for tar_make() and tar_outdated(), and I figured out how to make cli colors forward properly from the callr process. The output is much friendlier and less cumbersome. Example:

# _targets.R file:
library(targets)
list(
  tar_target(datasets, seq_len(1e4)),
  tar_target(models, datasets, pattern = map(datasets))
)
# R console:
tar_make()

Image

# R console:
tar_visnetwork()

Image

An improvement would be to tell callr to somehow respect the interactivity of the parent session and not split the progress bar into multiple lines, but I am not sure this is possible. It would need to happen either in the cli package or the callr package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant