Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(turbo): warn on empty cache #8997

Conversation

tknickman
Copy link
Member

Description

Warn when you have specified outputs that do not exist.

You can optionally pass --skip-empty-cache to skip caching entirely if the specified outputs do not exist. This should probably be the default, but it would be breaking behavior to enable for now.

Co-authored-by: Mehul Kar [email protected]

Copy link

vercel bot commented Aug 12, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
examples-nonmonorepo ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 12, 2024 9:13pm
turbo ❌ Failed (Inspect) Aug 12, 2024 9:13pm
8 Skipped Deployments
Name Status Preview Comments Updated (UTC)
examples-basic-web ⬜️ Ignored (Inspect) Visit Preview Aug 12, 2024 9:13pm
examples-designsystem-docs ⬜️ Ignored (Inspect) Visit Preview Aug 12, 2024 9:13pm
examples-gatsby-web ⬜️ Ignored (Inspect) Visit Preview Aug 12, 2024 9:13pm
examples-kitchensink-blog ⬜️ Ignored (Inspect) Visit Preview Aug 12, 2024 9:13pm
examples-native-web ⬜️ Ignored (Inspect) Visit Preview Aug 12, 2024 9:13pm
examples-svelte-web ⬜️ Ignored (Inspect) Visit Preview Aug 12, 2024 9:13pm
examples-tailwind-web ⬜️ Ignored (Inspect) Visit Preview Aug 12, 2024 9:13pm
examples-vite-web ⬜️ Ignored (Inspect) Visit Preview Aug 12, 2024 9:13pm

/// cache for the task. Enabling this can be a useful safety measure to
/// ensure proper outputs configuration.
#[clap(long, value_name = "BOOL", action = ArgAction::Set, default_value = "false", default_missing_value = "true", num_args = 0..=1)]
pub skip_empty_cache: bool,
Copy link
Member Author

@tknickman tknickman Aug 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sold on adding this as a flag (or this name if we do), but the behavior is nice to have - Enabling this fixes the case when you set outputs incorrectly, and nothing ends up being cached. This can be problematic because if you don't know this happened, and you don't make changes you can get a cache hit that also nothing, making it look like everything worked.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not super for just adding the flag. This is a setting that is should generally be set at the repo level. e.g. If CI has --skip-empty-cache a user can poison the cache with a bad run if they don't have it set.

Copy link
Member

@chris-olszewski chris-olszewski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving with some minor comments.

Comment on lines +1011 to +1013
"warning: no files were found that match the configured \
outputs - make sure \"outputs\" are correctly defined in \
your `turbo.json` for {}",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to write these into the cached task logs so when restoring this hit from cache you'll see this warning?

return Ok(OutputSaveResult::SkippedEmpty);
}
} else {
debug!("caching log output only for {:?}", self.task_id);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super minor, but I think this debug log is made unnecessary by the debug log earlier.

@@ -360,6 +364,21 @@ impl TaskCache {
globwalk::WalkType::All,
)?;

debug!("files to cache: {:?}", files_to_be_cached.len());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add in the task id here

Suggested change
debug!("files to cache: {:?}", files_to_be_cached.len());
debug!("{}: files to cache: {:?}", self.task_id, files_to_be_cached.len());

/// cache for the task. Enabling this can be a useful safety measure to
/// ensure proper outputs configuration.
#[clap(long, value_name = "BOOL", action = ArgAction::Set, default_value = "false", default_missing_value = "true", num_args = 0..=1)]
pub skip_empty_cache: bool,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not super for just adding the flag. This is a setting that is should generally be set at the repo level. e.g. If CI has --skip-empty-cache a user can poison the cache with a bad run if they don't have it set.

Ok(status) => {
match status {
OutputSaveResult::SavedEmpty => {
prefixed_ui.warn(format!(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might be a little noisy for the new default behavior? I'm fine with shipping as is, but we might need/want a way to silence this warning.

OutputSaveResult::SkippedEmpty => {
prefixed_ui.warn("skipped saving cache - no files found");
}
_ => (),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super minor: Explicitly matching against the other variants gets us a compiler check if we end up adding more variants in the future.

Suggested change
_ => (),
OutputSaveResult::Saved | OutputSaveResult::Disabled => (),

@tknickman
Copy link
Member Author

Closing in favor of #9236!

@tknickman tknickman closed this Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants