Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

livecheck: add --extract-plist #16759

Merged
merged 1 commit into from Mar 21, 2024

Conversation

razvanazamfirei
Copy link
Member

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew typecheck with your changes locally?
  • Have you successfully run brew tests with your changes locally?

Running brew livecheck --cask --eval-all downloads all casks using the ExtractPlist strategy. This takes up a significant amount (~20GB) of bandwidth and cache space.

If this flag is added, it will allow maintainers to check taps more quickly and with less resources consumed.

Copy link
Contributor

@apainintheneck apainintheneck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity I looked it up and this would affect 275 core casks. Outside of memory savings is this faster as well?

Either way this seems reasonable to me.

Library/Homebrew/livecheck/livecheck.rb Outdated Show resolved Hide resolved
Library/Homebrew/livecheck/livecheck.rb Outdated Show resolved Hide resolved
Library/Homebrew/livecheck/livecheck.rb Outdated Show resolved Hide resolved
@apainintheneck apainintheneck added the cask Homebrew Cask label Feb 28, 2024
@bevanjkay
Copy link
Member

bevanjkay commented Feb 28, 2024

Out of curiosity I looked it up and this would affect 275 core casks. Outside of memory savings is this faster as well?

@apainintheneck
It is significantly faster to run brew livecheck if you skip the extract_plist.
The strategy downloads the binary and checks the contents for a version number, whereas most of the time we are checking HTML webpage contents, or a curl response header.
As some of the files are quite large the network requests can take some time to complete. Note that we only use this strategy when all other alternatives are not possible.

I have my own script locally that runs livecheck on a tap after filtering out the extract_plist casks, which could be replaced with this flag.

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running brew livecheck --cask --eval-all downloads all casks using the ExtractPlist strategy. This takes up a significant amount (~20GB) of bandwidth and cache space.

I really appreciate the PR (great work @razvanazamfirei!) but: the above described behaviour makes no sense.

I'd argue instead that it should instead be one of:

  • --extract-plist is added instead so it's opt-in
  • --eval-all never attempts to extract plists
  • something else where the above behaviour doesn't manifest

@reitermarkus
Copy link
Member

reitermarkus commented Feb 28, 2024

Maybe this could work similarly to audit with --only and --except flags, so it can also be used more generically to validate changes e.g. when changing a strategy.

@apainintheneck
Copy link
Contributor

  • --extract-plist is added instead so it's opt-in

I like this idea. Do we have to worry about backwards compatibility with this command though. If so, we'll have to go through the whole deprecation cycle and the way it's currently written would be the fastest way to get access to it. It seems like more of an internal command so we're probably fine though.

@MikeMcQuaid
Copy link
Member

Do we have to worry about backwards compatibility with this command though.

100% not.

It seems like more of an internal command so we're probably fine though.

Agreed.

@MikeMcQuaid
Copy link
Member

We could even perhaps have a documented policy that some/all developer commands do not require deprecation periods at all.

@razvanazamfirei razvanazamfirei changed the title livecheck: add --no-extract-plist livecheck: add --extract-plist Mar 20, 2024
@razvanazamfirei
Copy link
Member Author

Thank you all for the suggestions! I've reworked this.

  1. The default behavior has been changed to skip ExtractPlist casks.
  2. Single cask checks maintain current behavior.
  3. Added --extract-plist flag to opt-in to the livecheck strategy.

Copy link
Contributor

@apainintheneck apainintheneck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good to me now. I left a nit but, of course, you can ignore that.

There does seem to be a problem with the generated man page and shell completions. It's deleting info related to a few commands. I think these were commands that @dduugg moved to the new command DSL recently so maybe that's related somehow. I really have no idea.

I ran the following and it changed some settings locally too but it tried to remove information related to a different command: brew bump-unversioned-casks

$ brew generate-man-completions
Writing completions to /usr/local/Homebrew/completions
Manpage and completions updated.
$ git diff --stats
 completions/bash/brew      | 24 ------------------------
 completions/fish/brew.fish | 14 --------------
 completions/zsh/_brew      | 19 -------------------
 docs/Manpage.md            | 21 ---------------------
 manpages/brew.1            | 14 --------------
 5 files changed, 92 deletions(-)
$ git log -1
commit 7b2bfee3638cba1571157f07fd447d9949bd0a6a (HEAD -> master, origin/master, origin/HEAD)
Merge: 0e981bd8f4 74aea8e92d
Author: Kevin <[email protected]>
Date:   Wed Mar 20 20:40:26 2024 -0700

    Merge pull request #16903 from apainintheneck/prevent-unexpected-network-calls-in-tests

    Prevent unexpected network calls in tests

Now I'm even more confused.

Library/Homebrew/livecheck/livecheck.rb Outdated Show resolved Hide resolved
Library/Homebrew/livecheck/livecheck.rb Show resolved Hide resolved
Library/Homebrew/livecheck/livecheck.rb Outdated Show resolved Hide resolved
@MikeMcQuaid
Copy link
Member

Thanks @razvanazamfirei!

@MikeMcQuaid MikeMcQuaid merged commit dddc24d into Homebrew:master Mar 21, 2024
26 checks passed
@razvanazamfirei razvanazamfirei deleted the no-plist-flag branch March 21, 2024 13:18
@dduugg
Copy link
Sponsor Member

dduugg commented Mar 21, 2024

There does seem to be a problem with the generated man page and shell completions. It's deleting info related to a few commands. I think these were commands that @dduugg moved to the new command DSL recently so maybe that's related somehow. I really have no idea.

Thanks for flagging, i'll take a look.

@dduugg
Copy link
Sponsor Member

dduugg commented Mar 21, 2024

There does seem to be a problem with the generated man page and shell completions. It's deleting info related to a few commands. I think these were commands that @dduugg moved to the new command DSL recently so maybe that's related somehow. I really have no idea.

Thanks for flagging, i'll take a look.

Looks like I fat-fingered Homebrew::DevCmd::BumpUnversionedCask, should be Homebrew::DevCmd::BumpUnversionedCasks. I'll fix that and a test to validate classes map to their file/cmd names as expected.

@dduugg dduugg mentioned this pull request Mar 22, 2024
7 tasks
@github-actions github-actions bot added the outdated PR was locked due to age label Apr 21, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cask Homebrew Cask outdated PR was locked due to age
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants