Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix byebug when running tests #16708

Closed
3 tasks done
apainintheneck opened this issue Feb 20, 2024 · 3 comments · Fixed by #17010
Closed
3 tasks done

Fix byebug when running tests #16708

apainintheneck opened this issue Feb 20, 2024 · 3 comments · Fixed by #17010
Labels
bug Reproducible Homebrew/brew bug help wanted We want help addressing this

Comments

@apainintheneck
Copy link
Contributor

apainintheneck commented Feb 20, 2024

brew doctor output

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  llvm@11

Verification

  • My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
  • I ran brew update twice and am still able to reproduce my issue.
  • This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

HOMEBREW_VERSION: 4.2.9-42-gc355461
ORIGIN: https://github.com/Homebrew/brew
HEAD: c35546167da22c3c00d34332ff84a1bb40153766
Last commit: 10 hours ago
Core tap HEAD: b0cbf72d8dfb4b17daf988b4cb48ca01f1a72052
Core tap last commit: 7 hours ago
Core tap JSON: 19 Feb 21:36 UTC
Core cask tap HEAD: 10b89b8ddeac7513b77c0d19afe6b759146b6d65
Core cask tap last commit: 8 hours ago
Core cask tap JSON: 19 Feb 21:36 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_AUTOREMOVE: set
HOMEBREW_CASK_OPTS: []
HOMEBREW_DEVELOPER: set
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.A51IW62L7n/org.macosforge.xquartz:0
HOMEBREW_EDITOR: nano
HOMEBREW_MAKE_JOBS: 4
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.1.4 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/bin/ruby
CPU: quad-core 64-bit ivybridge
Clang: 12.0.0 build 1200
Git: 2.24.3 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 10.15.7-x86_64
CLT: 12.0.0.32.29
Xcode: 12.4

What were you trying to do (and why)?

This was something that came up in the team Slack channel. @dduugg first noticed that byebug wasn't working in tests even when running brew tests --byebug and setting breakpoints. I was able to reproduce the bug afterwards.

What happened (include all command output)?

I set a breakpoint with byebug and expected the tests to stop when I hit the breakpoint. That didn't work. I then added the :byebug tag to the test itself and saw the byebug prompt but it wasn't interactive and the rest of the test continued to run after that point immediately.

What did you expect to happen?

I expected to be able to set brew tests --byebug and set breakpoints and for the test suit to stop when they hit those breakpoints. This is helpful for debugging weird test failures.

Step-by-step reproduction instructions (by running brew commands)

  1. Change the tests to add a breakpoint.
diff --git a/Library/Homebrew/test/resource_spec.rb b/Library/Homebrew/test/resource_spec.rb
index e509902bd..4df9ed094 100644
--- a/Library/Homebrew/test/resource_spec.rb
+++ b/Library/Homebrew/test/resource_spec.rb
@@ -82,7 +82,8 @@ RSpec.describe Resource do
   end
 
   describe "#version" do
-    it "sets the version" do
+    it "sets the version", :byebug do
+      byebug
       resource.version("1.0")
       expect(resource.version).to eq(Version.parse("1.0"))
       expect(resource.version).not_to be_detected_from_url
  1. Run brew tests --byebug --only=resource
Randomized with seed 45405
1 process for 1 spec, ~ 1 spec per process
...
[82, 91] in /usr/local/Homebrew/Library/Homebrew/test/resource_spec.rb
   82:   end
   83: 
   84:   describe "#version" do
   85:     it "sets the version", :byebug do
   86:       byebug
=> 87:       resource.version("1.0")
   88:       expect(resource.version).to eq(Version.parse("1.0"))
   89:       expect(resource.version).not_to be_detected_from_url
   90:     end
   91: 
.......................



Took 5 seconds
@apainintheneck apainintheneck added the bug Reproducible Homebrew/brew bug label Feb 20, 2024
@apainintheneck
Copy link
Contributor Author

My best guess is that this was caused by #12967 which turned off stdin to avoid flaky tests. We could probably just conditionally include stdin when running tests with --byebug set.

@issyl0 also suggested maybe moving to the debug gem since it's more actively maintained and works better with Ruby 3. It seems like we'd have to figure out how to build the psych gem which has native dependencies to make that work though. @dduugg saw some failures when he tried it locally and @Bo98 suggested that that might be related to how https://github.com/Homebrew/homebrew-portable-ruby gets built.

@apainintheneck apainintheneck added the help wanted We want help addressing this label Feb 20, 2024
@dduugg
Copy link
Sponsor Member

dduugg commented Feb 20, 2024

Thanks for taking the time to write this u, and for your add’l investigative work. For the time being, it looks like we can include debug as long as we also pin psych to < 5.

@MikeMcQuaid
Copy link
Member

@issyl0 also suggested maybe moving to the debug gem since it's more actively maintained and works better with Ruby 3.

This seems ideal 👍🏻

issyl0 added a commit that referenced this issue Apr 2, 2024
- This processed that we'd requested a debugger, but didn't drop us into
  a debugging console until I
  [stopped the stdin disablement](#16708 (comment)).
issyl0 added a commit that referenced this issue Apr 2, 2024
- This processed that we'd requested a debugger, but didn't drop us into
  a debugging console until I
  [stopped the stdin disablement](#16708 (comment)).

Usage:

```
❯ git diff
diff --git a/Library/Homebrew/test/migrator_spec.rb b/Library/Homebrew/test/migrator_spec.rb
index 87fadd5e95..db4700810a 100644
--- a/Library/Homebrew/test/migrator_spec.rb
+++ b/Library/Homebrew/test/migrator_spec.rb
@@ -69,6 +69,8 @@ RSpec.describe Migrator do
       tab.source["tap"] = "homebrew/core"
       tab.write

+      binding.break
+
       expect do
         described_class.new(new_formula, "oldname")
       end.to raise_error(Migrator::MigratorDifferentTapsError)

issyl0 at pictor in /opt/homebrew on bye-byebug
❯ brew tests --only=migrator --debug
Randomized with seed 59158
1 process for 1 spec, ~ 1 spec per process
.==> Relinking newname
.==> Unlinking oldname
...==> Moving oldname versions to /private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/newname
....==> Migrating formula oldname to newname
==> Unlinking oldname
==> Moving oldname versions to /private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/newname
==> Relinking newname
....[67, 76] in ~/migrator_spec.rb
    67|       tab = Tab.empty
    68|       tab.tabfile = HOMEBREW_CELLAR/"oldname/0.1/INSTALL_RECEIPT.json"
    69|       tab.source["tap"] = "homebrew/core"
    70|       tab.write
    71|
=>  72|       binding.break
    73|
    74|       expect do
    75|         described_class.new(new_formula, "oldname")
    76|       end.to raise_error(Migrator::MigratorDifferentTapsError)
=>#0    block in <top (required)> (3 levels) at ~/migrator_spec.rb:72
  #1    [C] BasicObject#instance_exec at /opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/gems/rspec-core-3.13.0/lib/rspec/core/example.rb:263
  # and 68 frames (use `bt' command for all frames)
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p tab
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p    # command(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p     # command t    # command ta    # command tab    # command(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p tab    # command
=> #<Tab:0x0000000107156be0 @Aliases=[], @Arch=nil, @built_as_bottle=false, @built_on={"os"=>"Macintosh", "os_version"=>"macOS 14", "cpu_family"=>"arm_firestorm_icestorm"}, @Compiler=:clang, @homebrew_version="4.2.16-55-gc8f60ec-dirty", @installed_as_dependency=false, @installed_on_request=false, @loaded_from_api=false, @poured_from_bottle=false, @runtime_dependencies=nil, @source={"path"=>nil, "tap"=>"homebrew/core", "tap_git_head"=>nil, "spec"=>"stable", "versions"=>{"stable"=>nil, "head"=>nil, "version_scheme"=>0}}, @source_modified_time=0, @stdlib=nil, @tabfile=#<Pathname:/private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/oldname/0.1/INSTALL_RECEIPT.json>, @time=nil, @unused_options=[], @used_options=[]>
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464)
```
@issyl0 issyl0 mentioned this issue Apr 2, 2024
7 tasks
issyl0 added a commit that referenced this issue Apr 7, 2024
- This processed that we'd requested a debugger, but didn't drop us into
  a debugging console until I
  [stopped the stdin disablement](#16708 (comment)).

Usage:

```
❯ git diff
diff --git a/Library/Homebrew/test/migrator_spec.rb b/Library/Homebrew/test/migrator_spec.rb
index 87fadd5e95..db4700810a 100644
--- a/Library/Homebrew/test/migrator_spec.rb
+++ b/Library/Homebrew/test/migrator_spec.rb
@@ -69,6 +69,8 @@ RSpec.describe Migrator do
       tab.source["tap"] = "homebrew/core"
       tab.write

+      binding.break
+
       expect do
         described_class.new(new_formula, "oldname")
       end.to raise_error(Migrator::MigratorDifferentTapsError)

issyl0 at pictor in /opt/homebrew on bye-byebug
❯ brew tests --only=migrator --debug
Randomized with seed 59158
1 process for 1 spec, ~ 1 spec per process
.==> Relinking newname
.==> Unlinking oldname
...==> Moving oldname versions to /private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/newname
....==> Migrating formula oldname to newname
==> Unlinking oldname
==> Moving oldname versions to /private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/newname
==> Relinking newname
....[67, 76] in ~/migrator_spec.rb
    67|       tab = Tab.empty
    68|       tab.tabfile = HOMEBREW_CELLAR/"oldname/0.1/INSTALL_RECEIPT.json"
    69|       tab.source["tap"] = "homebrew/core"
    70|       tab.write
    71|
=>  72|       binding.break
    73|
    74|       expect do
    75|         described_class.new(new_formula, "oldname")
    76|       end.to raise_error(Migrator::MigratorDifferentTapsError)
=>#0    block in <top (required)> (3 levels) at ~/migrator_spec.rb:72
  #1    [C] BasicObject#instance_exec at /opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/gems/rspec-core-3.13.0/lib/rspec/core/example.rb:263
  # and 68 frames (use `bt' command for all frames)
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p tab
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p    # command(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p     # command t    # command ta    # command tab    # command(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p tab    # command
=> #<Tab:0x0000000107156be0 @Aliases=[], @Arch=nil, @built_as_bottle=false, @built_on={"os"=>"Macintosh", "os_version"=>"macOS 14", "cpu_family"=>"arm_firestorm_icestorm"}, @Compiler=:clang, @homebrew_version="4.2.16-55-gc8f60ec-dirty", @installed_as_dependency=false, @installed_on_request=false, @loaded_from_api=false, @poured_from_bottle=false, @runtime_dependencies=nil, @source={"path"=>nil, "tap"=>"homebrew/core", "tap_git_head"=>nil, "spec"=>"stable", "versions"=>{"stable"=>nil, "head"=>nil, "version_scheme"=>0}}, @source_modified_time=0, @stdlib=nil, @tabfile=#<Pathname:/private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/oldname/0.1/INSTALL_RECEIPT.json>, @time=nil, @unused_options=[], @used_options=[]>
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464)
```
issyl0 added a commit that referenced this issue Apr 17, 2024
- This processed that we'd requested a debugger, but didn't drop us into
  a debugging console until I
  [stopped the stdin disablement](#16708 (comment)).

Usage:

```
❯ git diff
diff --git a/Library/Homebrew/test/migrator_spec.rb b/Library/Homebrew/test/migrator_spec.rb
index 87fadd5e95..db4700810a 100644
--- a/Library/Homebrew/test/migrator_spec.rb
+++ b/Library/Homebrew/test/migrator_spec.rb
@@ -69,6 +69,8 @@ RSpec.describe Migrator do
       tab.source["tap"] = "homebrew/core"
       tab.write

+      binding.break
+
       expect do
         described_class.new(new_formula, "oldname")
       end.to raise_error(Migrator::MigratorDifferentTapsError)

issyl0 at pictor in /opt/homebrew on bye-byebug
❯ brew tests --only=migrator --debug
Randomized with seed 59158
1 process for 1 spec, ~ 1 spec per process
.==> Relinking newname
.==> Unlinking oldname
...==> Moving oldname versions to /private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/newname
....==> Migrating formula oldname to newname
==> Unlinking oldname
==> Moving oldname versions to /private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/newname
==> Relinking newname
....[67, 76] in ~/migrator_spec.rb
    67|       tab = Tab.empty
    68|       tab.tabfile = HOMEBREW_CELLAR/"oldname/0.1/INSTALL_RECEIPT.json"
    69|       tab.source["tap"] = "homebrew/core"
    70|       tab.write
    71|
=>  72|       binding.break
    73|
    74|       expect do
    75|         described_class.new(new_formula, "oldname")
    76|       end.to raise_error(Migrator::MigratorDifferentTapsError)
=>#0    block in <top (required)> (3 levels) at ~/migrator_spec.rb:72
  #1    [C] BasicObject#instance_exec at /opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/gems/rspec-core-3.13.0/lib/rspec/core/example.rb:263
  # and 68 frames (use `bt' command for all frames)
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p tab
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p    # command(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p     # command t    # command ta    # command tab    # command(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464) p tab    # command
=> #<Tab:0x0000000107156be0 @Aliases=[], @Arch=nil, @built_as_bottle=false, @built_on={"os"=>"Macintosh", "os_version"=>"macOS 14", "cpu_family"=>"arm_firestorm_icestorm"}, @Compiler=:clang, @homebrew_version="4.2.16-55-gc8f60ec-dirty", @installed_as_dependency=false, @installed_on_request=false, @loaded_from_api=false, @poured_from_bottle=false, @runtime_dependencies=nil, @source={"path"=>nil, "tap"=>"homebrew/core", "tap_git_head"=>nil, "spec"=>"stable", "versions"=>{"stable"=>nil, "head"=>nil, "version_scheme"=>0}}, @source_modified_time=0, @stdlib=nil, @tabfile=#<Pathname:/private/tmp/homebrew-tests-20240403-85464-3uogqr/cellar/oldname/0.1/INSTALL_RECEIPT.json>, @time=nil, @unused_options=[], @used_options=[]>
(rdbg@/opt/homebrew/Library/Homebrew/vendor/bundle/ruby/3.1.0/bin/rspec#85464)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Reproducible Homebrew/brew bug help wanted We want help addressing this
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants