Skip to content

sorbet: enable experimental RSpec mode#21690

Draft
dduugg wants to merge 10 commits intomainfrom
deichelberger/sorbet-experimental-rspec
Draft

sorbet: enable experimental RSpec mode#21690
dduugg wants to merge 10 commits intomainfrom
deichelberger/sorbet-experimental-rspec

Conversation

@dduugg
Copy link
Copy Markdown
Member

@dduugg dduugg commented Mar 8, 2026

Summary

  • Enables Sorbet's --enable-experimental-rspec mode so example groups are type-checked with RSpec DSL awareness
  • Promotes all 753 spec files to # typed: true (from untyped) as a prerequisite
  • Generates tapioca gem RBIs for rspec and rspec-mocks (previously excluded)
  • Adds a tapioca DSL compiler that scans test files for custom matcher definitions and generates RSpec::Matchers stubs, reducing "Method does not exist" errors by ~560

Commits

  1. test: promote all spec files to # typed: true — blanket sigil promotion on all 753 spec files
  2. sorbet: enable experimental RSpec mode — adds --enable-experimental-rspec to sorbet config and explicit RSpec::Core::ExampleGroup includes in upstream.rbi
  3. sorbet: generate tapioca gem RBIs for rspec and rspec-mocks — removes the exclusions and regenerates those gem RBIs
  4. sorbet: add tapioca DSL compiler for custom RSpec matchers — scans test files for define/alias_matcher/define_negated_matcher/matcher calls and generates typed stubs

Error count

brew typecheck errors: 5,990 → 5,432 after this PR (−558). The remaining errors are unrelated to custom matchers (built-in dynamic predicate matchers, include/module-helper resolution, etc.).

🤖 Generated with Claude Code

dduugg added 5 commits March 8, 2026 11:48
Adds `# typed: true` to all 753 spec files that previously had no Sorbet
sigil, in preparation for enabling Sorbet's experimental RSpec mode.
Adds `--enable-experimental-rspec` to the Sorbet config so that Sorbet
type-checks RSpec example groups with awareness of the DSL structure
(describe/context/it/let/before/etc.).

Also adds explicit includes on `RSpec::Core::ExampleGroup` in upstream.rbi
so Sorbet resolves `RSpec::Matchers`, `RSpec::SharedContext`, and
`RSpec::Mocks::ExampleMethods` methods within example group bodies.
These gems were previously excluded from tapioca gem RBI generation.
Removing the exclusions and generating their RBIs gives Sorbet the type
information needed to resolve rspec/rspec-mocks constants and methods
under the new experimental RSpec mode.
Adds a tapioca DSL compiler that scans test files for custom RSpec matcher
definitions and generates RBI stubs for them on `RSpec::Matchers`.

Handles all four definition styles:
  - RSpec::Matchers.define :name do |args|
  - (RSpec::Matchers.)?define_negated_matcher :name, :base
  - (RSpec::Matchers.)?alias_matcher :new, :old
  - matcher :name do |args| (inside describe/context/shared_context)

Because example groups include `RSpec::Matchers`, adding stubs there makes
custom matchers visible to Sorbet's experimental RSpec type-checker.
The generated RBI reduces "Method does not exist" errors by ~560.
`RSpec::Matchers::MatcherDelegator` forwards unknown calls to its wrapped
`base_matcher` via `method_missing`. Sorbet cannot follow that delegation,
so aliased/negated wrappers of `output` (e.g. `not_to_output`) appear to
lack `to_stdout`, `to_stderr`, and their `_from_any_process` variants.

Declaring those four methods on `MatcherDelegator` in upstream.rbi — with
`T.self_type` as the return type to preserve chainability — eliminates the
"Method to_stderr does not exist on AliasedNegatedMatcher" errors (-129).
@MikeMcQuaid
Copy link
Copy Markdown
Member

"Looks" good when 🟢. I'd suggest you keep these all typed: false for now and fix them one-by-one? Will be easier to merge earlier and actually provide review.

…ioca

Instead of manually shim-ing individual methods (e.g. to_stdout/to_stderr)
onto RSpec::Matchers::MatcherDelegator, a tapioca DSL compiler now enumerates
every public method defined on a concrete RSpec::Matchers::BuiltIn subclass
(excluding those already on BaseMatcher or MatcherDelegator) and declares
them on MatcherDelegator.

This is more precise than a blanket T.untyped escape-hatch: only methods
that genuinely exist on some built-in matcher are permitted on delegators
(to_stdout/to_stderr from Output, by/from/to from Change, with/argument
from RespondTo, etc.). Calls to methods that exist on no built-in matcher
still raise a 7003 error.

Removes the manual upstream.rbi shim added in the previous commit.
@dduugg
Copy link
Copy Markdown
Member Author

dduugg commented Mar 9, 2026

@MikeMcQuaid This is just a scratch PR for me to track the progress in resolving major categories of rspec/sorbet compatibility (and possibly to link back to when upstreaming work to e.g. sorbet or tapioca). I think it would actually be counterproductive to start flipping the typed: true switch this earlier. It'd be super fragile, and I don't want to confuse folks/agents who fall into type errors when making reasonable, incremental changes. (If you'd prefer I not use a draft PR for this stage, just lmk, and i'll figure out another means of surfacing progress.)

@MikeMcQuaid
Copy link
Copy Markdown
Member

@dduugg cool works for me!

Copy link
Copy Markdown

@coursgranja4-commits coursgranja4-commits left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and the diff what about

dduugg added 4 commits March 11, 2026 12:25
Adds sorbet/rbi/shims/rspec.rbi to inform Sorbet that
RSpec::Core::ExampleGroup includes RSpec::Matchers,
RSpec::Mocks::ExampleMethods, RSpec::SharedContext, and
RuboCop::RSpec::ExpectOffense (both as instance and class methods, the
latter to handle block-scope inference in .each + it patterns).

Adds sorbet/rbi/shims/rubocop.rbi to define RuboCop::RSpec::ExpectOffense
with its methods (expect_offense, expect_correction, expect_no_offenses,
etc.), which are absent from the tapioca-generated rubocop gem RBI.

4309 type errors remain.
… a manual shim

The rubocop gem ships rubocop/rspec/expect_offense but does not require it
from its main entry point. sorbet/tapioca/require.rb already loads it as an
additional require when tapioca processes rubocop-rspec, so tapioca correctly
attributes RuboCop::RSpec::ExpectOffense to the rubocop gem. The trim
allowlist in typecheck.rb already includes RuboCop::RSpec::ExpectOffense.

After a full gem regeneration (brew typecheck --update-all) the module and its
AnnotatedSource class appear in rubocop@1.85.0.rbi, making the manual
sorbet/rbi/shims/rubocop.rbi shim redundant.
RSpec::Mocks::ExampleMethods includes ExpectHost which defines
`expect(target)` with a required argument. Because it is included
after RSpec::Matchers in the shim, it shadows
`RSpec::Matchers#expect(value = T.unsafe(nil), &block)` in Sorbet's
method lookup, causing a "Not enough arguments" error whenever the
block-only form is used:

  expect { some_code }.not_to raise_error

Explicitly define `expect` on RSpec::Core::ExampleGroup itself so it
takes precedence over both included modules, matching the real RSpec
signature from rspec-expectations:
https://github.com/rspec/rspec/blob/rspec-expectations-v3.13.5/rspec-expectations/lib/rspec/expectations/syntax.rb#L72-L74

brew tc error count: 3467 (unchanged)
RSpec::Mocks::ExampleMethods includes ExpectHost which defines
`expect(target)` with a required argument. Because it was included
after RSpec::Matchers, it sat higher in the MRO and shadowed
`RSpec::Matchers#expect(value = T.unsafe(nil), &block)`, causing:

  expect { brew "command", "info" }  # Not enough arguments, expected 1 got 0

Fix by including RSpec::Matchers last so it sits above
RSpec::Mocks::ExampleMethods in the ancestor chain, making
the rspec-expectations signature take precedence:
https://github.com/rspec/rspec/blob/rspec-expectations-v3.13.5/rspec-expectations/lib/rspec/expectations/syntax.rb#L72-L74

brew tc error count: 3466 (was 3467)
@issyl0
Copy link
Copy Markdown
Member

issyl0 commented Mar 14, 2026

@dduugg Let me know how I can help!

dduugg added a commit that referenced this pull request Mar 15, 2026
Add `--enable-experimental-rspec` to the Sorbet config so that Sorbet
type-checks RSpec spec files. This is an isolated piece of the work in
#21690, separated so that the follow-up work of dialling up the `typed:`
level of individual specs can land independently.

Enabling the flag surfaced a pre-existing type error in
`formula-analytics/pycall-setup.rbi` where `InfluxDBClient3#initialize`
was incorrectly declared as a singleton method (`def self.initialize`)
rather than an instance method (`def initialize`). Fixed here since the
error is only visible with RSpec mode enabled.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
dduugg added a commit that referenced this pull request Mar 15, 2026
Add `--enable-experimental-rspec` to the Sorbet config so that Sorbet
type-checks RSpec spec files. This is an isolated piece of the work in
#21690, separated so that the follow-up work of dialling up the `typed:`
level of individual specs can land independently.

Enabling the flag surfaced a pre-existing type error in
`formula-analytics/pycall-setup.rbi` where `InfluxDBClient3#initialize`
was incorrectly declared as a singleton method (`def self.initialize`)
rather than an instance method (`def initialize`). Fixed here since the
error is only visible with RSpec mode enabled.
@dduugg
Copy link
Copy Markdown
Member Author

dduugg commented Mar 15, 2026

@issyl0 I've isolated the work of enabling rspec mode: #21742

After that lands, follow-up work can happen dialing up the typedness of spec files, as seems reasonable to do. I can fast-follow with one or two, and you can take the lead from there, if you like.

@issyl0
Copy link
Copy Markdown
Member

issyl0 commented Mar 29, 2026

Hi @dduugg!

I saw that you'd split out adding the experimental flag, and that now this PR has a bunch of conflicts, so I did a follow up to make all tests typed: false to ease us into bumping them to true in the future.

I'm not sure what your eventual plans are for this - I'd love to help do some of this work (I love a list!), but I also noticed that your issue here was closed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants