Coverage returning all zeros

[RyanFerretti]

I'm sure this is something dumb that I missed... I just went through the setup for deep-cover and I essentially get no coverage when running the tool. This is a normal Rails app with RSpec. The Gemfile declaration in the test group is: gem 'deep-cover', '~> 0.7', require: false (actual version is 0.7.10).

At the top of my rails_helper.rb which is included in every spec I have:

require 'deep-cover'
require 'spec_helper'

I can see that deep-cover is running when I run deep-cover exec rspec because I get coverage output but everything is 0's:

As far as I understand from the README there are only the 2 steps to get deep-cover running... what am I missing here? I also saw that questions should be posted on StackOverflow but didn't see any there... happy to do that if this is the wrong place.

[marcandre]

I'm sorry you're having issues with running it.

I went through the motions and created an app from scratch in Rails 6, didn't encounter any particular issues. Here it is if you want to check it out. The readme has the output.

If you put a raise 'in users' first line of users.rb and raise 'in rails_helper' as first line of rails_helper.rb and you run deep-cover exec rspec, do you encounter the "in rails_helper" exception first like you should? If not, the backtrace could help how come the models are loaded first...

[marcandre]

Another thing to try: remove the require: false in the Gemfile; in theory it shouldn't really change anything, but we could check...

[RyanFerretti]

Thanks for the advice... removed the require: false and that didn't do anything as you suspected. Next I added:

and ran deep-cover exec rspec, which output:

Good!

Next I added in:

and ran deep-cover exec rspec again, which output:

So it appears that `rails_helper.rb` is loading first. I'll pull down the Rails 6 app you posted and see if things run locally.

[RyanFerretti]

I compared all of the versions I was using vs what you had submitted in that Rails 6 repo...

ruby - 2.7.0
rails - 6.0.2.1 (your version was 6.0.2.2)
rspec-core - 4.0.0 (your version was 3.9.1)
deep-cover - 0.8.0

I upgraded Rails and downgraded RSpec to match and still got all zeros. HOWEVER... I was playing around with things and I went back and removed require: false from the gem declaration and also removed require 'deep-cover' from the rails_helper.rb:

it worked... with the output:

... which seem to line up with the tests I have already written. So I think things are working!... I have not been able to get things working when the require 'deep-cover'line is included in rails_helper.rb. In fact, when there is no require:false on the gem declaration AND require 'deep-cover' is at the top of the rails_helper.rb file... the coverage is all zeros again.

If you want to close this then fine by me... but I'm also happy to troubleshoot more if it is helpful to others. Thanks again for the help!

[marcandre]

Wow, good investigation, this is very strange.

There is something quite interesting going on here. Do you have integration specs (using something like capybara)? If so, there's one process for the webserver and one for the spec runner, maybe one overwrites the other's counters somehow?

[marcandre]

The other case I looked at (#68) ended up being simple; deep-cover wasn't required at all when run via Open3.pipeline_w or Open3.capture3. I'm still quite curious about your particular setup.

One thing I used was inserting debugging code like raise 'oops' unless defined?(DeepCover) && DeepCover.running? in the library to make sure it was loaded and running...

Awaiting details on your setup...

[RyanFerretti]

Had a busy work week so just jumping back into this... I did have capybara required in the spec_helper.rb file. I wasn't actually using it as I've only written model specs as of now; removing it did not change anything as well. The only think I can think of being an external process is Sidekiq... I'll see if I can explore that tomorrow.

I added some running? checks around when I require deep-cover in the rails_helper file:

and did get:

Failure/Error: raise 'running' if DeepCover.running?

RuntimeError:
  running

So it does seem like it is running. Is there any method I can hook into or even inject code via eval to debug? I'll also exploring just ripping things out to see if I can get to a working state.

[marcandre]

I meant inserting that code in your google_calendar.rb, for example

[RyanFerretti]

I've added it to several model classes and it says that it is running on every instance... just no coverage when it gets to the end. I'm working on pulling out things to see if it changes. Here are the libraries I'm using currently,

I shouldn't take me much longer trim this app down in order to share. I'll report back.

[marcandre]

I shouldn't take me much longer trim this app down in order to share. I'll report back.

That would be awesome

[marcandre]

Looks like I also get all 0s from deep-cover exec on rspec-expectactions