ngauthier / capybara-slow_finder_errors Goto Github PK

View Code? Open in Web Editor NEW

210.0 210.0 19.0 15 KB

License: MIT License

Ruby 100.00%

capybara-slow_finder_errors's People

Contributors

Stargazers

Watchers

Forkers

cgunther pacoguzman jjulian carols10cents jonleighton gnagel beanieboi cwitthaus kirs irfanahmadein fig aried3r walamsyah kevinhq grnhse classicvalues radiopaedia alinavancea john-h-k

capybara-slow_finder_errors's Issues

SlowFinder errors obscure failing assertion error messages

Hi there. I really appreciate the effort you put into debugging and sharing these optimizations. My team enabled the gem for a week and then had to disable it (with an optional enable).

The gem works great. Perhaps too good 😄. The problem was the gem's error message got in our way while we were debugging a failing test. The SlowFinderError would obscure the reason why a test failed. Here is a concrete example so you can see it for yourself:

## Start a vanilla rails codebase

###
### in the Gemfile:
###
group :development, :test do
  gem 'capybara'
  gem 'poltergeist'
  gem 'capybara-slow_finder_errors', require: false
end

###
### in test/integration/red_green_test.rb
###
require 'test_helper'

# I'm including a bunch of this junk to keep the example as self contained as possible
require 'capybara/rails'
require "capybara/poltergeist"

if ENV['RAISE_SLOW_INTEGRATION_ASSERTIONS']
  require 'capybara/slow_finder_errors'
end

Rails.application.routes.eval_block(->{ get 'test/red_green', :to => 'red_green#index'})

class RedGreenController < ApplicationController
  def index
    render text: "<div id='foo'>hello world</div>"
  end
end

class CurrentRedGeenTest < ActionDispatch::IntegrationTest
  include Capybara::DSL

  setup do
    Capybara.current_driver = :poltergeist
  end

  test 'currently red/greening this test' do
    visit '/test/red_green'
    assert_selector '#foo'
    assert_selector '#bar' # I know this is failing, currently red/green'ing it or I broke it with some other change
    assert_text 'asdlkfj' # This will also fail
  end
end

Here is the output with and without slow_finder_errors:

∴ RAISE_SLOW_INTEGRATION_ASSERTIONS=true be ruby -Itest test/integration/red_green_test.rb
Run options: --seed 2280

# Running:

E

Finished in 3.084543s, 0.3242 runs/s, 0.0000 assertions/s.

  1) Error:
CurrentRedGeenTest#test_currently_red/greening_this_test:
Capybara::SlowFinderError: Timeout reached while running a *waiting* Capybara finder...perhaps you wanted to return immediately? Use a non-waiting Capybara finder. More info: http://blog.codeship.com/faster-rails-tests?utm_source=gem_exception
    test/integration/red_green_test.rb:27:in `block in <class:CurrentRedGeenTest>'

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

∴ be ruby -Itest test/integration/red_green_test.rb
Run options: --seed 39879

# Running:

E

Finished in 3.599927s, 0.2778 runs/s, 0.0000 assertions/s.

  1) Error:
CurrentRedGeenTest#test_currently_red/greening_this_test:
Capybara::ExpectationNotMet: expected to find css "#bar" but there were no matches
    test/integration/red_green_test.rb:27:in `block in <class:CurrentRedGeenTest>'

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

As you can see, slow_finder_errors is doing it's job, but it's also obscuring the real reason for the test failure. It gives you a line number, which is nice, but expected to find css "#bar" but there were no matches is very helpful.

Now this will only happen with capybara drivers that wait, like poltergeist or capybara-webkit. The default rack_test will fail immediately and consequently not trigger the SlowFinderError.

My team resorted to having slow_finder_errors opt-in, so that we'd run it once we know we have a green suite as a separate optimization step. We can keep doing that, but it's a bummer to not just have this run all the time.

I toyed with just manually printing the warning with the backtrace instead of raising an error:

# ...
rescue Capybara::ElementNotFound => e
  seconds = args.first || Capybara.default_wait_time
  if Time.now-start_time > seconds
    # raise SlowFinderError, "Timeout reached while running a *waiting* Capybara finder...perhaps you wanted to return immediately? Use a non-waiting Capybara finder. More info: http://blog.codeship.com/faster-rails-tests?utm_source=gem_exception"
    puts "SlowFinder: Timeout reached while running a *waiting* Capybara finder...perhaps you wanted to return immediately? Use a non-waiting Capybara finder. More info: http://blog.codeship.com/faster-rails-tests?utm_source=gem_exception"
    puts e.backtrace
  end
  raise
end
# ...

Which changes the semantics so that it's more informative than fail-fast:

∴ RAISE_SLOW_INTEGRATION_ASSERTIONS=true be ruby -Itest test/integration/red_green_test.rb
Run options: --seed 28231

# Running:

SlowFinder: Timeout reached while running a *waiting* Capybara finder...perhaps you wanted to return immediately? Use a non-waiting Capybara finder. More info: http://blog.codeship.com/faster-rails-tests?utm_source=gem_exception
/Users/zmoazeni/.gem/ruby/2.1.5/gems/capybara-2.4.4/lib/capybara/node/matchers.rb:97:in `block in assert_selector'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/capybara-2.4.4/lib/capybara/node/base.rb:84:in `synchronize'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/capybara-slow_finder_errors-0.1.1/lib/capybara/slow_finder_errors.rb:9:in `synchronize_with_timeout_error'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/capybara-2.4.4/lib/capybara/node/matchers.rb:93:in `assert_selector'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/capybara-2.4.4/lib/capybara/session.rb:676:in `block (2 levels) in <class:Session>'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/capybara-2.4.4/lib/capybara/dsl.rb:51:in `block (2 levels) in <module:DSL>'
/Users/zmoazeni/projects/tmp/test_app/test/integration/red_green_test.rb:27:in `block in <class:CurrentRedGeenTest>'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest/test.rb:108:in `block (3 levels) in run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest/test.rb:206:in `capture_exceptions'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest/test.rb:105:in `block (2 levels) in run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest/test.rb:258:in `time_it'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest/test.rb:104:in `block in run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:321:in `on_signal'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest/test.rb:278:in `with_info_handler'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest/test.rb:103:in `run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:768:in `run_one_method'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:295:in `run_one_method'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:289:in `block (2 levels) in run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:288:in `each'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:288:in `block in run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:321:in `on_signal'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:308:in `with_info_handler'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:287:in `run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:150:in `block in __run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:150:in `map'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:150:in `__run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:127:in `run'
/Users/zmoazeni/.gem/ruby/2.1.5/gems/minitest-5.5.0/lib/minitest.rb:56:in `block in autorun'
E

Finished in 3.610658s, 0.2770 runs/s, 0.0000 assertions/s.

  1) Error:
CurrentRedGeenTest#test_currently_red/greening_this_test:
Capybara::ExpectationNotMet: expected to find css "#bar" but there were no matches
    test/integration/red_green_test.rb:27:in `block in <class:CurrentRedGeenTest>'

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

Any thoughts? Should we continue to make this opt-in? Would you welcome a PR to change the gem to output warning information instead of failing fast with an error? Is there a third option I'm missing altogether?

Thanks for your consideration and your hard work!

Capybara 3.0 support?

Hi,

Capybara 3.0 is released. The gemspec is locked to 2.x. Can you update the gem to work with 3.0?

New release?

👋 Will there be a new release to capture these commits? I see that the last release (0.1.5) was from 2018. Thank you!

Capybara dependency should be stricter?

Doesn't work with Capybara 2.1 or 2.2, because the method signature for synchronize() changed somewhere between 2.2 and 2.4, in 2.1 there is no 2nd argument for options so the gem causes any test that uses synchronize to fail with ArgumentError: wrong number of arguments (2 for 0..1).

[Regression] SlowFinderError not raised on version 0.1.4

Example app:

require 'sinatra'

get '/' do
  erb '<h1>Hello World</h1>'
end

And the spec:

RSpec.describe "Simple app", js: true, type: :feature do
  it "does something" do
    visit "/"
    expect(page.has_content?("Goodbye World")).to be_falsy
  end
end

(Gist with all the code)

Note that i'm not using expect(page).not_to have_content("Goodbye World") because it seems that Capybara intelligently converts that to the negated matcher.

The expectation is correct though: the page shouldn't have the content "Goodbye world", but Capybara waits a full timeout before succeeding. And this gem fails to notice it 😿

But, if the version 0.1.3 is used instead:

// Gemfile
gem "capybara-slow_finder_errors", "0.1.3"

Then it does raise the SlowFinderError:

$ rspec
127.0.0.1 - - [21/Feb/2017:15:07:15 -0300] "GET / HTTP/1.1" 200 20 0.0337
F

Failures:

  1) Simple app does something
     Failure/Error: expect(page.has_content?("Goodbye World")).to be_falsy

     Capybara::SlowFinderError:
       Timeout reached while running a *waiting* Capybara finder...perhaps you wanted to return immediately? Use a non-waiting Capybara finder. More info: http://blog.codeship.com/faster-rails-tests?utm_source=gem_exception
     # ./spec/simple_app_spec.rb:4:in `block (2 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # Capybara::ExpectationNotMet:
     #   expected to find text "Goodbye World" in "Hello World"
     #   ./spec/simple_app_spec.rb:4:in `block (2 levels) in <top (required)>'

Finished in 8.31 seconds (files took 0.74391 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/simple_app_spec.rb:2 # Simple app does something

I think this was a regression introduced in PR #10, which deleted the SlowFinderError and let an Capybara::ElementNotFound error be raised, which seems to be caught by Capybara on these predicate methods.

Add other suggestions as well for capybara fixes

I'm getting a NoMethodError when the content is not in the page

I've found an error in the gem, after include the gem in one of my apps I'm getting in some scenarios the following error:

NoMethodError Exception: undefined method `allow_reload!' for nil:NilClass

Reviewing the code the solution is re-raise the catched exception, I can open a PR with that if you think valuable

What's the point of 0.1.5

Either I'm missing something, but I don't see any code changes between v0.1.4 and v0.1.5.

Is there a changelog or release notes document for this gem?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.