Tests running very slow on Linux about capybara-webkit HOT 38 CLOSED

As I suspected, there's an easier fix for the QtTcpServer version. Just needed to set TCP_NODELAY on the socket that gets created. Pushed a new branch (tcp-nodelay) with the fix.

from capybara-webkit.

+1 I'm seeing very fast run times on OS X dev machines, but very slow test runs on our ubuntu CI server.

from capybara-webkit.

+1 seeing really slow running test on my brand new Thinkpad Core i7, running on Ubuntu 11.04 (now on 11.10).

from capybara-webkit.

Just got it running on OS X here, and :js => true tests are running very slow, plus this error:

QTKitServer[14805:903] Error loading /Library/QuickTime/LiveType.component/Contents/MacOS/LiveType: dlopen(/Library/QuickTime/LiveType.component/Contents/MacOS/LiveType, 262): Library not loaded: /System/Library/PrivateFrameworks/LiveType.framework/Versions/A/LiveType
Referenced from: /Library/QuickTime/LiveType.component/Contents/MacOS/LiveType
Reason: image not found

I installed Qt from the packages mentioned in the Wiki

from capybara-webkit.

Update:

Hiding google analytics inclusion behind <% if Rails.env.production? %> fixed my speed issue

from capybara-webkit.

We're seeing this same thing (with a CI server running Ubuntu 11.04).

I tried hiding Google Analytics in the environment my tests run in (based on joemsak's comment), but there was no noticeable speed-up.

from capybara-webkit.

We are having the same issue (our CI server is running Ubuntu 11.04) as well. And the solution of hiding google analytics didn't work for us either.

from capybara-webkit.

Are there any other external JS you don't need for testing env? All we had to do was block those dependencies and we use a local jQuery for Rails.env.test? #=> true as well

from capybara-webkit.

we have separate env for test, dev and production. And in the test env, we don't use any external sources.

from capybara-webkit.

Did anyone find solution? Have the same issue.

from capybara-webkit.

Without a specific issue to address I'm going to suggest moving this conversation to the google group: https://groups.google.com/group/capybara-webkit

from capybara-webkit.

Anyone found solution? My tests are 3-5 times slower than with selenium.

from capybara-webkit.

+1, I'm hurting trying to optimize a suite where individual webkit tests are ~12 seconds! Going to group for discussion, see if we can narrow down options.

from capybara-webkit.

Same issue here!

My entire integration suite took ~ 13m on mac os x and ~ 1h30 on linux machines.

Tested against ubuntu desktop, ubuntu server, debian and centos.

from capybara-webkit.

Is SELinux enabled?

from capybara-webkit.

@chicks nope:

[root@1 ~]# sestatus
SELinux status:                 disabled

from capybara-webkit.

From what I can see, there is a ~4-5 second delay when running the tests... It seems to come from Capybara::Driver::Webkit::Browser#pipe_readable? which spends the time waiting for a Timeout#timeout.

The semi-obvious looking line Timeout.timeout(5) do is not the cause of the delay. Although, yesterday my times just over 5 seconds, I thought I had found the cause...

I'm still trying to get to the bottom of this but would appreciate any help from anyone who knows the codebase.

Thread ID: 123693640
Total: 4.196852

 %self     total     self     wait    child    calls  name
  0.00      4.20     0.00     0.00     4.20        1  Capybara::Driver::Webkit::Browser#start_server (/home/anthony/.rvm/gems/ruby-1.9.3-p194@Edvance/gems/capybara-webkit-0.11.0/lib/capybara/driver/webkit/browser.rb:138}
  0.00      4.20     0.00     4.20     0.00        1  IO#eof? (ruby_runtime:0}
  0.00      4.20     0.00     0.00     4.20        1  Capybara::Driver::Webkit::Browser#forward_stdout (/home/anthony/.rvm/gems/ruby-1.9.3-p194@Edvance/gems/capybara-webkit-0.11.0/lib/capybara/driver/webkit/browser.rb:171}
  0.00      0.00     0.00     0.00     0.00        1  <Class::Thread>#current (ruby_runtime:0}
  0.00      4.20     0.00     0.00     4.20        1  Capybara::Driver::Webkit::Browser#pipe_readable? (/home/anthony/.rvm/gems/ruby-1.9.3-p194@Edvance/gems/capybara-webkit-0.11.0/lib/capybara/driver/webkit/browser.rb:191}
  0.00      0.00     0.00     0.00     0.00        1  Thread#abort_on_exception= (ruby_runtime:0}


Thread ID: 124092340
Total: 4.196780

 %self     total     self     wait    child    calls  name
  0.00      0.00     0.00     0.00     0.00        1  Kernel#sleep (ruby_runtime:0}
  0.00      4.20     0.00     4.20     0.00        1  Timeout#timeout (/home/anthony/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/timeout.rb:60}

from capybara-webkit.

I too am noticing this issue. We recently implemented Cucumber and use capybara-webkit as the driver. Works find on my MacBook Pro where the tests run in under 30 seconds. But on my Ubuntu box they take considerably longer, and the slow down seems to happen when the javascript tests are called.

The tests go pretty quick until I get to a javascript test and then they slow down and stay slowed down even after all the javascript tests have completed. So that might be a starting point.

I also want to note that I think it's poor form of the maintainers to close the issue when there is no known fix. Closing it out because there isn't a specific issue to address, to me, seems to kind of ignore the fact that an issue exists. In other words, you're hiding the issue, not actually fixing it. This issue is very real and obviously not resolved. I understand that the root cause of the issue isn't actually known at the moment, but saying it's "closed" is misleading.

from capybara-webkit.

I am also experiencing this problem, and running our cucumber features with capybara-webkit now takes around ten minutes for ~100 scenarios. On a Mac the tests run much faster and the webkit executable uses much more CPU. On Linux the webkit process uses very little CPU at all.

As with @mkoby, I'm also using javascript tests with the javascript annotation.

@anthonybatchelor-examtime what did you use to profile Capybara? I would like to see if I get similar results.

from capybara-webkit.

@mfoo I used ruby-prof and kcachegrind. There seems to be roughly 200ms stalls for every interaction with capybara-webkit. That's for things like does this element exist or fill some text in this box.

from capybara-webkit.

(Edit: see next comment for the simpler fix -- or the pull request -- #358)

I don't quite understand why this was marked as closed.

For me, the slowness appears to be caused by QtTcpServer. On my OpenSUSE machine, I found there was a delay of about 0.04 seconds per command being sent over TCP. (It looks like the poll() in webkit_server doesn't return instantly, despite there being data available.)

I've pushed a branch that uses QtLocalServer instead on the Qt side, and forces the use of UNIXSocket on the Ruby side. It's available as unix-socket on 4moms/capybara-webkit. In my horribly-slow test suite, the changed version is around 200x faster.

The way I've fixed it is incomplete in several ways:

• It probably shouldn't force using UNIX sockets
• Possibly there's some better/easier way to fix the QtTcpServer version
• The test suite doesn't pass
• The socket isn't properly cleaned up (requires rm /tmp/capybara-webkit after each run)

But, it's there if anyone wants to play with it.

from capybara-webkit.

Wow, a 200x improvement is indeed impressive @benizi

Is this latest change stable, or does it still have the problems you commented before?

from capybara-webkit.

+1 to Antonio's comments - if it's fit for use (ie stable) as a drop in replacement, I'll switch to it immediately. Good work!

from capybara-webkit.

@antonio

Latest change should be stable. It doesn't use UNIX sockets, so doesn't have the same issues associated with my klunky first pass.

Taking "stable" in a different sense, though: I don't have green tests. (But I don't have green tests on master in the first place.)

from capybara-webkit.

I'll give it a shot then. Thanks!

from capybara-webkit.

Mmm I am afraid it's not working for me :(

Is there something I have to setup apart from pointing my Gemfile to a cloned version of your repo? First test fails with 'No response received from the server', while subsequent give 'Broken pipe'

from capybara-webkit.

Maybe you were using the original fix? (I switched branch names for the two fixes.) In my app with a slow test suite I changed this portion of my Gemfile from:

gem 'capybara-webkit'

to the following:

gem 'capybara-webkit',
  :git => 'https://github.com/4moms/capybara-webkit',
  :branch => 'tcp-nodelay'

Then after bundle update capybara-webkit it was much faster to run bundle exec rspec. Reverting the change and rerunning bundle update capybara-webkit, and it's back to multiple minutes instead of seconds.

from capybara-webkit.

I've got a different problem to Antonio testing this fix on Ubuntu 12.04, Ruby 1.9.3, Rails.

All my request tests are failing in the same way on visit some_path commands:

Failure/Error: visit new_school_year_path
     ArgumentError:
       wrong number of arguments(2 for 0)
     # ./spec/requests/school_years_spec.rb:39:in `block (4 levels) in <top (required)>'

Admittedly they fail very fast! ;-)

Update: Note that I followed Benizi's instructions above.

Update 2: Seems to be affecting only tests marked with ":js => true"

from capybara-webkit.

@davidkennedy-examtime I can't imagine changing the socket options would have that effect... maybe a result of also updating to the latest version of capybara-webkit? Or of updating something else in your Gemfile unintentionally?

from capybara-webkit.

More debug - I'd the following in my spec_helper.rband removing it has led to more tests passing - not all, but more.

  Capybara.register_driver :webkit do |app|
   Capybara::Driver::Webkit.new(app, :ignore_ssl_errors => true)
 end

Currently several tests just stall. I'm not sure what the cause is yet.

from capybara-webkit.

@benizi - I agree, it does seem unlikely, but I've only updated the one gem in isolation. Using the most recent vanilla gem works fine.

from capybara-webkit.

@davidkennedy-examtime - "most recent vanilla gem" = no :git option? Can you try the 'released-plus-tcp' branch I just pushed? It's master..tcp-nodelay cherry-picked onto the v0.12.1 tag.

gem 'capybara-webkit',
  :git => 'https://github.com/4moms/capybara-webkit',
  :branch => 'released-plus-tcp'

from capybara-webkit.

Clarification:

Editing Gemfile to be

   gem 'capybara-webkit'

followed by a bundle update capybara-webkit worked fine. This is the latest on RubyGems (0.12.1)

  gem 'capybara-webkit',
    :git => 'https://github.com/4moms/capybara-webkit'

followed by a bundle update capybara-webkit did not work. Note the lack of branch specifier. It fails on the tcp_nodelay branch too.

Editing Gemfile to be

  gem 'capybara-webkit',
    :git => 'https://github.com/thoughtbot/capybara-webkit'

followed by a bundle update capybara-webkit did not work. Looks like @benizi is right and there's some sort of upgrade bump I'm hitting?

And finally, editing Gemfile to be

gem 'capybara-webkit',
  :git => 'https://github.com/4moms/capybara-webkit',
  :branch => 'released-plus-tcp'

as requested above followed by a bundle update capybara-webkit ... I'm not sure! Failing tests, but they're failing on random places and not uniformly at the start as before. This might also be an upgrade problem, or related to how we use the library (we have a LOT of javascript). Investigating.

Update: Still looking into the fails (2 from 133 :js => true). The good news is that the whole suite was MUCH faster, ~9 mins to ~2 mins.

from capybara-webkit.

The fork from @benizi does function as a drop in replacement and gives a good improvement in speed (9m -> 6.5m, not the worlds nicest tests so this is a worst case) for webkit specific tests; the integration issues I had were timing problems (hidden divs, ajax calls etc) which had been masked by the overall slowness in the past.

I'm using the released-plus-tcp branch.

Update: For info, the particular speedup I'm seeing is 9m to 6.5m. However, these are very heavy tests so this probably represents a worst case. A couple of smaller tests are much improved.

from capybara-webkit.

Sorry for not getting back to you before. I tried once again, this time with the released-plus-tcp branch. See the results below

Without the patch:

Finished in 6 minutes 47.43 seconds
191 examples, 0 failures

With it:

Finished in 4 minutes 23.02 seconds
191 examples, 0 failures

All the tests have the js flag.

I'll use the patch for a while to test its stability (when I tried yesterday, some tests were failing randomly with the Broken pipe error).

Thanks again

from capybara-webkit.

For anybody still looking into the speed that tests run at, as of August 10th this patch is in capybara-webkit master.

It knocked a 300 scenario, 4500 step, 45 minute test suite (almost all async js) down to about 16 minutes.

I also noticed that with ext4 there's a lot of journaling going on when mysql wrote to disk all the time during the tests, and by moving the mysql database to a ramdisk we got the time down to 6 minutes. If you don't use a ramdisk and instead use in-memory tables, this requires hacking the migration, limiting database field types, and not using transactions.

from capybara-webkit.

I'm not sure whether or not this is the same issue, but I am using a version of capybara-webkit cloned directly from head and I am having a very similar problem on Windows.

Visiting a page (one step):

    user     system      total        real
1.719000   2.688000   4.407000 ( 44.955124)

from capybara-webkit.

Did you guys ever figure out what was going on with the "wrong number of arguments (2 for 0)" problem on the visit method? I'm having this exact same issue with the capybara-webkit gem, which goes away if I use the selenium driver instead.

from capybara-webkit.