igrigorik / em-synchrony Goto Github PK
View Code? Open in Web Editor NEWFiber aware EventMachine clients and convenience classes
Home Page: http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers
License: MIT License
Fiber aware EventMachine clients and convenience classes
Home Page: http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers
License: MIT License
One thing I've found handy while integrating/migrating my code to use EM::Synchrony is the following:
module FiberHelper
mattr_accessor :root_fiber, :instance_writer => false, :instance_reader => false
def self.root_fiber?
root_fiber == Fiber.current
end
def ensure_fiber(&blk)
if FiberHelper.root_fiber?
Fiber.new { blk.call }.resume
else
blk.call
end
end
end
FiberHelper.root_fiber ||= Fiber.current
If we're already running in some fibrous context, one doesn't need to do the wrapping. Much in the same way that EM.schedule
is handy when you're trying to ensure that some thread doesn't trample your async code: if you're already in the reactor, great! otherwise, make things safe.
This way, as I'm going through and converting methods one at a time, I can just wrap the body in ensure_fiber
and I don't have to worry about the calling context.
Thought it might make a decent convenience feature (with a slightly better name, perhaps).
Here's a thread for bit of context:
https://groups.google.com/d/msg/goliath-io/smLYeLYWHOA/WX9Fq6gj_O4J
Basically i am trying to use activerecord with em-synchrony/activerecord in a Goliath app. Problem is unlike rails, I can't just do establish_connection for each request and not expect some exceptions in case of too many connections at once or long running queries at once. Now that the the activerecord driver has moved into this repo from mysql2, need to add some sort of connection pool logic within context of fiber. Anyway, I don't understand nuts and bolts of it underneath, but here's some sample code to reproduce the problem:
https://gist.github.com/1185433 (app with endpoint and db config)
https://gist.github.com/1185438 (gems used and script to hit the endpoint in parallel to invoke this).
Running the script I get intermitted exceptions like:
Mysql2::Error: This connection is still waiting for a result, try again once you have the result: SELECT sleep(1.0)
I cant reproduce now but I also got 'Too many connections' exception earlier.
*This looks very similar to issue mentioned here, #51
Feel free to delete if dup.
Using synchrony clients (such as the Mysql2::EM::Fiber::Client) within a EM timer results (e.g. by using EM.add_periodic_timer) results in "`yield': can't yield from root fiber (FiberError)". Presumably this is because EM executed the block in the timer outside of the Fiber created by em-synchrony.
Wrapping the code in the timer block within a Fiber solves the error. E.g.
f = Fiber.current Fiber.new { block.call f.transfer }.transfer
em-synchrony should monkey-patch the EM timer methods to wrap timer blocks in Fibers or provide new timer methods that do so.
Authenticating to a mongo database does not seem to work when running it from within synchrony, I think I have pinpointed the problem though!
#authenticate does a find for a nonce from the $cmd database, and tries to execute some stuff from within the callback:
auth_resp = self.collection(SYSTEM_COMMAND_COLLECTION).first({'getnonce' => 1})
auth_resp.callback do |res|
...
end
Because .first gets overriden by synchrony, it does not have a callback method and the object is a BSON::OrderedHash. Using .callback on that throws this error:
em-mongo-fc3267511ddc/lib/em-mongo/database.rb:343:in `authenticate': undefined method `callback' for {"nonce"=>"684c385308963754", "ok"=>1.0}:BSON::OrderedHash (NoMethodError)
The authenticate method is quite extensive so I'm not sure what is the best way to go here.
I'll ping the author of em-mongo to see if he has any thoughts on this! :-)
If an invalid host is passed to EM for listening, like this:
EM::run do
EM::synchrony do
EM::start_server '*', 1234, Echo
end
end
It shows a completely unrelated error message like this:
/home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:52:in `resume': dead fiber called (FiberError)
from /home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:52:in `block in sync'
from /home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.4/lib/em/deferrable.rb:158:in `call'
from /home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.4/lib/em/deferrable.rb:158:in `set_deferred_status'
from /home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.4/lib/em/deferrable.rb:198:in `fail'
from /home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/connection/synchrony.rb:34:in `rescue in receive_data'
from /home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/redis-2.2.2/lib/redis/connection/synchrony.rb:29:in `receive_data'
from /home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `run_machine'
from /home/whitequark/.rvm/gems/ruby-1.9.2-p290/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `run'
from ./server.rb:34:in `<main>'
(In my case, a EM'd Redis connection was also present.)
I've been working on integrating EM::Synchrony/Fiber-style interfaces into several EM-based libraries I'm developing/maintaining, and I wanted to bring up for discussion some concerns I have about promoting the "replace methods with synchrony versions" pattern.
The main concern I have is that this is clearly Action at a distance. I've seen this in my own code, and it introduces bugs that can be difficult to track down. The main problem with the method-replacement style is that it works if none of the methods being replaced depend on another method being replaced.
Let's say I have the following (contrived) class:
class Foo < Blah
def bark
EM::DefaultDeferrable.new.tap do |dfr|
EM.next_tick do
dfr.succeed("woof!")
end
end
end
def bark_loudly
EM::DefaultDeferrable.new.tap do |dfr|
bark.callback do |str|
dfr.callback(str.upcase)
end
end
end
end
This is a fairly common pattern, the problem comes when someone replaces the bark
and bark_loudly
methods naievly.
class Foo < Blah
%w[bark bark_loudly].each do |name|
class_eval(<<-EOS, __FILE__, __LINE__+1)
alias :"a#{name}" :#{name}
def #{name}
EM::Synchrony.sync(a#{name})
end
EOS
end
end
This will work in the case of the bark
method, but bark_loudly
will break because it's calling bark
expecting to get a Deferrable back, but instead it gets a string.
(As a side note, the alias
line above will cause problems if the class is reloaded, as it will overwrite the original abark
method with the now-sync bark
method. A guard should be added to ensure that abark
isn't already defined.)
I've had this happen to me when using the WebMock library, as it (somewhat frustratingly) rewrites HTTPMethods
if Synchrony
is defined. My code is depending on the asynchronous nature of EM::HttpRequest
, and their assumption breaks my code. Not only that, it breaks my code in a way that's very difficult for me to track down, because no exception is thrown, it's just my code hangs forever. Now, I fully understand that this isn't part of em-synchrony, however I'm concerned because I think that em-synchrony is an important project, and I feel promoting this kind of pattern may cause more problems than it solves.
Unless all of the libraries being used understand how to communicate with the EM::Synchrony-rewritten methods, then there will be problems. What I've taken to doing in my own code is writing a synchrony-aware wrapper-class around the async class. Both the sync and async classes define #to_synchrony
and #to_async
methods which allow easy switching between styles. This preserves the original methods, and will not create "spooky action at a distance" in code that doesn't know about Synchrony, and yet will make it convenient for people who want to have easy access to the sync version.
The wrapper style isn't as slick as method replacement because it's somewhat more explicit, however, implicitly changing the definition of methods is changing global state in a way that can have unforseen consequences.
with this code
require 'rubygems'
require 'eventmachine'
require "em-synchrony"
require "em-synchrony/em-http"
require "em-synchrony/iterator"
require 'nokogiri'
EM.synchrony do
concurrency = 2
urls = ["http://www.x1.com", "http://www.x2.com"]
results = EM::Synchrony::Iterator.new(urls, concurrency).map do |url, iter|
# fire async requests, on completion advance the iterator
http = EventMachine::HttpRequest.new(url).aget
http.callback { iter.return(http) }
end
result2 = EM::Synchrony::Iterator.new(results, concurrency).map do |client, iter|
doc=Nokogiri::XML(client.response)
results_xml=nil
Fiber.new do
results_xml = EM::Synchrony::Iterator.new(doc.search("*"), concurrency).map do |node, iter2|
iter2.return(node.name)
end
end.resume
iter.return(results_xml)
end
p result2
EventMachine.stop
end
result2 is an array of nil. if like if second iterator don't return to first iterator the result. where i have write wrong code? is right or synchroni can only use of one level of profondity in stack?
Hi,
The script below only executes the first request. The second one is not triggered. I'm not sure if this is really supposed to work.
EM.synchrony { bot = EventMachine::HttpRequest.new("www.example.com") res1 = EM::Synchrony.sync(bot.ahead :path => "/url/blop.asp") res2 = EM::Synchrony.sync(bot.apost :path => "/url/foo.asp") EM.stop }
I tried using :keepalive => true in both requests but didnt work as well. For some reason keepalive option added ~10s between both requests. I can provide a tcpdump if needed.
Thanks
Hi there,
I randomly get this exception in a mysql2/fiber_pool/em-syncrhony/rails3.1 setup:
/home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/mysql2-0.3.7/lib/active_record/connection_adapters/em_mysql2_adapter.rb:53:in `resume': dead fiber called (FiberError)
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/mysql2-0.3.7/lib/active_record/connection_adapters/em_mysql2_adapter.rb:53:in `block in query'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:155:in `call'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:155:in `set_deferred_status'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/eventmachine-1.0.0.beta.3/lib/em/deferrable.rb:194:in `fail'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/mysql2-0.3.7/lib/active_record/connection_adapters/em_mysql2_adapter.rb:38:in `rescue in notify_readable'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/mysql2-0.3.7/lib/active_record/connection_adapters/em_mysql2_adapter.rb:33:in `notify_readable'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `run_machine'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `run'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/thin-1.2.11/lib/thin/backends/base.rb:61:in `start'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/thin-1.2.11/lib/thin/server.rb:159:in `start'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/thin-1.2.11/lib/thin/controllers/controller.rb:86:in `start'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/thin-1.2.11/lib/thin/runner.rb:185:in `run_command'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/thin-1.2.11/lib/thin/runner.rb:151:in `run!'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/gems/thin-1.2.11/bin/thin:6:in `<top (required)>'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/bin/thin:19:in `load'
from /home/rails/sites/adserver/shared/bundle/ruby/1.9.1/bin/thin:19:in `<main>'
is this a known problem or is there any solution for this?
I already had it without em-synchrony (just fiber-pool & mysql2).
I'm trying to find this beast as it happens in production only - so far :)
-Tobias
Thanks for providing em-synchrony.
How difficult would it be to produce an em-sequel-dbi gem in order to enjoy the benefits of non-blocking db i/o with a wider range of db engines? Do you know of anyone working on such a gem?
Thanks in advance.
I put the first example in the README (section "Fiber-aware Iterator: mixing sync / async code") into a file, installed em-synchrony 0.3.0.beta.1
, and ran it:
$ ruby em-synchrony-example.rb http://google.com/ http://drnicwilliams.com/ /Users/drnic/.rvm/gems/ruby-1.9.2-p180/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:21:in `class_eval': undefined method `get' for module `EventMachine::HTTPMethods' (NameError) from /Users/drnic/.rvm/gems/ruby-1.9.2-p180/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:21:in `class_eval' from /Users/drnic/.rvm/gems/ruby-1.9.2-p180/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:21:in `block in ' from /Users/drnic/.rvm/gems/ruby-1.9.2-p180/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:9:in `each' from /Users/drnic/.rvm/gems/ruby-1.9.2-p180/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:9:in `' from /Users/drnic/.rvm/gems/ruby-1.9.2-p180/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:8:in `' from /Users/drnic/.rvm/gems/ruby-1.9.2-p180/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-http.rb:7:in `' from /Users/drnic/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:58:in `require' from /Users/drnic/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:58:in `rescue in require' from /Users/drnic/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:35:in `require' from em-synchrony-example.rb:2:in `'
My script is:
require "rubygems" require "em-synchrony/em-http" EM.synchrony do urls = ARGV concurrency = urls.size # iterator will execute async blocks until completion, .each, .inject also work! results = EM::Synchrony::Iterator.new(urls, concurrency).map do |url, iter| # fire async requests, on completion advance the iterator http = EventMachine::HttpRequest.new(url).aget http.callback { iter.return(http) } http.errback { iter.return(http) } end p results # all completed requests EventMachine.stop end
How did I screw this up?`
I am using EM::Synchrony::Iterator to expand shortened urls and I found that it would periodically hang. I can't figure out what exactly causes it. It does not happen if I use Typhoeus::Hydra. I made a little test script that shows the hang. It occurs at random points, but it seems to happen always when the last url in the batch was processed. i.e. the iterator block fails to return control.
See https://gist.github.com/770795 for test code and https://gist.github.com/770794 for a list of sample urls. There's also an url in the sample urls list that has a non-resolvable domain (http://bits4babies.com/blog/?p=1741) which will also hang the iterator.
Wouldn't it be better to define blocking methods in em-http, e.g. sget
, instead of overriding the existing get
and keeping the old one as aget
. Right now I can't use em-synchrony because other libraries using em-http breaks.
When I run the below code, it dies with a Fiber error, In my EM code I have lots of callbacks that run with timers and call themselves, how would I do this using em-synchrony?
/usr/local/lib/ruby/gems/1.9.1/gems/em-synchrony-1.0.0/lib/em-synchrony/em-redis.rb:42:in `yield': can't yield from root fiber (FiberError)
require "em-synchrony"
require "em-synchrony/em-redis"
EM.synchrony do
r = EM::Protocols::Redis.connect
i = 0
test = lambda {
puts i
r.rpush "afoo", "s1"
r.rpush "afoo", "s2"
i += 1
EM.next_tick &test
}
test.call
#EventMachine.stop
end
# Gemfile
source :rubygems
gem "em-synchrony", :git => 'git://github.com/igrigorik/em-synchrony'
gem "eventmachine", :git => 'git://github.com/eventmachine/eventmachine'
gem "em-http-request", :git => 'git://github.com/igrigorik/em-http-request'
# example-aget.rb
require 'rubygems'
require 'bundler/setup'
require 'em-synchrony'
require 'em-synchrony/em-http'
EM.synchrony do
pages = EM::Synchrony::Iterator.new(1..2, 2).map do |num, iter|
page = EM::HttpRequest.new('http://google.com').aget :redirects => 1
page.callback { iter.return(page) }
end
p "Callbacks. Fetched pages: #{pages}"
EventMachine.stop
end
# example-get.rb
require 'rubygems'
require 'bundler/setup'
require 'em-synchrony'
require 'em-synchrony/em-http'
EM.synchrony do
pages = EM::Synchrony::Iterator.new(1..2, 2).map do |num, iter|
page = EM::HttpRequest.new('http://google.com').get :redirects => 1
iter.return(page)
end
p "No callbacks. Fetched pages: #{pages}"
EventMachine.stop
end
this code works...
EM.synchrony do
page = EM::HttpRequest.new('http://google.com').get :redirects => 1
p "No callbacks. Fetched page: #{page}"
EventMachine.stop
end
example-aget.rb works fine too.
However, when I try to run example-get.rb from above, I get the following error.
$ ruby example-get.rb
(eval):10:in `yield': can't yield from root fiber (FiberError)
from (eval):10:in `get'
from test.rb:8:in `block (2 levels) in <main>'
.....
Is this expected behavior, incorrect use on my part, or a bug in .get
used in conjunction with EM::Synchrony::Iterator
?
it looks like the AR monkey patch is missing something. If I perform a Widget.create(:name => 'top_posts') in the current em-synchrony/spec/activerecord_spec.rb I get this error :
It seems the same issue as brianmario/mysql2#202
Regards.
Steps to reproduce:
Model.reset_column_information
and run rake db:migrate
Exception occurs in active_record/connection_adapters/abstract/connection_pool
in #clear_table_cache!
in #152: right here
Exception is 'undefined method "delete" for nil:NilClass'
.
The latest gem release of em-synchrony
is from April. Any chance of releasing a new version? I'd love to start making use of the em-redis
bits.
em-synchrony works great when within a fiber you want to make use of an async library in a non-callback way. Alas, the EM callbacks are not wrapped within a fiber (except for the timers callbacks, where were recently added), which means you can't use em-synchrony within them without some work.
And example of where this is useful is something like the AMQP gem, where one subscribers to messages from a queue for further processing. The processing itself occurs as a result of a callback from EM, and this callback is not wrapped in a fiber, which means you can use em-synchrony to do further work (e.g. send the message over HTTP), without a bit of leg work.
You'd probably have to patch EM. event_callback and EM. run_deferred_callbacks.
Hi! I try to get ruby-net-nntp running with em-synchrony through the TCPSocket replacement. All that would be needed is a #readline method to get single lines from the input stream.
Are you able to provide that?
-alex
Hello!
I think I've spotted a bug in em-synchrony, although it may be related to another library. My service is using ActiveRecord outside of rails and it runs fine using database 'mysql2'. However, using 'em_mysql2' adapter and issue a ActiveRecord::Create I'm met with the exception:
NoMethodError - undefined method `[]' for nil:NilClass:
/var/lib/gems/1.9.1/gems/activerecord-3.1.0/lib/active_record/base.rb:717:in `column_defaults'
/var/lib/gems/1.9.1/gems/activerecord-3.1.0/lib/active_record/persistence.rb:327:in `attributes_from_column_definition'
/var/lib/gems/1.9.1/gems/activerecord-3.1.0/lib/active_record/locking/optimistic.rb:69:in `attributes_from_column_definition'
/var/lib/gems/1.9.1/gems/activerecord-3.1.0/lib/active_record/base.rb:1544:in `initialize'
/var/lib/gems/1.9.1/gems/activerecord-3.1.0/lib/active_record/base.rb:508:in `new'
/var/lib/gems/1.9.1/gems/activerecord-3.1.0/lib/active_record/base.rb:508:in `create'
Browsing through the source of ActiveRecord I see the code causing the exception:
# Returns a hash where the keys are column names and the values are
# default values when instantiating the AR object for this table.
def column_defaults
connection_pool.column_defaults[table_name]
end
... the connection_pool.column_defaults is Nil
.
mysql is 0.3.7
em-synchrony is 1.0.0
activerecord is 3.1.0
Thanks for any ideas!
Hi Ilya,
I kept commiting on my fork after my initial pull request.
My last commit jnak@3d28544 was not supposed to be merged in the master tree. Plus, I've found a workaround to use em-sync with blather without patching em-sync itself.
I've already left a comment there #37, but you closed the discussion.
Sorry about that,
Julien
It looks like that I have to add:
require "em-synchrony"
so the following is a bit of a noob question :-$
I've tried the script mentioned in the readme:
require "em-synchrony/em-mysqlplus"
EventMachine.synchrony do
db = EventMachine::Synchrony::ConnectionPool.new(size: 2) do
EventMachine::MySQL.new(host: "localhost")
end
multi = EventMachine::Synchrony::Multi.new
multi.add :a, db.aquery("select sleep(1)")
multi.add :b, db.aquery("select sleep(1)")
res = multi.perform
p "Look ma, no callbacks, and parallel MySQL requests!"
p res
EventMachine.stop
end
and the result of executing it is:
$ ./basics.rb
./basics.rb:14:in `<main>': undefined method `synchrony' for EventMachine:Module (NoMethodError)
I've tried the most basic EM script too but it works fine:
require 'eventmachine'
require 'em-mysqlplus'
EventMachine.run do
conn = EventMachine::MySQL.new(host: 'localhost')
query = conn.query("select sleep(1)")
query.callback {|res| p res.all_hashes }
query.errback {|res| p res.all_hashes }
puts "executing.."
end
the output is:
$ ./basics.rb
executing..
[{"sleep(1)"=>"0"}]
My environment is the following:
$ uname -a
Darwin devuby 10.6.0 Darwin Kernel Version 10.6.0: Wed Nov 10 18:13:17 PST 2010; root:xnu-1504.9.26~3/RELEASE_I386 i386 i386
$ rvm --version
rvm 1.2.5 by Wayne E. Seguin ([email protected]) [http://rvm.beginrescueend.com/]
$ ruby --version
ruby 1.9.2p136 (2010-12-25 revision 30365) [x86_64-darwin10.5.0]
$ gem --version
1.5.0
$ gem list
*** LOCAL GEMS ***
em-mysqlplus (0.1.5)
em-synchrony (0.2.0)
eventmachine (0.12.10)
mysqlplus (0.1.2)
Maurizio
em-mongo references silence_warnings
When I require it I get a failure.
require 'em-synchrony'
require 'em-synchrony/mongo'
require 'em-synchrony/mongoid'
This is easy enough to workaround for the moment by adding these methods myself.
the upcoming em-mongo version uses a different interface to #find and #first. A patch to follow later today that lets it work with both.
I noticed that since Multi#perform is implemented as just Fiber.yield, that it's really a join-type operation instead of a perform like other multi interfaces follow. Consider the following code:
require "em-synchrony/em-http"
EventMachine.synchrony do
multi = EventMachine::Synchrony::Multi.new
multi.add :a, EventMachine::HttpRequest.new("http://www.postrank.com").aget
multi.add :b, EventMachine::HttpRequest.new("http://www.postrank.com").apost
# ADDED LINE!
# All three requests will complete here if there's time.
response = EventMachine::HttpRequest.new("http://www.google.com/").get
res = multi.perform
p "Look ma, no callbacks, and parallel HTTP requests!"
p res
EventMachine.stop
end
From #72
* if a "sync" call is made, then fiber is paused until operation is done and connection is automatically returned to the pool
This is not always true i think.
if the 'sync' call invoke another sync call, the first conn wont be returned back to the pool till the second finished.
In this case, the connection should be reused.
As the current implementation, we will lost the former connection if we call the 'acquire' method twice:
def acquire(fiber)
if conn = @available.pop
@reserved[fiber.object_id] = conn
conn
else
Fiber.yield @pending.push fiber
acquire(fiber)
end
end
and still 'release' twice:
def execute(async)
f = Fiber.current
begin
conn = acquire(f)
yield conn
ensure
release(f) if not async
end
end
def release(fiber)
@available.push(@reserved.delete(fiber.object_id))
if pending = @pending.shift
pending.resume
end
end
then push a 'nil' to the list 'available'.
This is a case to reproduce the problem:
require 'em-synchrony'
CACHE_HASH = {}
class CacheClient < Hash
# the following three methods are all sync
# get value from CACHE_HASH
def get(key)
puts "#{self} get: #{key}"
CACHE_HASH[key]
end
# get value from CACHE_HASH
# if value does not exist
# then yield to accept a value
# then put it into the CACHE_HASH
# and return the value
def fetch(key)
puts "#{self} fetch: #{key}"
if value = get(key)
value
else
value = yield
set key, value
value
end
end
# put the value into CACHE_HASH
def set(key, value)
puts "#{self} set: #{key}, value: #{value}"
CACHE_HASH[key] = value
end
def inspect
"#{self.to_s}<#{super}>"
end
def to_s
"#{self.class.name}@#{self.object_id}"
end
end
CACHE = EM::Synchrony::ConnectionPool.new :size => 2 do
CacheClient.new
end
class CacheUtil
class << self
def get(key)
CACHE.get key
end
def fetch(key)
CACHE.fetch key
end
def set(key, value)
CACHE.set key, value
end
end
end
def log(key)
puts "+" * 20
puts key
puts "available: #{CACHE.instance_variable_get(:@available).inspect}"
puts "available size: #{CACHE.instance_variable_get(:@available).size}"
puts "reserved: #{CACHE.instance_variable_get(:@reserved).inspect}"
puts "reserved size: #{CACHE.instance_variable_get(:@reserved).size}"
puts "CACHE_HASH: #{CACHE_HASH.inspect}"
puts "-" * 20
end
log "0"
# the methods are all sync
CACHE.set "key2", "value2" # put "key2" into cache
CACHE.fetch "key1" do # get "key1", not exist; acquire
log "1"
# what we put into key1 are based on the value of key2
value2 = CACHE.get "key2" # get "key2"; acquire
# release
log "2"
"just_for_test:" + value2 # generate value for "key1" based on value of "key2"
# put the value into "key1"
end # release
log "3"
result:
++++++++++++++++++++
0
available: [CacheClient@5756940<{}>, CacheClient@5756920<{}>]
available size: 2
reserved: {}
reserved size: 0
CACHE_HASH: {}
--------------------
CacheClient@5756920 set: key2, value: value2
CacheClient@5756920 fetch: key1
CacheClient@5756920 get: key1
++++++++++++++++++++
1
available: [CacheClient@5756940<{}>]
available size: 1
reserved: {5755000=>CacheClient@5756920<{}>}
reserved size: 1
CACHE_HASH: {"key2"=>"value2"}
--------------------
CacheClient@5756940 get: key2
++++++++++++++++++++
2
available: [CacheClient@5756940<{}>]
available size: 1
reserved: {}
reserved size: 0
CACHE_HASH: {"key2"=>"value2"}
--------------------
CacheClient@5756920 set: key1, value: just_for_test:value2
++++++++++++++++++++
3
available: [CacheClient@5756940<{}>, nil]
available size: 2
reserved: {}
reserved size: 0
CACHE_HASH: {"key2"=>"value2", "key1"=>"just_for_test:value2"}
--------------------
Hi!
I'm working on providing an em-synchrony friendly version of ZK::EventMachine, but have run into trouble with the way that EM::Synchrony.sync handles callbacks that have their succeed
method invoked with multiple arguments. I make extensive use of this pattern throughout my code, so for me this is a pretty important thing to have working :)
Below I have an example fix for sync. I created a fork and branch of em-synchrony and made the change, but I couldn't figure out how to set up all of the required dependencies to run the specs.
I've included below a small demonstration of the problem (for me) and the proposed fix. If it's easier for you, you can also check out a gist of this code and run it that way.
The change is pretty minor, if you can instruct me on how to configure mongo and mysql, etc. so I can run the test suite, i'd be happy to. If you don't mind making the change in your environment and seeing if there are any ill effects, that'd work too.
#!/usr/bin/env ruby
require 'rubygems'
require 'eventmachine'
require 'em-synchrony'
module EventMachine
module Synchrony
def self.new_sync(df)
f = Fiber.current
xback = proc {|*args|
if f == Fiber.current
return *args
else
f.resume(*args)
end
}
df.callback &xback
df.errback &xback
Fiber.yield
end
end
end
def gimme_deferreds
a, b = EM::DefaultDeferrable.new, EM::DefaultDeferrable.new
EM.next_tick do
a.succeed(1, 2, [:foo, :bar])
b.succeed(:single_arg)
end
return a, b
end
def current_implementation
EM.synchrony do
a, b = gimme_deferreds
rval = EM::Synchrony.sync(a)
puts "a rval: #{rval.inspect}"
rval = EM::Synchrony.sync(b)
puts "b rval: #{rval.inspect}"
EM.next_tick { EM.stop_event_loop }
end
end
def proposed_fix
EM.synchrony do
a, b = gimme_deferreds
rval = EM::Synchrony.new_sync(a)
puts "a rval: #{rval.inspect}"
rval = EM::Synchrony.new_sync(b)
puts "b rval: #{rval.inspect}"
EM.next_tick { EM.stop_event_loop }
end
end
if __FILE__ == $0
puts <<-EOS
call signature:
a.succeed(1, 2, [:foo, :bar])
b.succeed(:single_arg)
EOS
puts "this is the return value of the current implementation:"
current_implementation
puts "\nthis is the proposed change:"
proposed_fix
end
__END__
call signature:
a.succeed(1, 2, [:foo, :bar])
b.succeed(:single_arg)
this is the return value of the current implementation:
a rval: 1
b rval: :single_arg
this is the proposed change:
a rval: [1, 2, [:foo, :bar]]
b rval: :single_arg
Thanks!
Jonathan
http://github.com/mccoder/em-synchrony/commit/0dce0e61217b482f7da7c9705355c2314f5ad94a
Example added in Readme.
Calls to read with a length parameters should only return a string shorter than requests if the stream's EOF has been reached. See http://www.ruby-doc.org/core-1.9.2/IO.html#method-i-read.
The current read implementation returns as there is any data available, without waiting for enough data as requested. The current behavior is sufficient to emulate recv, but not read. It breaks applications that expect read to wait for enough data to arrive.
I am just trying to report as much as I know about this issue. We have a Goliath app using em-mongo. After upgrading the mongo driver to 1.4 thousands of Goliath processes were created which took down the application. If you don't have time to debug this issue as we don't, I would suggest just releasing a new version of em-mongo that depends on mongo < 1.4
require 'em-activerecord'
RuntimeError: Missing EM-Synchrony dependency: gem install em-mysqlplus
from /home/micha/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.1.5/lib/em-synchrony/em-mysqlplus.rb:4:in rescue in <top (required)>' from /home/micha/.rvm/gems/ruby-1.9.2-p0/gems/em-synchrony-0.1.5/lib/em-synchrony/em-mysqlplus.rb:1:in
<top (required)>'
from internal:lib/rubygems/custom_require:29:in require' from <internal:lib/rubygems/custom_require>:29:in
require'
from /home/micha/.rvm/gems/ruby-1.9.2-
The original error should e given, to not swallow any other error.
The problem afaik is the require 'mysqlplus' which does not exist in mysqlplus @ http://github.com/igrigorik/em-mysqlplus/tree/master/lib/
ConnectionPool thinks it got a defferable from em-redis, while it actually got a Fixnum.
require "em-synchrony"
require "em-synchrony/em-redis"
EM.synchrony do
redis = EM::Synchrony::ConnectionPool.new(size: 2) do
EM::Protocols::Redis.connect
end
redis.ainfo { |result| puts results }
end
Run this and you get this error:
/home/david/.bundler/ruby/1.9.1/em-synchrony-4224adb118eb/lib/em-synchrony/connection_pool.rb:73:in
block in method_missing': undefined method
callback' for 14:Fixnum (NoMethodError)
from /home/david/.bundler/ruby/1.9.1/em-synchrony-4224adb118eb/lib/em-synchrony/connection_pool.rb:23:inexecute' from /home/david/.bundler/ruby/1.9.1/em-synchrony-4224adb118eb/lib/em-synchrony/connection_pool.rb:68:in
method_missing'
from main.rb:15:inblock in <main>' from /home/david/.bundler/ruby/1.9.1/em-synchrony-4224adb118eb/lib/em-synchrony.rb:24:in
call'
from /home/david/.bundler/ruby/1.9.1/em-synchrony-4224adb118eb/lib/em-synchrony.rb:24:in `block (2 levels) in synchrony'
Hi everyone,
I have an issue that I believe is not 100% specific to em-synchrony, but I didn't know where to post it.
I'm trying to reuse my ActiveRecords models with em-synchrony and em_mysql2 outside of rails. This means that I don't use @mperham rack-fiber_pool. Because of this, it seems that AR will use as many mysql connections as fibers calling the DB, unlike the em-synchrony connexion pool. I tried using the :pool parameter (which defaults to 5 anyway), but it does not change anything.
Am I doing something wrong ? Does anyone use async AR outside of a rack app ?
Some help here would be much appreciated :)
Cheers,
Julien
I'd like to run the specs! :)
$ bundle
Using addressable (2.2.6)
Using bson (1.3.1)
Using bson_ext (1.3.1)
Using diff-lcs (1.1.2)
Using hiredis (0.3.2)
Using em-hiredis (0.1.0)
Using eventmachine (1.0.0.beta.3) from git://github.com/eventmachine/eventmachine.git (at master) with native extensions
Using em-socksify (0.1.0)
Using http_parser.rb (0.5.1)
Using em-http-request (1.0.0.beta.4) from git://github.com/igrigorik/em-http-request (at master)
Using em-mongo (0.3.6)
Using em-mysqlplus (0.1.5)
Using em-redis (0.3.0)
Using mongo (1.3.1)
Using mysqlplus (0.1.2)
Using remcached (0.4.1)
Using rspec-core (2.6.4)
Using rspec-expectations (2.6.0)
Using rspec-mocks (2.6.0)
Using rspec (2.6.0)
Using bundler (1.0.13)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
$ rake
rake aborted!
undefined method `desc' for #<Bundler::GemHelper:0x8a5a160>
(See full trace by running task with --trace)
So I added Rake to the Gemfile to be able to do this:
$ bundle exec rake
/home/lars/.rvm/rubies/ruby-1.9.2-p180/bin/ruby -S bundle exec rspec ./spec/iterator_spec.rb ./spec/mysqlplus_spec.rb ./spec/hiredis_spec.rb ./spec/mongo_spec.rb ./spec/memcache_spec.rb ./spec/redis_spec.rb ./spec/remcached_spec.rb ./spec/timer_spec.rb ./spec/connection_pool_spec.rb ./spec/em-mongo_spec.rb ./spec/beanstalk_spec.rb ./spec/inlinesync_spec.rb ./spec/tcpsocket_spec.rb ./spec/http_spec.rb
/home/lars/workspace/em-synchrony/spec/iterator_spec.rb:1:in `require': no such file to load -- spec/helper/all (LoadError)
from /home/lars/workspace/em-synchrony/spec/iterator_spec.rb:1:in `<top (required)>'
from /home/lars/.rvm/gems/ruby-1.9.2-p180@em-synchrony/gems/rspec-core-2.6.4/lib/rspec/core/configuration.rb:419:in `load'
from /home/lars/.rvm/gems/ruby-1.9.2-p180@em-synchrony/gems/rspec-core-2.6.4/lib/rspec/core/configuration.rb:419:in `block in load_spec_files'
from /home/lars/.rvm/gems/ruby-1.9.2-p180@em-synchrony/gems/rspec-core-2.6.4/lib/rspec/core/configuration.rb:419:in `map'
from /home/lars/.rvm/gems/ruby-1.9.2-p180@em-synchrony/gems/rspec-core-2.6.4/lib/rspec/core/configuration.rb:419:in `load_spec_files'
from /home/lars/.rvm/gems/ruby-1.9.2-p180@em-synchrony/gems/rspec-core-2.6.4/lib/rspec/core/command_line.rb:18:in `run'
from /home/lars/.rvm/gems/ruby-1.9.2-p180@em-synchrony/gems/rspec-core-2.6.4/lib/rspec/core/runner.rb:80:in `run_in_process'
from /home/lars/.rvm/gems/ruby-1.9.2-p180@em-synchrony/gems/rspec-core-2.6.4/lib/rspec/core/runner.rb:69:in `run'
from /home/lars/.rvm/gems/ruby-1.9.2-p180@em-synchrony/gems/rspec-core-2.6.4/lib/rspec/core/runner.rb:11:in `block in autorun'
rake aborted!
ruby -S bundle exec rspec ./spec/iterator_spec.rb ./spec/mysqlplus_spec.rb ./spec/hiredis_spec.rb ./spec/mongo_spec.rb ./spec/memcache_spec.rb ./spec/redis_spec.rb ./spec/remcached_spec.rb ./spec/timer_spec.rb ./spec/connection_pool_spec.rb ./spec/em-mongo_spec.rb ./spec/beanstalk_spec.rb ./spec/inlinesync_spec.rb ./spec/tcpsocket_spec.rb ./spec/http_spec.rb failed
Tasks: TOP => default => spec
(See full trace by running task with --trace)
I'd like to write a patch for EM::Synchrony.sync, are the specs known to be broken?
HI,
While experimenting with with em_mysql2 adapater I stumbled upon something annoying, while I agree it is an edge case I wonder if someone has a fix, here is my test case: https://gist.github.com/1330609
The problem is that by replacing the synchronize call with simply calling the block if you create multiple fibers asking for the connection they will all get one (I did not looked into this but I suppose the fiber is suspended during the actual connection to mysql). the net result is that you can can have more connections than your maximum in the connection pool, while this is not a major problem it still annoys me.
In my test case I defined a pool of 1 connection with activerecord and by setting the DELAY constant to 0 (or removing the add_timer call) you can get as many connections as you want in the pool although the limit is still 1, just change the "5.times" part.
I get the same result when running the MultiIterator as well.
/Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:25: [BUG] Bus Error
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin11.1.0]
-- control frame ----------
c:0010 p:---- s:0032 b:0032 l:000031 d:000031 CFUNC :resume
c:0009 p:0021 s:0029 b:0029 l:000ef8 d:000538 BLOCK /Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:25
c:0008 p:---- s:0027 b:0027 l:000026 d:000026 FINISH
c:0007 p:---- s:0025 b:0025 l:000024 d:000024 CFUNC :call
c:0006 p:---- s:0023 b:0023 l:000022 d:000022 CFUNC :run_machine
c:0005 p:0248 s:0020 b:0020 l:000019 d:000019 METHOD /Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179
c:0004 p:0056 s:0013 b:0013 l:000ef8 d:000ef8 METHOD /Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:27
c:0003 p:0039 s:0006 b:0006 l:000358 d:001d38 EVAL em-sync.rb:4
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:000358 d:000358 TOP
---------------------------
-- Ruby level backtrace information ----------------------------------------
em-sync.rb:4:in `<main>'
/Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:27:in `synchrony'
/Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `run'
/Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `run_machine'
/Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:179:in `call'
/Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:25:in `block in synchrony'
/Users/bmckim/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:25:in `resume'
-- C level backtrace information -------------------------------------------
0 ruby 0x000000010682f53e rb_vm_bugreport + 110
1 ruby 0x0000000106724193 report_bug + 259
2 ruby 0x0000000106724301 rb_bug + 161
3 ruby 0x00000001067c6452 sigbus + 18
4 libsystem_c.dylib 0x00007fff8cff7cfa _sigtramp + 26
5 ??? 0x0000000106a05680 0x0 + 4406138496
[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html
Abort trap: 6
require "em-synchrony"
require "em-synchrony/em-http"
EM.synchrony do
concurrency = 2
urls = ['http://google.com', 'http://yahoo.com']
# iterator will execute async blocks until completion, .each, .inject also work!
results = EM::Synchrony::Iterator.new(urls, concurrency).map do |url, iter|
# fire async requests, on completion advance the iterator
http = EventMachine::HttpRequest.new(url).aget
http.callback { iter.return(http) }
http.errback { iter.return(http) }
end
p results # all completed requests
EventMachine.stop
end
gem 'eventmachine', '>=1.0.0.beta'
gem 'em-http-request', '1.0.0'
gem 'em-synchrony', '1.0.0'
*** LOCAL GEMS ***
addressable (2.2.6)
bundler (1.0.18)
em-http-request (1.0.0)
em-socksify (0.1.0)
em-synchrony (1.0.0)
eventmachine (1.0.0.beta.4)
http_parser.rb (0.5.2)
I am using the EM-compatable PostgreSQL driver, not MySQL.
I'm trying to get em-synchrony
to run under Thin/Sinatra in conjunction with sinatra_async
. Unfortunately, EM.synchrony
doesn't seem to play well when it's executed within a running reactor.
Is there a simple way for me to implement EM.sync
within a currently active EventMachine loop?
I can't get em-synchrony to work with em-redis.
require 'em-synchrony'
require 'em-synchrony/em-redis'
EM.synchrony do
redis = EM::Protocols::Redis.connect
redis.set('a', 'foobar')
EM.stop
end
results in :
/Users/ivan/.rvm/gems/ruby-1.9.2-p136/gems/em-synchrony-0.2.0/lib/em-synchrony/em-redis.rb:20: stack level too deep (SystemStackError)
What am I missing ?
Thanks in advance for your help.
https://github.com/igrigorik/em-synchrony/blob/master/lib/em-synchrony/connection_pool.rb#L66
If my client has a method 'add', it will be treated as a 'async' method. This might be not as we expected.
it "should treat add method as sync" do
class Client
def add
end
end
pool = EM::Synchrony::ConnectionPool.new :size => 1 do
Client.new
end
EM.run do
EM.synchrony do
pool.add
EM.stop
end
end
end
Have we a better way to do this? Can we use the result returned by the method which we called?
minimum to reproduce:
EM.synchrony do
TCPSocket = EventMachine::Synchrony::TCPSocket
obj = Net::HTTP.new("google.com", 443)
obj.use_ssl = true
obj.verify_mode = OpenSSL::SSL::VERIFY_NONE
obj.get("/")
EM.stop
end
triggers:
ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:658:in `initialize': wrong argument type EventMachine::Synchrony::TCPSocket (expected File) (TypeError)
from /Users/topper/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:658:in `new'
from /Users/topper/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:658:in `connect'
from /Users/topper/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:637:in `do_start'
from /Users/topper/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:626:in `start'
from /Users/topper/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:1168:in `request'
from /Users/topper/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:888:in `get'
When I try to run my test suite using Rails 3.1.0 I with:
gem "em-synchrony", git: "git://github.com/igrigorik/em-synchrony.git", require: [
"em-synchrony",
"em-synchrony/em-http",
"em-synchrony/activerecord"
]
in my Gemfile, and using the em_mysql2
driver it errors before when it tries to run the first test and aborts. although if I revert back to mysql2
driver it works perfectly, this is the full trace on the tests
bundle exec rake test --trace
** Invoke test (first_time)
** Execute test
** Invoke test:units (first_time)
** Invoke test:prepare (first_time)
** Invoke db:test:prepare (first_time)
** Invoke db:abort_if_pending_migrations (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:abort_if_pending_migrations
** Invoke test:functionals (first_time)
** Invoke test:prepare
** Execute test:functionals
Loaded suite /Users/ChristopherHein/.rvm/gems/ruby-1.9.2-p290@global/gems/rake-0.9.2/lib/rake/rake_test_loader
Started
PagesControllerTest:
ERROR should get index (0.03s)
NoMethodError: undefined method callback' for nil:NilClass /Users/ChristopherHein/.rvm/gems/ruby-1.9.2-p290@chrishein/gems/activesupport-3.1.0/lib/active_support/whiny_nil.rb:48:in
method_missing'
Finished in 0.036990 seconds.
1 tests, 0 assertions, 0 failures, 2 errors, 0 skips
** Invoke test:integration (first_time)
** Invoke test:prepare
** Execute test:integration
Errors running test:units, test:functionals!
I might be missing something, but it seems to work fine when I'm using rails 3.1.0.rc4 like your Async-Rails but it seems to be something with the full release of rails...
Any input/suggestions would be greatly appreciated.
Thanks for all the amazing things you hack on btw!
I'm using em-synchrony in Rails3 to perform head requests on about a hundred or so remote urls using EM::HttpRequest. The request runs for about 20 seconds and then returns an empty response to the browser, but continues to run in the background emitting log-lines.
Everything works perfectly if I constrain the number of URLs to around 10 or so, and running thin with the -D flag still demonstrates the same behaviour. The synchronous version of this action runs for over a minute but completes correctly. I also tried bumping up the size of the rack fiber pool, but the same thing happens at around the same time.
Any suggestions would be greatly appreciated. The code in question is available here:
https://github.com/simon-engledew/itube
ruby version: ruby 1.9.2p136 via. RVM on a mac
server: thin 1.2.7
em-synchrony: 0.2.0
When I run the following code :
link to gist: https://gist.github.com/1174519
I keep getting this error :
`block in pipelined': no block given (yield) (LocalJumpError)
Am I missing something?
I am using em-synchrony/em-mongo
on goliath trying to issue find or first return with an error :
[9838:ERROR] 2011-07-13 14:22:20 :: wrong number of arguments (5 for 6)
[9838:ERROR] 2011-07-13 14:22:20 :: /usr/local/lib/ruby/gems/1.9.1/gems/em-mongo-0.3.6/lib/em-mongo/connection.rb:102:in find' /usr/local/lib/ruby/gems/1.9.1/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-mongo.rb:34:in
find'
/usr/local/lib/ruby/gems/1.9.1/gems/em-synchrony-0.3.0.beta.1/lib/em-synchrony/em-mongo.rb:41:in `first'
code:
mongo_db.db('thumbnail-db').collection('thumbnails').first({"url" => url,"width" => width.to_i}, {})
Hi,
the em-synchrony gem does not depend of em-http-request, em-mysqlplus or remcached, but it's not possible to use it without them (LoadError
when I require it). It could be nice to add these dependencies to the gem.
The em-redis wrapper does some work to make redis commands starting with 'a' fiber aware. But it makes sure 'add' is not messed with. However, 'add' is not the only command starting with 'a', 'auth' does as well and is getting munged into the invalid 'uth' command when trying to use a password.
/home/ec2-user/.rvm/gems/ruby-1.9.2-p180/gems/em-redis-0.3.0/lib/em-redis/redis_protocol.rb:335:in `block in initialize': Redis server returned error code: ERR unknown command 'uth' (EventMachine::Protocols::Redis::RedisError)
I'm having some trouble running em-synchrony with the regular mongo gem.
The silence_warnings call in mongo.rb is causing a undefined method error for Object.
silence_warnings is defined in the all spec helper.
So it seems to me the code shouldn't be wrapped in that method, or I might be missing something obvious here :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.