Why is it necessary to reconnect each iteration for parallelizing ActiveRecord? about parallel HOT 10 CLOSED

Try and see if it works, I had lots of issues without this, maybe it's better in AR3,
for forged processes it definetly has to happen.

You can reconnect once per fork, but thats also tricky :)

from parallel.

I am also having a lot of issues using the Parallel gem and ActiveRecord, and without a decent explanation from you as to why reconnection for every iteration is necessary, I'm forced to conclude that this gem is unsafe with ActiveRecord.

from parallel.

conclude what you want, I don't have any sla's :D

afaik you have to reconnect affter forking, not sure about threading, but I saw a lot of weird issues so far, so I always use forking + reconnect

from parallel.

whoops wrong button

from parallel.

if you can digg down and find any issue let me know, or add it to the readme
even having some reproducable failing test-case would be nice

from parallel.

http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/ConnectionPool.html seems to indicate that though ActiveRecord is thread safe, you have to manage the connection pool manually. This seems like something that this gem could/should support.

from parallel.

This gem should ideally not have to worry about issues of other gems,
if you can fix it without hacking AR and making it opt-in/or a pasteable snippet that would be great :)

from parallel.

I found one possible solution using ActiveRecord::Base.connection_pool.with_connection, and I suggest you add it (or a better example) to the readme:

Parallel.each(Message.where(some_condition).all, :in_threads => 8) do |message|
  ActiveRecord::Base.connection_pool.with_connection do
    message.update_attribute(:some_attribute, some_value)
  end
end

from parallel.

Added it to the readme, let's see if it works :)

from parallel.

It works well for me with mongoid.
Mongoid v2: Mongoid.reconnect!
Mongoid v3: Mongoid.default_session.disconnect

from parallel.