I'm getting a "load_missing_constant" issue when I try to use script/generate. I'm using ree-1.8.7-2010.02 and bundler. I'm also using navvy 0.3.2. Any idea what's going on?

Thanks in advance,
Scott

$ script/generate navvy --active_record
/Users/ScottMessinger/.rvm/gems/ree-1.8.7-2010.02@mongocc/gems/activesupport-2.3.9/lib/active_support/dependencies.rb:466:in load_missing_constant': uninitialized constant Rails::Railtie (NameError) from /Users/ScottMessinger/.rvm/gems/ree-1.8.7-2010.02@mongocc/gems/activesupport-2.3.9/lib/active_support/dependencies.rb:106:inconst_missing'
from /Users/ScottMessinger/.rvm/gems/ree-1.8.7-2010.02@mongocc/gems/navvy-0.3.2/lib/navvy.rb:27
from /Users/ScottMessinger/.rvm/gems/ree-1.8.7-2010.02@mongocc/gems/activesupport-2.3.9/lib/active_support/dependencies.rb:182:in require' from /Users/ScottMessinger/.rvm/gems/ree-1.8.7-2010.02@mongocc/gems/activesupport-2.3.9/lib/active_support/dependencies.rb:182:inrequire'
from /Users/ScottMessinger/.rvm/gems/ree-1.8.7-2010.02@mongocc/gems/activesupport-2.3.9/lib/active_support/dependencies.rb:547:in new_constants_in' from /Users/ScottMessinger/.rvm/gems/ree-1.8.7-2010.02@mongocc/gems/activesupport-2.3.9/lib/active_support/dependencies.rb:182:inrequire'
from /Users/ScottMessinger/code/mongocc/config/initializers/navvy.rb:1

Hi,

When running multiple Navvy workers concurrently, it seems like workers are able to pick up each others jobs when invoked exactly at the same moment.

For example, say I have 25 jobs queued, and I spawn 4 workers at the same time, it seems that 2 workers are taking on the first job in the queue, and the other 2 are taking on the second job in the queue (instead of just picking up 4 jobs, one per worker).

I already saw that Job.next(1) will ensure that only the next available job will be fetched and picked up. However, I believe that since I spawn multiple workers at the exact same time, that the first worker picks up the job and marks it as started, but before it can mark it as started, the other worker already pulled down that same object, meaning that it assumes it can also process that job. (So then it gets marked as started twice basically, by two different workers).

Also, I noticed this in for example the Mongoid extension:

def self.next(limit = self.limit)
  where(:failed_at => nil).
  where(:completed_at => nil).
  where(:run_at.lte => Time.now).
  order_by([[:priority, :desc], [:created_at, :asc]]).
  limit(limit).to_a
end

Is there a reason why you don't include a where(:started_at => nil)? Judging by this, and the fact that you can pass in a limit to pull in an array of objects to process, it seems like Navvy wasn't meant to run multiple workers concurrently, or am I missing something?

In any case, by using Job.next(1) in the rake task, and adding the where(:started_at => nil) to the Mongoid extension, it partially solves the problem, but there will still be times where a single job will be processed by 2 or more workers, which of course is not what you want and will cause issues. Looking for a good solution.

I guess the main question is: Can (or should) Navvy be able to run multiple workers concurrently?

Let me know, thanks!

The #next method for the ActiveRecord adapter calls Time.now in the query. Time.now does not get converted to db time. This should be changed to Time.current to respect the database's time zone.

Running Navvy::Worker.start
After it runs my task, it calls count_all and raises this issue:
Mysql::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '== '4' OR parent_id == '4') AND failed_at IS NOT NULL)' at line 1: SELECT count(*) AS count_all FROM jobs WHERE ((id == '4' OR parent_id == '4') AND failed_at IS NOT NULL).

The problem is it should be id= '4' OR parent_id = '4' (and not double '==')

Trying to run a worker after following the setup guide for rails 2.3+

Hello!

I know development on Navvy wasn't active for two years now and my question may be hard to answer because of that.

I found Navvy to suit my needs and want to use it on Heroku. Missing concurrency is no issue for me, due to financial limitations I'll run one worker at most. In order to further save money I'd like to write something like HireFire for Navvy: Hire the worker when there are jobs, else fire him.

My attempt was to write a plugin for the Sequel adapter of Navvy::Job and extend the before_create and after_destroy hooks. It didn't work, so I looked at Navvy's source code. I discovered that Navvy avoids calling hooks of Sequel::Model; it uses insert() instead of create() and delete() instead of destroy().

Could you please explain why this is the case? I don't want to fork Navvy, force it using Sequel hooks and notice it doesn't work as expected.

Thank you very much in advance!

I'm currently developing two seperate apps that run on the same MySQL server. I just introduced navvy in one project, but I'm pretty sure I will be using it in the second project soon.

Because they share the same database, both navvy workers will use the same jobs table. I expect this to cause problems.

In order to prevent these problems I was wondering if there is a way to create jobs in a namespace and let the workers query for just these jobs. If not, consider this a feature request :D!

pure ruby set-up guide missing

The backticks used to escape column names in the active_record adapter are causing issues when using the Postgres AR adapter. Postgres uses double quotes for escaping column names (IIRC), with backticks having a different function.

The obvious solution is just to remove all the backticks from job/active_record.rb, but I wonder if there is a way to escape the column names through AR?