Hi,
We use RefineryCMS-blog which is a simple engine within RefineryCMS. acts_as_indexed
is used for indexing posts, comments and categories.
For example, a Post
has the declaration (code snippet):
acts_as_indexed :fields => [:title, :body]
We are running acts_as_indexed-0.7.8
, refinerycms-2.0.9
, refinerycms-blog-2-0-stable
. Refinery 2.0.9 locks us these specific version . We have a lot of Refinery customisations we can't easily upgrade RefineryCMS.
This morning we had the below sequence of errors:
1. EOFError: end of file reached
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:87 in "load"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:87 in "block (3 levels) in operate"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:86 in "open"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:86 in "open"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:86 in "block (2 levels) in operate"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:180 in "block in lock_file"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:177 in "open"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:177 in "open"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:177 in "lock_file"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:83 in "block in operate"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:80 in "each"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:80 in "operate"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:30 in "remove"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/search_index.rb:50 in "remove_record"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/search_index.rb:54 in "update_record"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/class_methods.rb:87 in "index_update"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/instance_methods.rb:28 in "update_index"
2. Errno::ENOENT: No such file or directory - [rails_root]/tmp/index/production/refinery/blog/post/99_111_109.ind.tmp
/usr/local/rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/fileutils.rb:1515 in "stat"
/usr/local/rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/fileutils.rb:1515 in "block in fu_each_src_dest"
/usr/local/rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/fileutils.rb:1531 in "fu_each_src_dest0"
/usr/local/rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/fileutils.rb:1513 in "fu_each_src_dest"
/usr/local/rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/fileutils.rb:508 in "mv"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:169 in "write_file"
3. ArgumentError: dump format error(0x2)
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:87 in "load"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:87 in "block (3 levels) in operate"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:86 in "open"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:86 in "open"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:86 in "block (2 levels) in operate"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:180 in "block in lock_file"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:177 in "open"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:177 in "open"
/vendor/bundle/ruby/1.9.1/gems/acts_as_indexed-0.7.8/lib/acts_as_indexed/storage.rb:177 in "lock_file"
Given the order of errors, I presume that for some reason the index file couldn't be written correctly, which then caused read and write errors.
I have checked the server and we have plenty of hard drive space, there were no deployments at the time and there were no other outages anywhere. I was able to resolve it by deleting the tmp/index
directory, but in the heat of the moment I forgot to keep a backup of the broken file to investigate it further.
We have had the same problem 3 or 4 times in the last 6 months or so it's starting to become a bit of a problem with no known cause or obvious fix.
I realise that you have now moved the codebase onto v0.8.3 but perhaps you could give me some guidance here? Is there anything we can do? It doesn't seem to be a problem with Refinery's integration of acts_as_indexed into their app. Looking at the code there doesn't seem to be any handling of error cases, is it done somewhere else? What should acts_as_indexed do if the index file is somehow corrupted?
Thanks for your help - much appreciated. If you need any other information please let me know.
Dave