Crashes almost immediately about rethinkdb HOT 14 CLOSED

Thanks Slava. We're a little inundated with questions, so it'll take some time sort everything out, but we'll investigate this as soon as we can.

from rethinkdb.

Sure, let me know how I can help investigate it. And thanks for looking into it. :)

from rethinkdb.

Hi Slava, our current status when investigating this bug is that we've created a Linux Mint 13 VM inside of VirtualBox (with the same uname -a string except for machine name) and failed to reproduce the problem. The server starts up normally.

This is on a fresh installation of Linux Mint 13, running in a VirtualBox VM on a Ubuntu 12.04 host (running on an i7-2760QM on some Intel 330 or 520 SSD), the filesystem being ext4, no weird installation options.

Does this behavior happen consistently for you?

from rethinkdb.

Thanks for looking.

Yes, it's consistent. I tried deleting the folder. I tried running under sudo, etc..

This is also VirtualBox with Minn 13 upgraded to latest versions of packages.

Is there a way to enable debug or something like that? Or something I can provide that will give a hint of what's going on? Like strace, core dump or something?

from rethinkdb.

Thanks for the quick response. I think running (strace rethinkdb 2>&1) > trace.txt might be very helpful.

Edit: Sorry, ignore this.

from rethinkdb.

Sorry, I meant, try running strace -o trace -f rethinkdb from an empty directory.

Here's the metadata file opening output we get here.

2117  stat("rethinkdb_data/metadata", 0x1d83e80) = -1 ENOENT (No such file or directory)
2117  open("rethinkdb_data/metadata", O_RDWR|O_CREAT|O_DIRECT|O_NOATIME, 0644) = 17
...
2117  stat("rethinkdb_data/metadata", {st_mode=S_IFREG|0644, st_size=524288, ...}) = 0
2117  open("rethinkdb_data/metadata", O_RDWR|O_CREAT|O_DIRECT|O_NOATIME, 0644) = 17

from rethinkdb.

$ cat trace | grep metad
2903  stat("rethinkdb_data/metadata",  <unfinished ...>
2903  open("rethinkdb_data/metadata", O_RDWR|O_CREAT|O_DIRECT|O_NOATIME, 0644) = -1 EINVAL (Invalid argument)

Does that help?

from rethinkdb.

@slava-vishnyakov - what are your mount args for ext4? Are you doing anything special?

from rethinkdb.

I don't think that I'm doing anything special.

This is from fstab:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
UUID=ec69baac-863f-42b8-8845-27822ba2b187 /               ext4    errors=remount-ro 0       1

from rethinkdb.

Oh wait, can the encrypted home folder be the reason?

from rethinkdb.

Yep, it was due to running in encrypted home folder. Runned outside of it - everything works.
You can probably add that to a list of reasons,

Thanks for help, guys!

from rethinkdb.

Awesome we should add this to known issues.

from rethinkdb.

I'm seeing this same issue from within a Vagrant-mounted volume:

vagrant@vagrantbase:/opt/repo$ rethinkdb --config-file /opt/repo/conf/rethinkdb.conf --web-static-directory /opt/repo/rethinkdb/current/web
info: Running rethinkdb 1.3.2-0ubuntu1~precise (GCC 4.6.3)...
info: Loading data from directory /opt/repo/var/rethinkdb/instances
Version: rethinkdb 1.3.2-0ubuntu1~precise (GCC 4.6.3)
error: Inaccessible database file: "/opt/repo/var/rethinkdb/instances/metadata": Invalid argument
       Some possible reasons:
       - the database file couldn't be created or opened for reading and
         writing
       - the database file is located on a filesystem that doesn't support
         O_DIRECT open flag (e.g. some encrypted or journaled file systems)
       - user which was used to start the database is not an owner of the file
Trace/breakpoint trap

from rethinkdb.

I believe this is fixed in 1.4 (which is going to be out in a few days). In 1.4 (and next), if rethink can't open a file under O_DIRECT, it falls back to the page cache and gives the user a warning about potential performance ramifications.

from rethinkdb.