Docker not working in macOS / OSX about mldb HOT 18 CLOSED

For the shared folder it says that the folder I added is overlapped with Users folder - perhaps it is already shared.

My output as below:

$ ls -lan ~/mldb_data
total 0
drwxr-xr-x    3 503  20   102 Jan 18 10:28 .
drwxr-xr-x+ 151 502  20  5134 Jan 18 10:23 ..
-rw-r--r--    1 503  20     0 Jan 18 10:28 123

$ docker run --rm=true -v ~/mldb_data:/mldb_data -e MLDB_IDS="`id mldb`"   -p 127.0.0.1:5555:80 quay.io/datacratic/mldb:latest  ls -la /mldb_data
total 4
drwxr-xr-x  3 0 0  102 Jan 18 02:28 .
drwxr-xr-x 58 0 0 4096 Jan 18 02:28 ..
-rw-r--r--  1 0 0    0 Jan 18 02:28 123

The mldb_data is owned by 503 and not root, not sure why it says that its owned by root.

from mldb.

You can use this workaround in the meantime.

docker run  --entrypoint="chown" --rm=true -v ~/mldb_data:/mldb_data quay.io/datacratic/mldb:latest   -R _mldb: /mldb_data

This will change the ~/mldb_data owner to the one Docker is expecting. Once you've run the command you can start MLDB normally.

from mldb.

Can you post the command line you're using to start the docker container?

On Nov 13, 2016 2:27 AM, "Davis" [email protected] wrote:

Hi guys, I've been trying to get the MLDB running on my local using docker
from quay.io/datacratic/mldb:latest

However, I've been facing with the following error:

Aborting boot: directory mapped to /mldb_data owned by root

FYI, I've created a new user called mldb (non-root) and change command to id
mldb to use the non-root user, however it still returns the same error.

Any help would be appreciated.

OS: macOS 10.12.1
Docker: 1.12.3-beta30

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#750, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACQXTY6Hy23mxf-5nJrtGasWfVTAxf8qks5q9rv7gaJpZM4KwoVb
.

from mldb.

Given the default command in the documentation:

docker run --rm=true \ -v </absolute/path/to/mldb_data>:/mldb_data \ -e MLDB_IDS="id" \ -p 127.0.0.1:<mldbport>:80 \ quay.io/datacratic/mldb:latest

What did you put for </absolute/path/to/mldb_data>? What are the permissions of that folder?

Are you at PyconCA? If so plz send an email to [email protected] so we can find each other and I can help you fix it before the tutorial.

from mldb.

Here's the command that I ran:
docker run --rm=true -v ~/mldb_data:/mldb_data -e MLDB_IDS="id mldb" -p 127.0.0.1:5555:80 quay.io/datacratic/mldb:latest

I did create the folder manually and also tried removing it, which it was auto-created when running the docker.

And I'm not at PyconCA.

from mldb.

Hi @davistan , can you show the permissions of the ~/mldb_data folder?
It should be owned by the mldb user, otherwise mldb won't be able to write to it (hence the abort).

Finally, I guess it is github markdown that is eating the back ticks, but just to make sure here's what the command should look like:

docker run --rm=true -v ~/mldb_data:/mldb_data -e MLDB_IDS="`id mldb`" \
  -p 127.0.0.1:5555:80 quay.io/datacratic/mldb:latest

(along with a chown mldb:mldb ~/mldb_data)

from mldb.

Yes, I did that too before creating this issue. Here's my ls dump:

drwxr-xr-x+  2 mldb      staff     68 Nov 13 15:11 mldb
drwxr-xr-x+  2 mldb      staff     68 Nov 13 15:12 mldb_data

The mldb is another directory I tried, but still the same result.
One possibility is I saw something that staff is also part of root, which may be the issue?

The mldb (standard) user is created from the macOS's System Preferences -> Users & Groups

from mldb.

Exact same issue.

from mldb.

Searching from code - looks like this line responsible for the issue.

Perhaps someone can fix this or remove this check?

from mldb.

Unfortunately, the fix is not that simple.
The check is there as a safeguard, if it is removed, mldb will encounter write errors when it won't be able to write to its home directory.

I didn't have a chance to test this on osx or macos, but from the info you have shown here, it seems that docker maps the volumes with root permission instead of the original permissions, which is weird.

When mapping the mldb_data folder from your mac to the container, did you add the source directory as a shared folder in the docker configuration. See instruction here: https://docs.docker.com/docker-for-mac/#/file-sharing

Also, to confirm that you're seeing the real directory in the docker, could you:

• create a file in the mldb_data on your mac:
  touch .../mldb_data/testing.123; ls -lan ...mldb_data/testing.123
• boot a container based on the mldb image and run a simple ls to show the mldb_data directory contents:

  docker run --rm=true -v ~/mldb_data:/mldb_data -e MLDB_IDS="`id mldb`" \
    -p 127.0.0.1:5555:80 quay.io/datacratic/mldb:latest  ls -lan /mldb_data
  

Both command should show similar output:

$ ls -lan ~/mldb_data
total 8
drwxrwxr-x  2 12019 12019 4096 Jan 18 02:06 .
drwxr-xr-x 21 12019 12019 4096 Jan 18 02:06 ..
-rw-rw-r--  1 12019 12019    0 Jan 18 02:06 test.123

$ docker run --rm=true -v ~/mldb_data:/mldb_data -e MLDB_IDS="`id mldb`"   -p 127.0.0.1:5555:80 quay.io/datacratic/mldb:latest  ls -la /mldb_data
total 8
drwxrwxr-x  2 12019 12019 4096 Jan 18 02:06 .
drwxr-xr-x 58 root  root  4096 Jan 18 02:07 ..
-rw-rw-r--  1 12019 12019    0 Jan 18 02:06 test.123

Can you run the above commands and post the result?

Thanks.

from mldb.

thanks for the quick feedback.

Can you try running the ls as the _mldb user in the docker?

docker run [...] sudo -u _mldb ls -lan /mldb_data

I know this test sounds crazy, but this paragraph from the docker docs sounds crazier (emphasis mine):

Ownership

Initially, any containerized process that requests ownership metadata of an object **is told that its uid and gid own the object. **

When any containerized process changes the ownership of a shared file system object, e.g. with chown, the new ownership information is persisted in the com.docker.owner extended attribute of the object.

Subsequent requests for ownership metadata will return the previously set values.

Ownership-based permissions are only enforced at the OS X file system level with all accessing processes behaving as the user running Docker.

So if I'm reading this correctly, whoever reads the file attributes is told that it owns the file, while the real permissions are enforced by the osx file system and thus opaque to docker.

from mldb.

Yes, looks like it - confusing on docker side.

Here's my output after running with sudo -u _mldb:

total 4
drwxr-xr-x  3 1234567 1234567  102 Jan 18 02:28 .
drwxr-xr-x 58       0       0 4096 Jan 18 02:59 ..
-rw-r--r--  1 1234567 1234567    0 Jan 18 02:28 123

from mldb.

that's some mighty magic voodoo, I wonder if changes are persistent.

Maybe we can get crazier, can you try this:

docker run [...] chown -R 503:20 /mldb_data

And then run the original docker run command to start mldb?

(You're my remote eyes and hands here, sorry about that, but I'm genuinely curious about all that weird permission behaviour. It is a good contender for the worst offender of the principle of least surprise for 2017!)

from mldb.

Don't mind to help on some open source projects.

I got the below when trying chown.
chown: changing ownership of '/mldb_data/123': Operation not permitted
chown: changing ownership of '/mldb_data': Operation not permitted

I think in the macOS it is 503 but in docker, it is 0 or 1234567 (via _mldb).
So chown 503 inside docker is not recognized either.

from mldb.

Hmm ok. osxfs is probably started by your user (or some other user) who doesn't have the required rights to chown files to '503' (mldb) on osx.

I have a better understanding of the issue now, thanks for your help.

Until we figure a fix for this issue, may I suggest trying out mldb on https://mldb.ai ?

from mldb.

from mldb.

I am having the same issues using MLDB on my Mac Laptop.

Tried all the things as mentioned in the comments above and none worked. May I ask for the better documentation for kickstarting MLDB on Mac.

from mldb.

MLDB now builds on OSX, which means it no longer needs to run under Docker. Please re-open the issue if it's still relevant and you're still having problems.

from mldb.