Giter Club home page Giter Club logo

Comments (16)

mr-c avatar mr-c commented on May 29, 2024 1

Looks like we are running using the same version of docker

Docker version 20.10.8, build 3967b7d

I wonder if your /tmp is on a different partition than your other data directories.

try TMPDIR=$PWD cwltool arguments.cwl --src Hello.java ?

from common-workflow-language.

mr-c avatar mr-c commented on May 29, 2024 1

Hurray, progress!

If you want to pass extra arguments to cwltool you have to do so before the path to the CWL document and any inputs:

cwltool arguments.cwl --src Hello.java --tmpdir-prefix=$PWD

should be cwltool --tmpdir-prefix=$PWD arguments.cwl --src Hello.java

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024 1

OK, thank you very much.

I will feed back if I figure out the problem.

from common-workflow-language.

mr-c avatar mr-c commented on May 29, 2024

Hello @DawnEve

cwl-runner 1.0.20180302231433

Your cwltool version is over three years old. The latest version is 3.1.20210816212154 Can you upgrade and try again?

https://github.com/common-workflow-language/cwltool#install

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024

Hello @DawnEve

cwl-runner 1.0.20180302231433

Your cwltool version is over three years old. The latest version is 3.1.20210816212154 Can you upgrade and try again?

https://github.com/common-workflow-language/cwltool#install

$ cwl-runner --version
/usr/bin/cwl-runner 1.0.20180302231433

$ cwltool --version
/home/wangjl/.local/bin/cwltool 3.1.20210825140344

My cwltool is installed only weeks ago, it seems more update if the last part is time stamp.

What's the difference between cwltool and cwl-runner ? Their version differs so much.

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024

Thanks for your reply. @mr-c

After updating my cwl-runner, the result is not the same as before, but still not work:

Version info:

$ cwl-runner --version
/usr/bin/cwl-runner 3.1.20210825140344

The script:

$ cat arguments.cwl 
#!/usr/bin/env cwl-runner

cwlVersion: v1.0
class: CommandLineTool
label: Example trivial wrapper for Java 9 compiler
hints:
  DockerRequirement:
    dockerPull: openjdk:9.0.1-11-slim
baseCommand: javac
arguments: ["-d", $(runtime.outdir)]
inputs:
  src:
    type: File
    inputBinding:
      position: 1
outputs:
  classfile:
    type: File
    outputBinding:
      glob: "*.class"

$ cat arguments-job.yml 
src:
  class: File
  path: Hello.java

The results:

$ cwl-runner arguments.cwl arguments-job.yml
INFO /usr/bin/cwl-runner 3.1.20210825140344
INFO Resolved 'arguments.cwl' to 'file:///home/wangjl/test/cwl_test/arguments.cwl'
INFO [job arguments.cwl] /tmp/eedqjde6$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/eedqjde6,target=/TfavYp \
    --mount=type=bind,source=/tmp/3yyw1atf,target=/tmp \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/Hello.java,target=/var/lib/cwl/stg0b1649c3-91bc-411e-b5d8-523aaab07ffc/Hello.java,readonly \
    --workdir=/TfavYp \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --cidfile=/tmp/bmuzwb7g/20210910161633-044329.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/TfavYp \
    openjdk:9.0.1-11-slim \
    javac \
    -d \
    /TfavYp \
    /var/lib/cwl/stg0b1649c3-91bc-411e-b5d8-523aaab07ffc/Hello.java
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/eedqjde6.
See 'docker run --help'.
## stop here, and I enter ctrl+C:
^CTraceback (most recent call last):
  File "/usr/bin/cwl-runner", line 11, in <module>
    sys.exit(main())
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/main.py", line 1249, in main
    tool, initialized_job_order_object, runtimeContext, logger=_logger
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 59, in __call__
    return self.execute(process, job_order_object, runtime_context, logger)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 148, in execute
    self.run_jobs(process, job_order_object, logger, runtime_context)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 255, in run_jobs
    job.run(runtime_context)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 852, in run
    self._execute(runtime, env, runtimeContext, monitor_function)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 343, in _execute
    default_stderr=runtimeContext.default_stderr,
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 992, in _job_popen
    monitor_function(sproc)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 869, in docker_monitor
    time.sleep(1)
KeyboardInterrupt

$ ls /tmp/eedqjde6 ## this dir exists. 
$

And I try to locate the error, using the dir above:

$ docker \
    run \
    -it \
    --mount=type=bind,source=/tmp/eedqjde6,target=/TfavYp \
    --mount=type=bind,source=/tmp/3yyw1atf,target=/tmp \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/Hello.java,target=/var/lib/cwl/stg0b1649c3-91bc-411e-b5d8-523aaab07ffc/Hello.java,readonly \
    --workdir=/TfavYp \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --cidfile=/tmp/bmuzwb7g/20210910161633-044329.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/TfavYp \
    openjdk:9.0.1-11-slim bash
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/eedqjde6.
See 'docker run --help'.
==> at last, when I came back to run this again, this works. But re-run cwl-runner willn't work, and prompt the same error above.


I use the old-fashioned syntax:
$ docker \
    run \
    -it \
    -v /tmp/eedqjde6:/TfavYp \
    -v /tmp/3yyw1atf:/tmp \
    -v /home/wangjl/test/cwl_test/Hello.java:/var/lib/cwl/stg0b1649c3-91bc-411e-b5d8-523aaab07ffc/Hello.java:ro \
    --workdir=/TfavYp \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --cidfile=/tmp/bmuzwb7g/20210910161633-044329.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/TfavYp \
    openjdk:9.0.1-11-slim bash
I have no name!@5cf0d1ccbb86:~$ touch a
touch: cannot touch 'a': Permission denied

I find some exceptions:

  1. Not writable in the working dir: touch a -->Permission denied
  2. Not as root inside Docker: the cmd line is $ not #.

When I remove the line --user=1001:1001 \

$ docker \
    run \
    -it \
    -v /tmp/eedqjde6:/TfavYp \
    -v /tmp/3yyw1atf:/tmp \
    -v /home/wangjl/test/cwl_test/Hello.java:/var/lib/cwl/stg0b1649c3-91bc-411e-b5d8-523aaab07ffc/Hello.java:ro \
    --workdir=/TfavYp \
    --read-only=true \
    --rm \
    --cidfile=/tmp/bmuzwb7g/20210910161633-044329.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/TfavYp \
    openjdk:9.0.1-11-slim bash
root@4a8fc2f19718:~# javac \
>     -d \
>     /TfavYp \
>     /var/lib/cwl/stg0b1649c3-91bc-411e-b5d8-523aaab07ffc/Hello.java
root@4a8fc2f19718:~# ls
Hello.class
root@4a8fc2f19718:~# java Hello
Hello world, from Java!
root@4a8fc2f19718:~# exit
exit

This time, the user inside Docker is root, and the working dir is writable.

This test proved that the dir exists and can be used.

Perhapse removing the line --user=1001:1001 in Docker cmd is crucial. But how can I do that in cwl?
Or there are some other ways to tackle this problem?

The $ cwl-runner arguments.cwl arguments-job.yml prompt the same error as the 1st try:
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/ubc0i2oo.

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024

It is out of my understanding:

Another example:

$ cwl-runner linkfile.cwl arguments-job.yml
INFO /usr/bin/cwl-runner 3.1.20210825140344
INFO Resolved 'linkfile.cwl' to 'file:///home/wangjl/test/cwl_test/linkfile.cwl'
INFO [job linkfile.cwl] /tmp/9e7thfma$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/9e7thfma,target=/AMHdxx \
    --mount=type=bind,source=/tmp/1334halz,target=/tmp \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/Hello.java,target=/AMHdxx/Hello.java,readonly \
    --workdir=/AMHdxx \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --cidfile=/tmp/h5fbdipf/20210910172233-296122.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/AMHdxx \
    openjdk:9.0.1-11-slim \
    javac \
    Hello.java
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/9e7thfma.
See 'docker run --help'.
^CTraceback (most recent call last):
  File "/usr/bin/cwl-runner", line 11, in <module>
    sys.exit(main())
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/main.py", line 1249, in main
    tool, initialized_job_order_object, runtimeContext, logger=_logger
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 59, in __call__
    return self.execute(process, job_order_object, runtime_context, logger)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 148, in execute
    self.run_jobs(process, job_order_object, logger, runtime_context)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 255, in run_jobs
    job.run(runtime_context)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 852, in run
    self._execute(runtime, env, runtimeContext, monitor_function)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 343, in _execute
    default_stderr=runtimeContext.default_stderr,
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 992, in _job_popen
    monitor_function(sproc)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 869, in docker_monitor
    time.sleep(1)
KeyboardInterrupt

Even when we make sure the dir exist and writable, Docker still say the path not exist:

$ docker --version
Docker version 20.10.8, build 3967b7d

## Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-147-generic x86_64)

$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/9e7thfma,target=/AMHdxx \
    --mount=type=bind,source=/tmp/1334halz,target=/tmp \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/Hello.java,target=/AMHdxx/Hello.java,readonly \
    --workdir=/AMHdxx \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --cidfile=/tmp/h5fbdipf/20210910172233-296122.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/AMHdxx \
    openjdk:9.0.1-11-slim bash
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/9e7thfma.
See 'docker run --help'.
wangjl@sustc-HG:~/test/cwl_test$ ls /tmp/9e7thfma/
$ touch /tmp/9e7thfma/xx
wangjl@sustc-HG:~/test/cwl_test$ ls /tmp/9e7thfma/
xx
wangjl@sustc-HG:~/test/cwl_test$ ## The dir exists and is writable.

from common-workflow-language.

mr-c avatar mr-c commented on May 29, 2024

What is your version of Docker, @DawnEve ? Is your user part of the docker group? (Check via the groups command)

What's the difference between cwltool and cwl-runner ? Their version differs so much.

cwl-runner is the generic name for any CWL implementation. cwltool is the reference implementation. Others are listed at https://www.commonwl.org/#Implementations

I'm guessing you installed cwlref-runner which installs cwltool under the cwl-runner name.

The example you gave runs for me:

$ cwltool 927_arguments.cwl --src ./cwltool/schemas/v1.1/tests/Hello.java
INFO /home/michael/cwltool/env3.6/bin/cwltool 3.1.20210816212154
INFO Resolved '927_arguments.cwl' to 'file:///home/michael/cwltool/927_arguments.cwl'
INFO [job 927_arguments.cwl] /tmp/41vlaxub$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/41vlaxub,target=/WFbgBh \
    --mount=type=bind,source=/tmp/k9fqwhsg,target=/tmp \
    --mount=type=bind,source=/home/michael/cwltool/cwltool/schemas/v1.1/tests/Hello.java,target=/var/lib/cwl/stg7ec0e9f2-5543-4077-8844-010642ceb173/Hello.java,readonly \
    --workdir=/WFbgBh \
    --read-only=true \
    --user=1000:1000 \
    --rm \
    --cidfile=/tmp/9ybdn_to/20210910171806-228090.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/WFbgBh \
    openjdk:9.0.1-11-slim \
    javac \
    -d \
    /WFbgBh \
    /var/lib/cwl/stg7ec0e9f2-5543-4077-8844-010642ceb173/Hello.java
INFO [job 927_arguments.cwl] Max memory used: 0MiB
INFO [job 927_arguments.cwl] completed success
{
    "classfile": {
        "location": "file:///home/michael/cwltool/Hello.class",
        "basename": "Hello.class",
        "class": "File",
        "checksum": "sha1$fdb876b40ad9ebc7fee873212e02d5940588642e",
        "size": 184,
        "path": "/home/michael/cwltool/Hello.class"
    }
}
INFO Final process status is success

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024

@mr-c All is normal, except when using docker.

$ docker --version
Docker version 20.10.8, build 3967b7d

## Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-147-generic x86_64)

$ groups
wangjl docker # cannot run docker if not docker group, as I tested before

The same as new version of cwl-runner:

$ cwltool --version
/home/wangjl/.local/bin/cwltool 3.1.20210825140344

# this script is an example from: https://www.commonwl.org/user_guide/08-arguments/index.html
$ cwltool arguments.cwl --src Hello.java
INFO /home/wangjl/.local/bin/cwltool 3.1.20210825140344
INFO Resolved 'arguments.cwl' to 'file:///home/wangjl/test/cwl_test/arguments.cwl'
INFO [job arguments.cwl] /tmp/0jqqh7l6$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/0jqqh7l6,target=/eMzMyO \
    --mount=type=bind,source=/tmp/_6f2za3y,target=/tmp \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/Hello.java,target=/var/lib/cwl/stg121140a0-b52e-47bd-a5a0-9f635095d45c/Hello.java,readonly \
    --workdir=/eMzMyO \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --cidfile=/tmp/517x5twn/20210911094632-704708.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/eMzMyO \
    openjdk:9.0.1-11-slim \
    javac \
    -d \
    /eMzMyO \
    /var/lib/cwl/stg121140a0-b52e-47bd-a5a0-9f635095d45c/Hello.java
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/0jqqh7l6.
See 'docker run --help'.
^CTraceback (most recent call last):
  File "/home/wangjl/.local/bin/cwltool", line 11, in <module>
    sys.exit(run())
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/main.py", line 1403, in run
    sys.exit(main(*args, **kwargs))
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/main.py", line 1249, in main
    tool, initialized_job_order_object, runtimeContext, logger=_logger
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 59, in __call__
    return self.execute(process, job_order_object, runtime_context, logger)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 148, in execute
    self.run_jobs(process, job_order_object, logger, runtime_context)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 255, in run_jobs
    job.run(runtime_context)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 852, in run
    self._execute(runtime, env, runtimeContext, monitor_function)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 343, in _execute
    default_stderr=runtimeContext.default_stderr,
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 992, in _job_popen
    monitor_function(sproc)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 869, in docker_monitor
    time.sleep(1)
KeyboardInterrupt

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024

@mr-c I guess it is a problem of Docker, what is your version of Docker? I can try and re-install another version of Docker.

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024

Amazing! This cmd works! Although leaves a dir to be removed.
I tried workflow example TMPDIR=$PWD cwl-runner 1st-workflow.cwl 1st-workflow-job.yml, and it works, too.

I tried to understand this line, like cwltool arguments.cwl --src Hello.java --tmpdir-prefix=$PWD but failed. Can you give some hints?

As for the /tmp:
/tmp is not a separate partition, perhaps under /.

$ df -h | grep tmp
$ nothing here.
$ df -h
/dev/sda2              234G  205G   18G  93% /
/dev/sda1              511M  5.7M  506M   2% /boot/efi
/dev/sdb               3.6T  2.5T  922G  74% /data

Other disks are mounted upon usage.

It works:

$ TMPDIR=$PWD cwltool arguments.cwl --src Hello.java
INFO /home/wangjl/.local/bin/cwltool 3.1.20210825140344
INFO Resolved 'arguments.cwl' to 'file:///home/wangjl/test/cwl_test/arguments.cwl'
INFO [job arguments.cwl] /home/wangjl/test/cwl_test/ysoa20rz$ docker \
    run \
    -i \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/ysoa20rz,target=/pwwjko \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/m0xnctd4,target=/tmp \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/Hello.java,target=/var/lib/cwl/stgca9e3047-b039-4a07-9a5a-8ea951d08048/Hello.java,readonly \
    --workdir=/pwwjko \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --cidfile=/home/wangjl/test/cwl_test/1tekpgru/20210911150405-524129.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/pwwjko \
    openjdk:9.0.1-11-slim \
    javac \
    -d \
    /pwwjko \
    /var/lib/cwl/stgca9e3047-b039-4a07-9a5a-8ea951d08048/Hello.java
INFO [job arguments.cwl] Max memory used: 12MiB
INFO [job arguments.cwl] completed success
{
    "classfile": {
        "location": "file:///home/wangjl/test/cwl_test/Hello.class",
        "basename": "Hello.class",
        "class": "File",
        "checksum": "sha1$6f2a091492a911598cbc1f01a5c73993a00abb22",
        "size": 427,
        "path": "/home/wangjl/test/cwl_test/Hello.class"
    }
}
INFO Final process status is success

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024

I tested when not under $HOME, like /data where data really stored, it went wrong again.
I may keep $HOME slim and mount data disks on /data, /data2, /data3 and so like on demand, so run under /data is important.

Previous in /home/wangjl/test/cwl_test/,
Now in /data/wangjl/test/testCWL/:

$ cwltool --tmpdir-prefix=$PWD/ arguments.cwl --src Hello.java
INFO /home/wangjl/.local/bin/cwltool 3.1.20210825140344
INFO Resolved 'arguments.cwl' to 'file:///data/wangjl/test/testCWL/arguments.cwl'
INFO [job arguments.cwl] /data/wangjl/test/testCWL/kr_bq7x4$ docker \
    run \
    -i \
    --mount=type=bind,source=/data/wangjl/test/testCWL/kr_bq7x4,target=/WjipXT \
    --mount=type=bind,source=/data/wangjl/test/testCWL/w45v5nap,target=/tmp \
    --mount=type=bind,source=/data/wangjl/test/testCWL/Hello.java,target=/var/lib/cwl/stg500c9c3d-a169-4d9b-a1fd-d5fc7d817785/Hello.java,readonly \
    --workdir=/WjipXT \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --cidfile=/data/wangjl/test/testCWL/yr7ddi1n/20210911200820-184884.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/WjipXT \
    dawneve/openjdk:latest \
    javac \
    -d \
    /WjipXT \
    /var/lib/cwl/stg500c9c3d-a169-4d9b-a1fd-d5fc7d817785/Hello.java
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /data/wangjl/test/testCWL/kr_bq7x4.
See 'docker run --help'.
^CTraceback (most recent call last):
  File "/home/wangjl/.local/bin/cwltool", line 11, in <module>
    sys.exit(run())
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/main.py", line 1403, in run
    sys.exit(main(*args, **kwargs))
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/main.py", line 1249, in main
    tool, initialized_job_order_object, runtimeContext, logger=_logger
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 59, in __call__
    return self.execute(process, job_order_object, runtime_context, logger)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 148, in execute
    self.run_jobs(process, job_order_object, logger, runtime_context)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/executors.py", line 255, in run_jobs
    job.run(runtime_context)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 852, in run
    self._execute(runtime, env, runtimeContext, monitor_function)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 343, in _execute
    default_stderr=runtimeContext.default_stderr,
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 992, in _job_popen
    monitor_function(sproc)
  File "/home/wangjl/.local/lib/python3.6/site-packages/cwltool/job.py", line 869, in docker_monitor
    time.sleep(1)
KeyboardInterrupt

I did some basic test about Docker, who will not start up under /data:

## OK under /home
$ docker run -it --mount type=bind,source=/home/,target=/eMzMyO --rm  openjdk:9.0.1-11-slim bash
root@4703b7b8c0e3:/# exit

## Error under /data
$ docker run -it --mount type=bind,source=/data/,target=/eMzMyO --rm  openjdk:9.0.1-11-slim bash
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /data/.
See 'docker run --help'.

## Even works under /:
$ docker run -it --mount type=bind,source=/,target=/eMzMyO --rm  openjdk:9.0.1-11-slim bash
root@8c387e7d6ff7:/# exit
exit

But I couldn't find their difference between /data and /home, (/home is under /):

$ ls -lth / # the same privilege
drwxr-xr-x  10 root root 4.0K Sep  3 16:37 data
drwxr-xr-x   9 root root 4.0K Sep  3 15:36 home


$ df -lhT # both ext4
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sdb       ext4      3.6T  2.5T  922G  74% /data
/dev/sda2      ext4      234G  205G   18G  93% /

from common-workflow-language.

mr-c avatar mr-c commented on May 29, 2024

Okay, glad to see the progress. I'm fairly certain that this is not an issue with the CWL reference implementation (cwltool), so I'd like to close it for now.

If you find the solution, please tell us at https://cwl.discourse.group/ (as this repository is more for gathering feedback on the CWL standards themselves, and not technical support with a particular CWL runner nor software container engine like Docker).

Perhaps the real cause is something we can detect and warn the user about; in which case we can add code to do that over in https://github.com/common-workflow-language/cwltool

I hope the rest of your journey with CWL and cwltool goes more smoothly!

from common-workflow-language.

mr-c avatar mr-c commented on May 29, 2024

One last piece of advice, I see that your Ubuntu and Linux kernel version is rather old. Upgrading to newer version of Ubuntu like 20.04 LTS would give you version 5.4.0.84.88 of the Linux kernel; I'm guessing that will have new features and fixes that should improve your software container experience

Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-147-generic x86_64)

from common-workflow-language.

DawnEve avatar DawnEve commented on May 29, 2024

I update to Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-85-generic x86_64), but problem remains.

I figured out the problem from seveval posts and discussion with my classmates.
Something always went wrong with snap installed Docker, but apt installed version works.

That means:

  1. Do NOT install Docker using snap, as time is far away, I am not sure what cmd I used.
    But we can know from ps that the old docker path is /run/snap.docker, and /data is not in the namespace of Docker pid.
$ ps -aux | grep docker
root        1032  0.2  0.2 2157984 91868 ?       Ssl  19:54   0:25 dockerd --group docker --exec-root=/run/snap.docker --data-root=/var/snap/docker/common/var-lib-docker --pidfile=/run/snap.docker/docker.pid --config-file=/var/snap/docker/796/config/daemon.json
root        1411  0.3  0.1 1946568 48316 ?       Ssl  19:54   0:38 containerd --config /run/snap.docker/containerd/containerd.toml --log-level error
root        1517  0.0  0.2 2048712 77528 ?       Ssl  19:54   0:04 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
wangjl     73749  0.0  0.0   6432  2656 pts/1    S+   22:37   0:00 grep --color=auto docker


$ sudo nsenter -m -t1032
-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
root@sustc-HG:/# ls /data
ls: cannot access '/data': No such file or directory
root@sustc-HG:/# ls /home
chengww  hgyi  linuxbrew  wangjl 
root@sustc-HG:/# exit
logout
  1. It is OK to install Docker using apt on Ubuntu. /data now is under Docker's namaspace.
$ sudo apt-get autoremove docker docker-ce docker-engine  docker.io  containerd runc
$ sudo apt-get remove docker docker-engine docker.io containerd runc
$ sudo snap remove docker
$ docker --version
-bash: /usr/bin/docker: No such file or directory

$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io 
$ docker --version
Docker version 20.10.8, build 3967b7d

$ ps -aux | grep docker
root       97273  0.3  0.2 1975672 85152 ?       Ssl  23:13   0:01 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
wangjl     98355  0.0  0.0   6432   664 pts/1    R+   23:19   0:00 grep --color=auto docker
wangjl@sustc-HG:~$ sudo nsenter -m -t97273
root@sustc-HG:/# ls /data
chengww  hgyi  lost+found  public  wangjl 
root@sustc-HG:/# ls /home
chengww  hgyi  linuxbrew  wangjl
root@sustc-HG:/# exit
logout
  1. With apt installed the same version of Docker, the cwl performs well:
$ cwl-runner arguments.cwl arguments-job.yml
INFO /home/wangjl/.local/bin/cwl-runner 2.0.20200224214940
INFO Resolved 'arguments.cwl' to 'file:///data/wangjl/test/testCWL/arguments.cwl'
INFO [job arguments.cwl] /tmp/5ezlcczn$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/5ezlcczn,target=/TKZQjn \
    --mount=type=bind,source=/tmp/8jd5_fpd,target=/tmp \
    --mount=type=bind,source=/home/wangjl/test/cwl_test/Hello.java,target=/var/lib/cwl/stgaeaf45a3-1d98-4171-b5e9-25af960cedfb/Hello.java,readonly \
    --workdir=/TKZQjn \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/TKZQjn \
    --cidfile=/tmp/cbgnbeea/20210912233720-224249.cid \
    openjdk:9.0.1-11-slim \
    javac \
    -d \
    /TKZQjn \
    /var/lib/cwl/stgaeaf45a3-1d98-4171-b5e9-25af960cedfb/Hello.java
INFO [job arguments.cwl] Max memory used: 0MiB
INFO [job arguments.cwl] completed success
{
    "classfile": {
        "location": "file:///data/wangjl/test/testCWL/Hello.class",
        "basename": "Hello.class",
        "class": "File",
        "checksum": "sha1$6f2a091492a911598cbc1f01a5c73993a00abb22",
        "size": 427,
        "path": "/data/wangjl/test/testCWL/Hello.class"
    }
}
INFO Final process status is success

from common-workflow-language.

mr-c avatar mr-c commented on May 29, 2024

Thanks for the detailed analysis! I'll open an issue with cwltool: that we should check for the docker snap if there is a failure and recommend against that installation method.

from common-workflow-language.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.