I would try the previously working version of docker2singularity. If it fails it could be a space issue. Better yet you can pull directly from a Docker registry with singularity pull docker://image-uri

from docker2singularity.

@vsoch:

Thank you for your reply. I will try your first suggestion. How can I get the previous working version of docker2singularity?

The last time I used docker2singularity was on 2021.09.14, I used the command:
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /tmp/test:/output --privileged -t --rm quay.io/singularity/docker2singularity rstudio-4.0.4-20210914
(where rstudio-4.0.4-20210914 is a docker image that I modified and created on 2021.09.14).

"docker images" command on the systems now shows:
REPOSITORY TAG IMAGE ID CREATED SIZE
rstudio-4.0.4-20211017 latest 3a6abac08567 9 hours ago 2.88 GB
quay.io/singularity/docker2singularity latest 1264dd7994b2 6 weeks ago 416 MB
......

Specifically, what should be the change in my above command? Thanks!

from docker2singularity.

Choose an older tag.

from docker2singularity.

@vsoch:

May you please walk me through how to locate an older tag in my case? I am not experienced on this. Thank you!

from docker2singularity.

Sure, the tags are on the image registry: https://quay.io/repository/singularity/docker2singularity?tab=tags. Try to remember the date when it worked, and choose a tag around then. Try a few. The tag goes after the image name, e.g., quay.io/singularity/docker2singularity:<tag>

from docker2singularity.

@vsoch Thanks again, this helps.

The latest version and the newer version v3.8.2 and v3.8.3 are about 2 months ago. Recall my latest successful work was last month 2021.09.14, I guess I have already used one of these version.
I pulled an older version 3.8.1 and tried the above command again, this time, hit the error:
While performing build: while creating SIF: while creating container: writing data object for SIF file: copying data object file to SIF file: write /tmp/rstudio-4.0.4-20211017-2021-10-18-2142069924bd.sif: no space left on device

This is the same error I saw before I did "docker system prune -a". I am sorry to bother you, how do I get around to this?
"df -h" command shows the following, I am puzzled where could be the issue?

df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 16G 0 16G 0% /dev
tmpfs 16G 0 16G 0% /dev/shm
tmpfs 16G 1.8M 16G 1% /run
tmpfs 16G 0 16G 0% /sys/fs/cgroup
/dev/mapper/VG0-rootlv 10G 8.7G 1.4G 87% /
/dev/mapper/VG0-usrlv 10G 7.8G 2.3G 78% /usr
/dev/sda1 477M 180M 268M 41% /boot
/dev/mapper/VG0-varlv 60G 3.4G 57G 6% /var
/dev/mapper/VG0-tmplv 10G 34M 10G 1% /tmp
/dev/mapper/VG0-optlv 10G 5.6G 4.5G 56% /opt
/dev/mapper/VG0-homelv 70G 41G 30G 58% /home

from docker2singularity.

You don’t have enough room on your filesystem. Make more room or try somewhere else. I also already suggested pulling directly from a Docker registry.

from docker2singularity.

@vsoch: I appreciate your help.

Correct me please, from the df -h command, /var has used only 6%, it total has 60G, 57G available. In linux system the images are stored in /var/lib/docker, I am puzzled why /var does not have space for a 2.8G rstudio container (docker image: rstudio-4.0.4-20211017, size 2883M)?

"pulling directly from a Docker registry" -- what does this mean?

from docker2singularity.

Push the container to a Docker registry and pull down with Singularity, as I showed you earlier. If you need further help please join a singularity slack and ask there.

from docker2singularity.

@vsoch

Sure. I will try this. Thanks again for your helps.

from docker2singularity.

@vsoch:
I am sorry to bother you for help.

On my Redhat 7.9 system, I have an image rstudio-4.0.4-20211017
REPOSITORY TAG IMAGE ID CREATED SIZE
rstudio-4.0.4-20211017 latest 3a6abac08567 15 hours ago 2.88 GB

I have an account on docker.io, I have a repository hpcteam/rstudio-4.0.4 and want to hold this image.

What's the correct command to allow rstudio-4.0.4-20211017 to be pushed to docker.io, the repository?

Thank you.

from docker2singularity.

docker tag <your image name> hpcteam/rstudio-4.0.4 
docker push hpcteam/rstudio-4.0.4 

And you probably will need to login, docker login.

from docker2singularity.

@vsoch: I really appreciate for your quick response.

After I did a tag
$ docker tag rstudio-4.0.4-20211017 hpcteam/rstudio-4.0.4:20211017

I am able to find the new tagged image
[ryao@r1drpswdev3 ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hpcteam/rstudio-4.0.4 20211017 3a6abac08567 15 hours ago 2.88 GB
rstudio-4.0.4-20211017 latest 3a6abac08567 15 hours ago 2.88 GB

Then I tried to push
$ docker push hpcteam/rstudio-4.0.4
The push refers to a repository [registry.access.redhat.com/hpcteam/rstudio-4.0.4]
5fee665b2ff6: Preparing
2743b040803e: Preparing
2c991c434766: Preparing
dac39b3acc9f: Preparing
b696421832a4: Preparing
05f3b67ed530: Waiting
ec1817c93e7c: Waiting
9e97312b63ff: Waiting
e1c75a5e0bfa: Waiting
error parsing HTTP 403 response body: invalid character '<' looking for beginning of value: "\n<TITLE>Access Denied</TITLE>\n\n

Access Denied

\nReference #18.85911160.1634587117.3812c44d\n\n\n"

I noticed that I should I login, so I login docker.io:
$ docker login docker.io
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: hpcteam
Password:
Login Succeeded
$ docker push hpcteam/rstudio-4.0.4
The push refers to a repository [registry.access.redhat.com/hpcteam/rstudio-4.0.4]
5fee665b2ff6: Preparing
2743b040803e: Preparing
2c991c434766: Preparing
dac39b3acc9f: Preparing
b696421832a4: Preparing
05f3b67ed530: Waiting
ec1817c93e7c: Waiting
9e97312b63ff: Waiting
e1c75a5e0bfa: Waiting
error parsing HTTP 403 response body: invalid character '<' looking for beginning of value: "\n<TITLE>Access Denied</TITLE>\n\n

Access Denied

\nReference #18.7c911160.1634587314.2624f38a\n\n\n"

Did I ignore anything in this case? I believe the permission repository hpcteatm/rstudio-4.0.4 in https://hub.docker.com/repositories is public.

from docker2singularity.

Looks like you don't have permission to push to your org.

from docker2singularity.

@vsoch: Thanks!
I notice the first line of the push output:
The push refers to a repository [registry.access.redhat.com/hpcteam/rstudio-4.0.4]
is this supposed to be docker.io/hpcteam/rstudio-4.0.4, do I misconfigure anything on the server side?

from docker2singularity.

That means your docker install defaults to that registry (not typical so I didn't include it). I'd try tagging the image with docker.io and then re-doing the push.

from docker2singularity.

@vsoch 👍 awesome.

Followed your suggestion I tag the image with docker.io and redo the push, it works. Thank you very much for your help.
I guess docker was installed on the system by an rpm provided by redhat, thus it access registry.access.redhat.com by default.

from docker2singularity.

Ha! That's very funny, but in retrospect makes sense. Glad you got it working. Now try pulling down to Singularity:

$ singularity pull docker://docker.io/hpcteam/rstudio-4.0.4

or this will probably work as singularity defaults to Docker Hub

$ singularity pull docker://hpcteam/rstudio-4.0.4

from docker2singularity.

@vsoch 👍

I used singularity pull command
singularity pull docker://docker.io/hpcteam/rstudio-4.0.4:20211017
successfully pulled an image. I will test that shortly.
Again, you have saved my day! I really appreciate your patience with me.

from docker2singularity.

Yep sure thing! Glad it worked. For the future, when you need this kind of back and forth support I recommend a Singularity slack - there are two now :)

from docker2singularity.

@vsoch:
Right. I will do. This time I was originally stuck on docker2singularity convert container, glad you point to a right direction. Have a good evening.

from docker2singularity.

Thanks! Closing the issue since we got you settled. Have a good one!

from docker2singularity.