Giter Club home page Giter Club logo

Comments (15)

m90 avatar m90 commented on August 23, 2024

As already mentioned here: #306 (comment) I surmise your user directive is what's causing this. Setting a user is not really supported by this image as it comes with way too many edge cases and problems (#270).

Assuming this is indeed the problem, this is what I'd do in your case is:

  • Remove the user from your service
  • Store your backup in an intermediate location
  • In a post-copy command, copy the intermediate file to your desired location. A post-copy command can be run using a different non-root user.

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

Oh! I'll try to do that and report back, thanks a lot

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

Okay so I tried but it isn't working:

  ## DOCKER VOLUME BACKUP ##
  docker-volume-backup:
    image: offen/docker-volume-backup:latest
    container_name: docker-volume-backup
    environment:
      DOCKER_HOST: tcp://dockerproxy:2375
      BACKUP_CRON_EXPRESSION: "46 13 * * *"
      BACKUP_COMPRESSION: "gz"
      GZIP_PARALLELISM: 1
      BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
      BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
      BACKUP_SKIP_BACKENDS_FROM_PRUNE: s3,webdav,ssh,azure,dropbox
      BACKUP_ARCHIVE: "/archive"
      BACKUP_RETENTION_DAYS: "7"
      BACKUP_PRUNING_LEEWAY: "1m"
      BACKUP_PRUNING_PREFIX: "backup-"
      EXEC_FORWARD_OUTPUT: "true"
      EXEC_LABEL: "database"
    restart: always
    labels:
      - docker-volume-backup.copy-post=adduser -S -D 0 -G users
      - docker-volume-backup.copy-post=/bin/sh -c 'mv /archive/backup-* /nfs_backups/'
    networks:
      - managementStackNetwork
      - socketProxyNetwork
    volumes:
      - ${DOCKERCONFDIR}:/backup:ro
      - ${DOCKERCONFDIR}/docker-volume-backup/archive:/archive:rw
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
      - /nfs/backups/docker-volumes-backups:/nfs_backups:rw

These are the logs:

crond: USER root pid  10 cmd backup 2>&1
time=2023-11-26T13:46:00.777+01:00 level=INFO msg="Stopping 21 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 28 running container(s)."
time=2023-11-26T13:48:54.394+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-26T13-46-00.tar.gz`."
time=2023-11-26T13:49:15.071+01:00 level=INFO msg="Restarted 21 container(s) and the matching service(s)."
time=2023-11-26T13:50:19.412+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-26T13-46-00.tar.gz` in `/archive`." storage=Local
time=2023-11-26T13:50:19.481+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-26T13:50:19.657+01:00 level=INFO msg="None of 1 existing backups were pruned." storage=Local
time=2023-11-26T13:50:19.988+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-26T13-46-00.tar.gz`."
time=2023-11-26T13:50:19.988+01:00 level=INFO msg="Finished running backup tasks."

I probably used the post label wrongly, right? What can I do to make it work?

from docker-volume-backup.

m90 avatar m90 commented on August 23, 2024

You can only define the label once, i.e. you will likely want to use:

 - docker-volume-backup.copy-post=/bin/sh -c 'adduser -S -D 0 -G users && mv /archive/backup-* /nfs_backups/'

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

I'll try and report back when I get the chance, thank you

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

Nothing, didn't work, logs here:

time=2023-11-27T04:00:00.824+01:00 level=INFO msg="Stopping 21 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 30 running container(s)."
time=2023-11-27T04:03:41.000+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-27T04-00-00.tar.gz`."
time=2023-11-27T04:04:00.315+01:00 level=INFO msg="Restarted 21 container(s) and the matching service(s)."
time=2023-11-27T04:06:33.075+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-27T04-00-00.tar.gz` in `/archive`." storage=Local
time=2023-11-27T04:06:33.134+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-27T04:06:33.410+01:00 level=INFO msg="None of 2 existing backups were pruned." storage=Local
time=2023-11-27T04:06:33.899+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-27T04-00-00.tar.gz`."
time=2023-11-27T04:06:33.899+01:00 level=INFO msg="Finished running backup tasks."

And compose here:

...
 restart: always
    labels:
      - docker-volume-backup.copy-post=/bin/sh -c 'adduser -S -D 0 -G users && mv /archive/backup-* /nfs_backups/'
    networks:
      - managementStackNetwork
      - socketProxyNetwork
    volumes:
      - ${DOCKERCONFDIR}:/backup:ro
      - ${DOCKERCONFDIR}/docker-volume-backup/archive:/archive:rw
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
      - /nfs/backups/docker-volumes-backups:/nfs_backups:rw

What could it be this time?

from docker-volume-backup.

m90 avatar m90 commented on August 23, 2024

It's very hard for me to debug this remotely. Do you mount the Docker socket or use DOCKER_HOST correctly? Without access to the Docker daemon it's not possible to exec commands.

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

Fair enough on your end, I don't know how can I help with debugging if there's a better way I'll gladly set it up, anyways I don't mount the socket directly, I use DOCKER_HOST: tcp://dockerproxy:2375, which has the correct permissions since other containers use it aswell without a problem.

from docker-volume-backup.

m90 avatar m90 commented on August 23, 2024

The question in your setup is: why do commands not run at all? Maybe this is about EXEC_LABEL or similar.

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

What do you mean EXEC_LABEL? Anyways I'll test with the full socket mount when I can and report back.

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

Update: I decided to mount the docker.sock directly but still, the commands are not executed. I have one where it's supposed to run a mariadb backup, here:

labels:
      - docker-volume-backup.archive-pre=/bin/sh -c 'mysqldump --all-databases > /tmp/dump-$(date +"%Y_%m_%d-%I_%M_%p").sql'

The backup above doesn't get created, I tested the command and it works outside the container, anyways these are the actual logs, nothing unusual:

time=2023-11-28T00:23:00.966+01:00 level=INFO msg="Stopping 3 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 13 running container(s)."
time=2023-11-28T00:24:40.524+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-28T00-23-00.tar.gz`."
time=2023-11-28T00:24:43.783+01:00 level=INFO msg="Restarted 3 container(s) and the matching service(s)."
time=2023-11-28T00:25:21.955+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-28T00-23-00.tar.gz` in `/archive`." storage=Local
time=2023-11-28T00:25:21.957+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-28T00:25:21.964+01:00 level=INFO msg="None of 1 existing backups were pruned." storage=Local
time=2023-11-28T00:25:22.210+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-28T00-23-00.tar.gz`."
time=2023-11-28T00:25:22.211+01:00 level=INFO msg="Finished running backup tasks."

Though I find it pretty weird that it wasn't able to execute commands even with the full socket access. The nfs backup folder is empty unfortunately, any ideas?

from docker-volume-backup.

m90 avatar m90 commented on August 23, 2024

According to your original post you are setting EXEC_LABEL, which can prevent commands from being run. You can check the docs for details on how this works.

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

I finally understood what you said and some stuff happened! I removed the EXEC_LABEL and look:

crond: USER root pid  10 cmd backup 2>&1
time=2023-11-28T10:12:01.113+01:00 level=INFO msg="Running docker-volume-backup.archive-pre command /bin/sh -c 'mysqldump --all-databases > /tmp/bookstack_db-dump-$(date +\"%Y_%m_%d-%I_%M_%p\").sql' for container bookstack_db"
time=2023-11-28T10:12:01.113+01:00 level=INFO msg="Running docker-volume-backup.archive-pre command bash -c 'pg_dump -U postgres -F t linkwarden_postgres_db | gzip >/var/lib/postgresql/data/backups/linkwarden_postgres_db-dump-$(date +%Y-%m-%d).tar.gz' for container linkwarden_postgres_db"
time=2023-11-28T10:12:01.813+01:00 level=INFO msg="Stopping 24 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 34 running container(s)."
time=2023-11-28T10:15:48.983+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-28T10-12-01.tar.gz`."
time=2023-11-28T10:16:22.683+01:00 level=INFO msg="Restarted 24 container(s) and the matching service(s)."
time=2023-11-28T10:17:47.296+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-28T10-12-01.tar.gz` in `/archive`." storage=Local
time=2023-11-28T10:17:47.304+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-28T10:17:47.335+01:00 level=INFO msg="Running docker-volume-backup.copy-post command /bin/sh -c 'adduser -S -D 0 -G users && mv /archive/backup-* /nfs_backups/' for container docker-volume-backup"
mv: can't create '/nfs_backups/backup-2023-11-28T10-12-01.tar.gz': Permission denied
mv: can't create symlink '/nfs_backups/backup-latest.tar.gz' to 'backup-2023-11-28T10-12-01.tar.gz': Permission denied
time=2023-11-28T10:17:48.421+01:00 level=ERROR msg="Fatal error running backup: runLabeledCommands: error from errgroup: runLabeledCommands: error executing command: exec: running command exited 1"
time=2023-11-28T10:17:48.731+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-28T10-12-01.tar.gz`."

Though as you can see, still nothing in terms of moving the files to the nfs share. I tried running the adduser command (without -S and -S) and I get this output in the container:

/nfs_backups # adduser 0 -G users
adduser: user '0' in use

Honestly the only problem I have now is that it doesn't write to the nfs share. If it's out of scope, I'll figure it out on my own, thank you for your help

from docker-volume-backup.

shyonae avatar shyonae commented on August 23, 2024

Ah, I finally did it! I put the user: 0:100 option in my container again and everything went well (the symlink for the latest backup isn't preserved but it's fine):

crond: USER root pid  10 cmd backup 2>&1
time=2023-11-28T18:13:00.834+01:00 level=INFO msg="Running docker-volume-backup.archive-pre command /bin/sh -c 'mysqldump --all-databases > /tmp/bookstack_db-dump-$(date +\"%Y_%m_%d-%I_%M_%p\").sql' for container bookstack_db"
time=2023-11-28T18:13:00.834+01:00 level=INFO msg="Running docker-volume-backup.archive-pre command bash -c 'pg_dump -U postgres -F t linkwarden_postgres_db | gzip >/var/lib/postgresql/data/backups/linkwarden_postgres_db-dump-$(date +%Y-%m-%d).tar.gz' for container linkwarden_postgres_db"
time=2023-11-28T18:13:01.339+01:00 level=INFO msg="Stopping 24 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 34 running container(s)."
time=2023-11-28T18:16:51.942+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-28T18-13-00.tar.gz`."
time=2023-11-28T18:17:17.418+01:00 level=INFO msg="Restarted 24 container(s) and the matching service(s)."
time=2023-11-28T18:19:26.235+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-28T18-13-00.tar.gz` in `/archive`." storage=Local
time=2023-11-28T18:19:26.254+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-28T18:19:26.319+01:00 level=INFO msg="Running docker-volume-backup.copy-post command /bin/sh -c 'adduser -S -D 0 -G users && mv /archive/backup-* /nfs_backups/' for container docker-volume-backup"
mv: can't preserve ownership of '/nfs_backups/backup-2023-11-28T10-12-01.tar.gz': Operation not permitted
mv: can't preserve ownership of '/nfs_backups/backup-2023-11-28T18-13-00.tar.gz': Operation not permitted
mv: can't preserve ownership of '/nfs_backups/backup-latest.tar.gz': Operation not permitted
time=2023-11-28T18:25:50.465+01:00 level=INFO msg="None of 0 existing backups were pruned." storage=Local
time=2023-11-28T18:25:50.508+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-28T18-13-00.tar.gz`."
time=2023-11-28T18:25:50.510+01:00 level=INFO msg="Finished running backup tasks."

The thread can be closed for me, if you have a better solution let me know!

from docker-volume-backup.

m90 avatar m90 commented on August 23, 2024

Happy to hear you got it working.

from docker-volume-backup.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.