Comments (15)
As already mentioned here: #306 (comment) I surmise your user
directive is what's causing this. Setting a user
is not really supported by this image as it comes with way too many edge cases and problems (#270).
Assuming this is indeed the problem, this is what I'd do in your case is:
- Remove the
user
from your service - Store your backup in an intermediate location
- In a
post-copy
command, copy the intermediate file to your desired location. Apost-copy
command can be run using a different non-root user.
from docker-volume-backup.
Oh! I'll try to do that and report back, thanks a lot
from docker-volume-backup.
Okay so I tried but it isn't working:
## DOCKER VOLUME BACKUP ##
docker-volume-backup:
image: offen/docker-volume-backup:latest
container_name: docker-volume-backup
environment:
DOCKER_HOST: tcp://dockerproxy:2375
BACKUP_CRON_EXPRESSION: "46 13 * * *"
BACKUP_COMPRESSION: "gz"
GZIP_PARALLELISM: 1
BACKUP_FILENAME: backup-%Y-%m-%dT%H-%M-%S.tar.gz
BACKUP_LATEST_SYMLINK: backup-latest.tar.gz
BACKUP_SKIP_BACKENDS_FROM_PRUNE: s3,webdav,ssh,azure,dropbox
BACKUP_ARCHIVE: "/archive"
BACKUP_RETENTION_DAYS: "7"
BACKUP_PRUNING_LEEWAY: "1m"
BACKUP_PRUNING_PREFIX: "backup-"
EXEC_FORWARD_OUTPUT: "true"
EXEC_LABEL: "database"
restart: always
labels:
- docker-volume-backup.copy-post=adduser -S -D 0 -G users
- docker-volume-backup.copy-post=/bin/sh -c 'mv /archive/backup-* /nfs_backups/'
networks:
- managementStackNetwork
- socketProxyNetwork
volumes:
- ${DOCKERCONFDIR}:/backup:ro
- ${DOCKERCONFDIR}/docker-volume-backup/archive:/archive:rw
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
- /nfs/backups/docker-volumes-backups:/nfs_backups:rw
These are the logs:
crond: USER root pid 10 cmd backup 2>&1
time=2023-11-26T13:46:00.777+01:00 level=INFO msg="Stopping 21 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 28 running container(s)."
time=2023-11-26T13:48:54.394+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-26T13-46-00.tar.gz`."
time=2023-11-26T13:49:15.071+01:00 level=INFO msg="Restarted 21 container(s) and the matching service(s)."
time=2023-11-26T13:50:19.412+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-26T13-46-00.tar.gz` in `/archive`." storage=Local
time=2023-11-26T13:50:19.481+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-26T13:50:19.657+01:00 level=INFO msg="None of 1 existing backups were pruned." storage=Local
time=2023-11-26T13:50:19.988+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-26T13-46-00.tar.gz`."
time=2023-11-26T13:50:19.988+01:00 level=INFO msg="Finished running backup tasks."
I probably used the post label wrongly, right? What can I do to make it work?
from docker-volume-backup.
You can only define the label once, i.e. you will likely want to use:
- docker-volume-backup.copy-post=/bin/sh -c 'adduser -S -D 0 -G users && mv /archive/backup-* /nfs_backups/'
from docker-volume-backup.
I'll try and report back when I get the chance, thank you
from docker-volume-backup.
Nothing, didn't work, logs here:
time=2023-11-27T04:00:00.824+01:00 level=INFO msg="Stopping 21 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 30 running container(s)."
time=2023-11-27T04:03:41.000+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-27T04-00-00.tar.gz`."
time=2023-11-27T04:04:00.315+01:00 level=INFO msg="Restarted 21 container(s) and the matching service(s)."
time=2023-11-27T04:06:33.075+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-27T04-00-00.tar.gz` in `/archive`." storage=Local
time=2023-11-27T04:06:33.134+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-27T04:06:33.410+01:00 level=INFO msg="None of 2 existing backups were pruned." storage=Local
time=2023-11-27T04:06:33.899+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-27T04-00-00.tar.gz`."
time=2023-11-27T04:06:33.899+01:00 level=INFO msg="Finished running backup tasks."
And compose here:
...
restart: always
labels:
- docker-volume-backup.copy-post=/bin/sh -c 'adduser -S -D 0 -G users && mv /archive/backup-* /nfs_backups/'
networks:
- managementStackNetwork
- socketProxyNetwork
volumes:
- ${DOCKERCONFDIR}:/backup:ro
- ${DOCKERCONFDIR}/docker-volume-backup/archive:/archive:rw
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
- /nfs/backups/docker-volumes-backups:/nfs_backups:rw
What could it be this time?
from docker-volume-backup.
It's very hard for me to debug this remotely. Do you mount the Docker socket or use DOCKER_HOST
correctly? Without access to the Docker daemon it's not possible to exec commands.
from docker-volume-backup.
Fair enough on your end, I don't know how can I help with debugging if there's a better way I'll gladly set it up, anyways I don't mount the socket directly, I use DOCKER_HOST: tcp://dockerproxy:2375
, which has the correct permissions since other containers use it aswell without a problem.
from docker-volume-backup.
The question in your setup is: why do commands not run at all? Maybe this is about EXEC_LABEL or similar.
from docker-volume-backup.
What do you mean EXEC_LABEL? Anyways I'll test with the full socket mount when I can and report back.
from docker-volume-backup.
Update: I decided to mount the docker.sock directly but still, the commands are not executed. I have one where it's supposed to run a mariadb backup, here:
labels:
- docker-volume-backup.archive-pre=/bin/sh -c 'mysqldump --all-databases > /tmp/dump-$(date +"%Y_%m_%d-%I_%M_%p").sql'
The backup above doesn't get created, I tested the command and it works outside the container, anyways these are the actual logs, nothing unusual:
time=2023-11-28T00:23:00.966+01:00 level=INFO msg="Stopping 3 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 13 running container(s)."
time=2023-11-28T00:24:40.524+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-28T00-23-00.tar.gz`."
time=2023-11-28T00:24:43.783+01:00 level=INFO msg="Restarted 3 container(s) and the matching service(s)."
time=2023-11-28T00:25:21.955+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-28T00-23-00.tar.gz` in `/archive`." storage=Local
time=2023-11-28T00:25:21.957+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-28T00:25:21.964+01:00 level=INFO msg="None of 1 existing backups were pruned." storage=Local
time=2023-11-28T00:25:22.210+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-28T00-23-00.tar.gz`."
time=2023-11-28T00:25:22.211+01:00 level=INFO msg="Finished running backup tasks."
Though I find it pretty weird that it wasn't able to execute commands even with the full socket access. The nfs backup folder is empty unfortunately, any ideas?
from docker-volume-backup.
According to your original post you are setting EXEC_LABEL, which can prevent commands from being run. You can check the docs for details on how this works.
from docker-volume-backup.
I finally understood what you said and some stuff happened! I removed the EXEC_LABEL and look:
crond: USER root pid 10 cmd backup 2>&1
time=2023-11-28T10:12:01.113+01:00 level=INFO msg="Running docker-volume-backup.archive-pre command /bin/sh -c 'mysqldump --all-databases > /tmp/bookstack_db-dump-$(date +\"%Y_%m_%d-%I_%M_%p\").sql' for container bookstack_db"
time=2023-11-28T10:12:01.113+01:00 level=INFO msg="Running docker-volume-backup.archive-pre command bash -c 'pg_dump -U postgres -F t linkwarden_postgres_db | gzip >/var/lib/postgresql/data/backups/linkwarden_postgres_db-dump-$(date +%Y-%m-%d).tar.gz' for container linkwarden_postgres_db"
time=2023-11-28T10:12:01.813+01:00 level=INFO msg="Stopping 24 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 34 running container(s)."
time=2023-11-28T10:15:48.983+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-28T10-12-01.tar.gz`."
time=2023-11-28T10:16:22.683+01:00 level=INFO msg="Restarted 24 container(s) and the matching service(s)."
time=2023-11-28T10:17:47.296+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-28T10-12-01.tar.gz` in `/archive`." storage=Local
time=2023-11-28T10:17:47.304+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-28T10:17:47.335+01:00 level=INFO msg="Running docker-volume-backup.copy-post command /bin/sh -c 'adduser -S -D 0 -G users && mv /archive/backup-* /nfs_backups/' for container docker-volume-backup"
mv: can't create '/nfs_backups/backup-2023-11-28T10-12-01.tar.gz': Permission denied
mv: can't create symlink '/nfs_backups/backup-latest.tar.gz' to 'backup-2023-11-28T10-12-01.tar.gz': Permission denied
time=2023-11-28T10:17:48.421+01:00 level=ERROR msg="Fatal error running backup: runLabeledCommands: error from errgroup: runLabeledCommands: error executing command: exec: running command exited 1"
time=2023-11-28T10:17:48.731+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-28T10-12-01.tar.gz`."
Though as you can see, still nothing in terms of moving the files to the nfs share. I tried running the adduser command (without -S and -S) and I get this output in the container:
/nfs_backups # adduser 0 -G users
adduser: user '0' in use
Honestly the only problem I have now is that it doesn't write to the nfs share. If it's out of scope, I'll figure it out on my own, thank you for your help
from docker-volume-backup.
Ah, I finally did it! I put the user: 0:100
option in my container again and everything went well (the symlink for the latest backup isn't preserved but it's fine):
crond: USER root pid 10 cmd backup 2>&1
time=2023-11-28T18:13:00.834+01:00 level=INFO msg="Running docker-volume-backup.archive-pre command /bin/sh -c 'mysqldump --all-databases > /tmp/bookstack_db-dump-$(date +\"%Y_%m_%d-%I_%M_%p\").sql' for container bookstack_db"
time=2023-11-28T18:13:00.834+01:00 level=INFO msg="Running docker-volume-backup.archive-pre command bash -c 'pg_dump -U postgres -F t linkwarden_postgres_db | gzip >/var/lib/postgresql/data/backups/linkwarden_postgres_db-dump-$(date +%Y-%m-%d).tar.gz' for container linkwarden_postgres_db"
time=2023-11-28T18:13:01.339+01:00 level=INFO msg="Stopping 24 container(s) labeled `docker-volume-backup.stop-during-backup=true` out of 34 running container(s)."
time=2023-11-28T18:16:51.942+01:00 level=INFO msg="Created backup of `/backup` at `/tmp/backup-2023-11-28T18-13-00.tar.gz`."
time=2023-11-28T18:17:17.418+01:00 level=INFO msg="Restarted 24 container(s) and the matching service(s)."
time=2023-11-28T18:19:26.235+01:00 level=INFO msg="Stored copy of backup `/tmp/backup-2023-11-28T18-13-00.tar.gz` in `/archive`." storage=Local
time=2023-11-28T18:19:26.254+01:00 level=INFO msg="Created/Updated symlink `backup-latest.tar.gz` for latest backup." storage=Local
time=2023-11-28T18:19:26.319+01:00 level=INFO msg="Running docker-volume-backup.copy-post command /bin/sh -c 'adduser -S -D 0 -G users && mv /archive/backup-* /nfs_backups/' for container docker-volume-backup"
mv: can't preserve ownership of '/nfs_backups/backup-2023-11-28T10-12-01.tar.gz': Operation not permitted
mv: can't preserve ownership of '/nfs_backups/backup-2023-11-28T18-13-00.tar.gz': Operation not permitted
mv: can't preserve ownership of '/nfs_backups/backup-latest.tar.gz': Operation not permitted
time=2023-11-28T18:25:50.465+01:00 level=INFO msg="None of 0 existing backups were pruned." storage=Local
time=2023-11-28T18:25:50.508+01:00 level=INFO msg="Removed tar file `/tmp/backup-2023-11-28T18-13-00.tar.gz`."
time=2023-11-28T18:25:50.510+01:00 level=INFO msg="Finished running backup tasks."
The thread can be closed for me, if you have a better solution let me know!
from docker-volume-backup.
Happy to hear you got it working.
from docker-volume-backup.
Related Issues (20)
- Is `BACKUP_FILENAME_EXPAND` still working for multiple conf files in `/etc/dockervolumebackup/conf.d` HOT 12
- Add option for Azure Blob Storage connection strings for authentication HOT 4
- loadConfigsFromEnvFiles does not ignore comments in file configs HOT 3
- "Page not found" error when using with balena HOT 9
- Hard to understand error message during exec HOT 3
- archive-pre is triggered on each container backup instead of just container with exec-label HOT 3
- Health Check HOT 6
- Notifications Delay HOT 1
- Replace Log line feeds with HTML <br> tag HOT 3
- Backup the entire content without directory HOT 4
- Backup to subfolders HOT 2
- Resolve date in AWS_S3_PATH HOT 1
- Can't get stop during backup working HOT 2
- Remove local backup after upload to S3 HOT 3
- Invalid memory address crash when running a backup HOT 16
- Fail to configure cron schedule HOT 3
- Docker swarm error HOT 5
- "swarm exec" support HOT 1
- "input/output error" when copying file to NFS mount HOT 4
- Azure storage: Support access tier HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-volume-backup.