Comments (13)
I have a reproducible compose yml (trigger manually):
version: "3.8"
services:
db:
image: postgres:14-alpine
container_name: postgres_debug_q7Im
restart: unless-stopped
labels:
- docker-volume-backup.stop-during-backup=db
volumes:
- postgres_debug_q7Im:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=1FHJMSwt0yhIN1zS7I4DilGUhThBKq0x
- POSTGRES_USER=q7Im
- POSTGRES_DB=q7Im
backup:
image: offen/docker-volume-backup:latest
restart: always
environment:
BACKUP_FILENAME: "backup-%Y-%m-%dT%H-%M-%S.tar.gz"
BACKUP_FILENAME_EXPAND: "true"
BACKUP_LATEST_SYMLINK: "backup.latest.tar.gz"
BACKUP_RETENTION_DAYS: "7"
BACKUP_PRUNING_PREFIX: "backup-"
BACKUP_STOP_CONTAINER_LABEL: "db"
volumes:
- postgres_debug_q7Im:/backup/my-app-backup:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /root/volbaks_debug:/archive
volumes:
postgres_debug_q7Im:
name: postgres_debug_q7Im
from docker-volume-backup.
Interesting findings. I'll try this branch and let you know in sometime
from docker-volume-backup.
The permissions fix is now included in v2.13.0
from docker-volume-backup.
I moved discussion about the restore procedure to #75
from docker-volume-backup.
Could you provide a reproducible example for this issue? This does not happen for the setups I use this image for so I can't really tell what is happening here judging from two screenshots. It would also be helpful to provide the information requested in the issue template https://github.com/offen/docker-volume-backup/blob/main/.github/ISSUE_TEMPLATE.md
from docker-volume-backup.
Is there anything special about the directories for which this happens? Could it be that these are empty for example?
from docker-volume-backup.
It appears that it's the opposite. Dirs with data in them get owned by root whereas empty dirs have original ownership. Container in question is a standard postgres container. Path being backed up is /var/lib/postgresql/data
inside container.
-
I'm submitting a ...
- [ x] bug report
- feature request
- support request
-
What is the current behavior?
original filesystem permissions/ownership not preserved. some dirs get owned by root
-
If the current behavior is a bug, please provide the configuration and steps to reproduce and if possible a minimal demo of the problem.
-
What is the expected behavior?
keep original filesystem permissions/ownership
-
What is the motivation / use case for changing the behavior?
-
Please tell us about your environment:
- Image version:
- Docker version: 20.10.12
- docker-compose version: v2.2.3
- container: postgres:14-alpine
from docker-volume-backup.
Thanks for the example which helped me reproduce your issue. This seems to be an issue in this repo https://github.com/m90/targz
I'll see if I can find out why this is happening there (it's a bit mysterious to me right now because most work is offloaded to the go stdlib there and I would assume someone had noticed that already). I'll let you know what I find out.
from docker-volume-backup.
One more question: I also noticed in the backup that it's only the directories which are owned by root
, all files are owned by UID 70 (which I would guess is Postgres, correct me if I'm wrong). How exactly do you untar the backup? This could also be something that inadvertently happens on untaring (i.e. by having to sudo
)?
Maybe you could try running Postgres as a user that also exists on your host system (this is explained in the Postges Docker docs) and see if the issue persists?
from docker-volume-backup.
After some digging I think what's happening here is the following: a non-empty directory is not written to the tar archive explicitly, but instead its contents are, implicitly specifying a path of directories to be created if missing.
When you untar such an archive, the "missing" directories will be created and owned by whoever is running tar
at that moment, not who has been the owner at archive time. I can confirm this by mounting such a "broken" backup back into a postgres:14-alpine
container and extracting the archive there, which gives me the expected ownership:
/tmp/backup/my-app-backup $ ls -lah
total 132K
drwx------ 19 postgres postgres 4.0K Feb 20 10:15 .
drwxr-xr-x 3 postgres postgres 4.0K Feb 20 10:15 ..
-rw------- 1 postgres postgres 3 Feb 20 10:07 PG_VERSION
drwxr-xr-x 6 postgres postgres 4.0K Feb 20 10:15 base
drwx------ 2 postgres postgres 4.0K Feb 20 10:15 global
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_commit_ts
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_dynshmem
-rw------- 1 postgres postgres 4.7K Feb 20 10:07 pg_hba.conf
-rw------- 1 postgres postgres 1.6K Feb 20 10:07 pg_ident.conf
drwxr-xr-x 4 postgres postgres 4.0K Feb 20 10:15 pg_logical
drwxr-xr-x 4 postgres postgres 4.0K Feb 20 10:15 pg_multixact
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_notify
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_replslot
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_serial
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_snapshots
drwx------ 2 postgres postgres 4.0K Feb 20 10:15 pg_stat
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_stat_tmp
drwx------ 2 postgres postgres 4.0K Feb 20 10:15 pg_subtrans
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_tblspc
drwx------ 2 postgres postgres 4.0K Feb 20 10:07 pg_twophase
drwx------ 3 postgres postgres 4.0K Feb 20 10:15 pg_wal
drwx------ 2 postgres postgres 4.0K Feb 20 10:15 pg_xact
-rw------- 1 postgres postgres 88 Feb 20 10:07 postgresql.auto.conf
-rw------- 1 postgres postgres 28.0K Feb 20 10:07 postgresql.conf
-rw------- 1 postgres postgres 24 Feb 20 10:07 postmaster.opts
I need to think about how and where to fix this best a little (and if it can even be fixed), but I guess these findings should give you a workaround for now?
from docker-volume-backup.
So I think I fixed the issue in #72 however this means I needed to refactor some vendored code in package targz
that I never really understood. As this functionality is vital to this image, I would prefer not to merge this directly but do some proper testing beforehand so users can be sure there are no regressions in the created archives, potentially breaking restore procedures.
If you find the time, it would be much appreciated if you could try building the version from the dir-ownership
branch
git clone [email protected]:offen/docker-volume-backup.git
cd docker-volume-backup
git checkout dir-ownership
docker build -t offen/docker-volume-backup:local .
and then try to use that image in your setup, checking if the issue persists.
I will also start using this version against some less critical backups of mine so I can observe if there are issues cropping up.
from docker-volume-backup.
I can confirm that it seems to be working fine now. May I suggest an easier instruction to restore on the README?
It just seems unnecessarily complicated. Also it doesn't tell to remove the volume first, which can be a problem if the volume already exists.
This is what I propose:
docker volume rm volume_name
# this removes volume if it exists
docker run --rm -it -v volume_name:/backup/my-app-backup -v /path/to/local_backups:/archive:ro alpine tar -xvzf /archive/full_filename.tar.gz
# this creates volume first
from docker-volume-backup.
Thanks for checking. The branch also works for me, so I'll merge it and release it soon.
from docker-volume-backup.
Related Issues (20)
- Hard to understand error message during exec HOT 3
- archive-pre is triggered on each container backup instead of just container with exec-label HOT 3
- Health Check HOT 6
- Notifications Delay HOT 1
- Replace Log line feeds with HTML <br> tag HOT 3
- Backup the entire content without directory HOT 4
- Backup to subfolders HOT 2
- Resolve date in AWS_S3_PATH HOT 1
- Can't get stop during backup working HOT 2
- Remove local backup after upload to S3 HOT 3
- Invalid memory address crash when running a backup HOT 16
- Fail to configure cron schedule HOT 3
- Docker swarm error HOT 5
- "swarm exec" support HOT 1
- "input/output error" when copying file to NFS mount HOT 4
- Azure storage: Support access tier HOT 2
- schedule the docker-volume-backup? HOT 1
- [Feature] Automated backups of all named volumes HOT 6
- [FR] upgrade `shoutrrr` to v0.8.0 to support custom fields in calls to Generic Webhooks HOT 3
- [Feature] Allow BACKUP_COMPRESSION to be disabled HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-volume-backup.