filesystem permissions not backed up correctly about docker-volume-backup HOT 13 CLOSED

I have a reproducible compose yml (trigger manually):

version: "3.8"
services:
  db:
    image: postgres:14-alpine
    container_name: postgres_debug_q7Im
    restart: unless-stopped
    labels:
      - docker-volume-backup.stop-during-backup=db
    volumes:
      - postgres_debug_q7Im:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=1FHJMSwt0yhIN1zS7I4DilGUhThBKq0x
      - POSTGRES_USER=q7Im
      - POSTGRES_DB=q7Im
  backup:
    image: offen/docker-volume-backup:latest
    restart: always
    environment:
      BACKUP_FILENAME: "backup-%Y-%m-%dT%H-%M-%S.tar.gz"
      BACKUP_FILENAME_EXPAND: "true"
      BACKUP_LATEST_SYMLINK: "backup.latest.tar.gz"
      BACKUP_RETENTION_DAYS: "7"
      BACKUP_PRUNING_PREFIX: "backup-"
      BACKUP_STOP_CONTAINER_LABEL: "db"
    volumes:
      - postgres_debug_q7Im:/backup/my-app-backup:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /root/volbaks_debug:/archive

volumes:
  postgres_debug_q7Im:
    name: postgres_debug_q7Im

from docker-volume-backup.

Interesting findings. I'll try this branch and let you know in sometime

from docker-volume-backup.

The permissions fix is now included in v2.13.0

from docker-volume-backup.

I moved discussion about the restore procedure to #75

from docker-volume-backup.

Could you provide a reproducible example for this issue? This does not happen for the setups I use this image for so I can't really tell what is happening here judging from two screenshots. It would also be helpful to provide the information requested in the issue template https://github.com/offen/docker-volume-backup/blob/main/.github/ISSUE_TEMPLATE.md

from docker-volume-backup.

Is there anything special about the directories for which this happens? Could it be that these are empty for example?

from docker-volume-backup.

It appears that it's the opposite. Dirs with data in them get owned by root whereas empty dirs have original ownership. Container in question is a standard postgres container. Path being backed up is /var/lib/postgresql/data inside container.

• I'm submitting a ...

  • [ x] bug report
  • feature request
  • support request

• What is the current behavior?

original filesystem permissions/ownership not preserved. some dirs get owned by root

• If the current behavior is a bug, please provide the configuration and steps to reproduce and if possible a minimal demo of the problem.

• What is the expected behavior?

keep original filesystem permissions/ownership

• What is the motivation / use case for changing the behavior?

• Please tell us about your environment:

  • Image version:
  • Docker version: 20.10.12
  • docker-compose version: v2.2.3
  • container: postgres:14-alpine

from docker-volume-backup.

Thanks for the example which helped me reproduce your issue. This seems to be an issue in this repo https://github.com/m90/targz

I'll see if I can find out why this is happening there (it's a bit mysterious to me right now because most work is offloaded to the go stdlib there and I would assume someone had noticed that already). I'll let you know what I find out.

from docker-volume-backup.

One more question: I also noticed in the backup that it's only the directories which are owned by root, all files are owned by UID 70 (which I would guess is Postgres, correct me if I'm wrong). How exactly do you untar the backup? This could also be something that inadvertently happens on untaring (i.e. by having to sudo)?

Maybe you could try running Postgres as a user that also exists on your host system (this is explained in the Postges Docker docs) and see if the issue persists?

from docker-volume-backup.

After some digging I think what's happening here is the following: a non-empty directory is not written to the tar archive explicitly, but instead its contents are, implicitly specifying a path of directories to be created if missing.

When you untar such an archive, the "missing" directories will be created and owned by whoever is running tar at that moment, not who has been the owner at archive time. I can confirm this by mounting such a "broken" backup back into a postgres:14-alpine container and extracting the archive there, which gives me the expected ownership:

/tmp/backup/my-app-backup $ ls -lah
total 132K   
drwx------   19 postgres postgres    4.0K Feb 20 10:15 .
drwxr-xr-x    3 postgres postgres    4.0K Feb 20 10:15 ..
-rw-------    1 postgres postgres       3 Feb 20 10:07 PG_VERSION
drwxr-xr-x    6 postgres postgres    4.0K Feb 20 10:15 base
drwx------    2 postgres postgres    4.0K Feb 20 10:15 global
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_commit_ts
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_dynshmem
-rw-------    1 postgres postgres    4.7K Feb 20 10:07 pg_hba.conf
-rw-------    1 postgres postgres    1.6K Feb 20 10:07 pg_ident.conf
drwxr-xr-x    4 postgres postgres    4.0K Feb 20 10:15 pg_logical
drwxr-xr-x    4 postgres postgres    4.0K Feb 20 10:15 pg_multixact
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_notify
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_replslot
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_serial
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_snapshots
drwx------    2 postgres postgres    4.0K Feb 20 10:15 pg_stat
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_stat_tmp
drwx------    2 postgres postgres    4.0K Feb 20 10:15 pg_subtrans
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_tblspc
drwx------    2 postgres postgres    4.0K Feb 20 10:07 pg_twophase
drwx------    3 postgres postgres    4.0K Feb 20 10:15 pg_wal
drwx------    2 postgres postgres    4.0K Feb 20 10:15 pg_xact
-rw-------    1 postgres postgres      88 Feb 20 10:07 postgresql.auto.conf
-rw-------    1 postgres postgres   28.0K Feb 20 10:07 postgresql.conf
-rw-------    1 postgres postgres      24 Feb 20 10:07 postmaster.opts

I need to think about how and where to fix this best a little (and if it can even be fixed), but I guess these findings should give you a workaround for now?

from docker-volume-backup.

So I think I fixed the issue in #72 however this means I needed to refactor some vendored code in package targz that I never really understood. As this functionality is vital to this image, I would prefer not to merge this directly but do some proper testing beforehand so users can be sure there are no regressions in the created archives, potentially breaking restore procedures.

If you find the time, it would be much appreciated if you could try building the version from the dir-ownership branch

git clone [email protected]:offen/docker-volume-backup.git
cd docker-volume-backup
git checkout dir-ownership
docker build -t offen/docker-volume-backup:local .

and then try to use that image in your setup, checking if the issue persists.

I will also start using this version against some less critical backups of mine so I can observe if there are issues cropping up.

from docker-volume-backup.

I can confirm that it seems to be working fine now. May I suggest an easier instruction to restore on the README?
It just seems unnecessarily complicated. Also it doesn't tell to remove the volume first, which can be a problem if the volume already exists.

This is what I propose:

docker volume rm volume_name # this removes volume if it exists

docker run --rm -it -v volume_name:/backup/my-app-backup -v /path/to/local_backups:/archive:ro alpine tar -xvzf /archive/full_filename.tar.gz # this creates volume first

from docker-volume-backup.

Thanks for checking. The branch also works for me, so I'll merge it and release it soon.

from docker-volume-backup.