When I delete the folder in the container, it is not synchronized to the host computer. The folder on the host computer is always there.

I have your container running via docker-compose without detaching docker-compose so I can watch the logs. If I change a file on my Mac and then inspect /source by executing a shell in your container, I can see the file changed in source, but there is no reaction in the log and the change does not propogate to destination. Here's the compose section:

  bg-sync:
    image: cweagans/bg-sync
    networks:
      - internal
    volumes:
      - .:/source
      - docroot:/var/www
    privileged: true
    environment:
      SYNC_VERBOSE: 1
      SYNC_DESTINATION: /var/www
      SYNC_MAX_INOTIFY_WATCHES: 40000
      SYNC_NODELETE_SOURCE: 0
      SYNC_EXTRA_UNISON_PROFILE_OPTS: |
       ignore = Name {node_modules,bower_components,.DS_Store}
       ignore = Path docroot/sites/default/files
volumes:
  docroot:

Any recommendation on how to debug?

The sync works nicely the first time, but after doing changes, I can see the updates in /source but no in the destination folder.

I think I'm having the same problem as Mike here: #8, but I'm definitely using Docker for Mac. I can see:

osxfs                   465.6G    371.4G     80.5G  82% /source
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/sda1                73.1G     10.2G     59.1G  15% /Users/aromero2/dev/iop/Common

/Users/aromero2/dev/iop/Common is the sync destination.
Any idea?

Hi,

this is my docker-composer service:

  files_syncer:
    build:
      context: services/files_syncer
      args:
        - USER_ID=${USER_ID}
    volumes:
      - "./:/source:cached"
      - "code:/code"
      - "files_syncer_root:/root"
    environment:
      - SYNC_DESTINATION=/code
      - UNISON_USER=app
      - UNISON_UID=${USER_ID}
      - UNISON_GROUP=app
      - UNISON_GID=${USER_ID}
      - SYNC_MAX_INOTIFY_WATCHES=250000
      - SYNC_PREFER=newer
      - SYNC_VERBOSE=1
    privileged: true

I don't understand, why I get the following message:

Warning: No archive files were found for these roots, whose canonical names are:
	/source
	/code
This can happen either
because this is the first time you have synchronized these roots, 
or because you have upgraded Unison to a new version with a different
archive format.

I let the complete process run (successfully) and stop the watcher.
When I start it again, the message is shown again.
I would have expected, that this message does not show up again.

It would be nice to add this feature like UNISON_OWNER
https://github.com/mickaelperrin/docker-unison-unox

In case you didn't know, docker-for-mac has got a very much needed flag which makes this container not as needed, thankfully. :cached is implemented now and :delegated I think will be soon. They added a blog post now as well. Maybe get a link to it in the readme?

https://blog.docker.com/2017/05/user-guided-caching-in-docker-for-mac/

On Windows the sync works. however, it only works once. If a change is made to the host, nothing will be synced. If you restart the container, the changes are synced. It works fine on Mac.

Can you help?

I assumed it could, but it only syncs to the first service specified in the volumes_from list. Plus: my containers won't share the same mount: with docker inspect shows that each one has a distinct mount point. Is it an intrinsic limitation or am I missing something in the configs?

services:
  web:
    working_dir: /app-sync
    volumes:
      - /app-sync

  spring:
    working_dir: /app-sync
    volumes:
      - /app-sync

  sidekiq:
    working_dir: /app-sync
    volumes:
      - /app-sync

  bg-sync:
    image: cweagans/bg-sync
    volumes:
      - .:/source
    volumes_from:
      - spring
      - web
      - sidekiq
    environment:
      - SYNC_DESTINATION=/app-sync
      - SYNC_MAX_INOTIFY_WATCHES=40000
      - SYNC_VERBOSE=1
    privileged: true

Hi,

I'm getting Destination directory does not exist!
How should I use it?

version: '3'
volumes:
  code:
services:
  bg-sync:
    image: cweagans/bg-sync
    volumes:
      - code:/source
    environment:
      SYNC_VERBOSE: 1
      SYNC_DESTINATION: /source/www
      SYNC_EXTRA_UNISON_PROFILE_OPTS: |
        ignore = Path /source/www/appname/vendor
    privileged: true

The actual path in the code service is /app/www but it doesn't work either .

Is there a way to exclude certain files or folders from being synced? I'm thinking specifically of my .git directory which doesn't need to be synced into the container.

Same docker-compose.yml as in the README only added SYNC_PREFER=newer. Adding files are synced just fine in both directions. Removing files in contrast is synced in only one direction host -> container but not vice versa.

Steps to reproduce:

1. $ git clone https://gist.github.com/leymannx/277095bcc750300e749d5ae185181b74.git bg-sync
2. $ cd bg-sync
3. $ docker-compose up -d (wait until done)
4. $ touch foo.txt
5. $ docker exec -it -w /var/www/myapp bgsync_web_1 bash -c "touch bar.txt"
6. Confirm both files exist in both locations:
   1. $ ls
   2. $ docker exec -it -w /var/www/myapp bgsync_web_1 bash -c "ls"

Now delete the files:

1. Again inside the bg-sync folder from step 1. on the host: $ rm foo.txt
2. $ docker exec -it -w /var/www/myapp bgsync_web_1 bash -c "rm bar.txt"
3. Confirm that bar.txt still exists on the host but both files are removed from the container:
   1. $ ls
   2. $ docker exec -it -w /var/www/myapp bgsync_web_1 bash -c "ls"

macOS 10.13.3
Docker for Mac (stable) 17.12.0-ce-mac49 (21995)

I guess for most people that's just OK as they are more interested to sync from the host into the container which works just fine also for removing files. Also as $ echo "bla bla bla" >> file.txt inside the container is synced still fine to the host. It's only rm that's not synced.

I'm syncing directories which can be modified inside of the container and on the host (ie vendor/node_modules/bower_components). At the moment default config forces nodeletion on source root. It should be a configurable setting to allow more flexibility.

Problem

Ignoring git folder from syncing is wrong as many application may rely on git info like commit hash for versioning and stuff.

Solution

Remove line: https://github.com/cweagans/docker-bg-sync/blob/master/sync.sh#L159

Extra solution

To make ignore patterns configurable

Hello!

I tried to use your solution to fix filesystem speed on the Docker for Mac and ran into the strange issue: synced directory in my app container stays empty. Here is my docker-compose config:

version: "2"
services:
  backend:
    image: idfly/ruby-app
    container_name: backend.proj.local
    working_dir: /app
    entrypoint: ['bundle', 'exec', 'rails', 's', '-p', '80', '-b', '0.0.0.0']
    env_file:
      - ../config.env
      - ./config.env
    links:
      - 'mysql:mysql.proj.local'
      - 'mail:mail.proj.local'
    volumes:
      - /app
      - ../../data:/data

  backend-bg-sync:
    image: cweagans/bg-sync
    volumes:
      - ../../backend:/source
    volumes_from:
      - backend
    environment:
      - SYNC_DESTINATION=/app
      - SYNC_MAX_INOTIFY_WATCHES=40000
      - SYNC_VERBOSE=1
    privileged: true
...
...

So after running docker-compose up I see the error from my backend service Could not locate Gemfile or .bundle/ directory. I commented out entrypoint in my docker-compose file and ran docker-compose run backend ls -la /app. It printed an empty directory.

Here is the output of docker inspect on bg-sync container

"Mounts": [
            {
                "Name": "9c63170dab44816f94a6778e3172c908db2b9381c3496611e482b15f46ead742",
                "Source": "/var/lib/docker/volumes/9c63170dab44816f94a6778e3172c908db2b9381c3496611e482b15f46ead742/_data",
                "Destination": "/app",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Source": "/Users/charlie/CodeProjects/proj/backend",
                "Destination": "/source",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],

Here is the output of docker inspect on backend service container:

"Mounts": [
            {
                "Name": "9c63170dab44816f94a6778e3172c908db2b9381c3496611e482b15f46ead742",
                "Source": "/var/lib/docker/volumes/9c63170dab44816f94a6778e3172c908db2b9381c3496611e482b15f46ead742/_data",
                "Destination": "/app",
                "Driver": "local",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],

As I can see, they both mounts the same directory. I can see my rails app there, and in the bg-sync container everything seems to be ok. Sync between /source and /app in the bg-sync container and my host directory works properly. But /app in my backend container stays empty.

Can you help me to resolve this issue, please?

I am using Docker for Mac 1.12.1, OS X El Capitan 10.11.16

Trying to use bg-sync with linuxserver/radarr but I'm unable to use the rsync path (/downloads) in radarr due to permission issues. I've tried correcting this with the UNISON_USER, UNISON_UID, etc variables (setting them the same as the radarr container variables) but every UNISON_GID I specify is always in use (including the default). And when using radarr without rsync I'm able to bind the volume and use it inside radarr so I know I have the right GID/UID.

`version: '2'

services:
radarr:
image: linuxserver/radarr
container_name: radarr
environment:
- PUID=501
- PGID=20
- TZ=America/Los_Angeles
volumes:
- ~/.docker/radarr/config:/config
# - /Volumes/downloads/movies:/movies
- /downloads
# - /Volumes/downloads/#recycle:/recycle
ports:
- 10.1.0.3:7878:7878
restart: unless-stopped
bg-sync:
image: cweagans/bg-sync
container_name: bg-sync
volumes:
- /Volumes/downloads/_downloading:/source
volumes_from:
- radarr
environment:
- TZ=America/Los_Angeles
- SYNC_DESTINATION=/downloads
- SYNC_MAX_INOTIFY_WATCHES=40000
- SYNC_VERBOSE=1
- UNISON_USER=server
- UNISON_UID=501
- UNISON_GROUP=staff
- UNISON_GID=20
privileged: true
restart: unless-stopped

networks:
default:
external:
name: bt`

Any documentation or pointers on this would be appreciated.

I'm running the following containers:
Nginx:latest
Php-fpm-7.0
MariaDB:latest

I was mounting a WordPress directory to both Nginx and PHP and a MySQL directory to MariaDB. I updated my environment to use bg-sync to create separate volumes for each container above to manage syncing.

I got everything working and it indeed sped thing up. Some pages were still too slow but there was improvement.

Unfortunately, the Nginx container requires the WordPress directory to be mounted before it starts otherwise a restart is required. As you know a restart will stop the container and the sync will be broken. I couldn't figure out a workaround to get all containers up on their own.

I had to add supervisor to the Nginx container so I could log in and start Nginx manually after the WP directory was fully synced. This allowed me to test the performance.

I appreciate your effort in trying to create a workaround for this issue. Hopefully the Docker team will figure it out soon.

Taking the basic docker-compose.yml from the README but SYNC_PREFER: newer and restart: always set. Everything works fine except after you restart the Docker for Mac app (or restart you Mac). From then on the sync is unidirectional again as if SYNC_PREFER=newer never was set.

Steps to reproduce:

1. $ git clone https://gist.github.com/leymannx/5d27bd6f0f0b08b13e58baa4bcd866ae.git bg-sync
2. $ cd bg-sync
3. $ docker-compose up -d (wait until done)
4. $ touch hello.txt
5. $ docker exec -it -w /var/www/myapp bgsync_web_1 bash -c "touch world.txt"
6. Confirm both files exist in both locations:
   1. $ ls
   2. $ docker exec -it -w /var/www/myapp bgsync_web_1 bash -c "ls"

Restart Docker for Mac app (or restart Mac).

1. Again inside the bg-sync folder from step 1. on the host: $ touch alpha.txt
2. $ docker exec -it -w /var/www/myapp bgsync_web_1 bash -c "touch beta.txt"
3. Confirm there's no alpha.txt in the container:
   1. $ ls
   2. $ docker exec -it -w /var/www/myapp bgsync_web_1 bash -c "ls"

macOS 10.13.3
Docker for Mac (stable) 17.12.0-ce-mac49 (21995)

Workaround is to simply leave restart: always out and/or always (re-)start the containers manually $ docker-compose start.

Although the Dockerfile and the sync script is kept brilliantly small I don't have a clue how to fix this.

In the sync.sh, I saw # Files to ignore, and yes, files in path like .git/ were not synced into app container.

However, I couldn't find where to add custom ignore paths (in my example, .phpintel) in README.

So, can u add one more start option to support this without fork this repo and rebuild a container by modify sysc.sh?

Gave this a try using the example configuration. It eventually synced everything from /source into /var/www. However, after the sync was done, I edited a file on my local Mac (that is in the /source mount) and it was never synced to /var/www.

I ssh'd to the sync container and verified that the /source/README.txt file was changed and the /var/www/README.txt was still the original.

Is there a polling interval to set here or is there an issue with the file watching?

Maybe the issue is that the directory on the Mac is mounted into the docker-machine using docker-machine-nfs? Which maybe doesn't generate the events needed here?