kouroshparsa / parallel_sync Goto Github PK

When no private key is specified, you'd get this error.
You need to improve the error handling and also try using the default ssh key at ~/.ssh/id_rsa

  File "/var/lib/jenkins/venv/lib/python2.7/site-packages/parallel_sync/rsync.py", line 42, in download
    validate=validate, additional_params=additional_params)
  File "/var/lib/jenkins/venv/lib/python2.7/site-packages/parallel_sync/rsync.py", line 63, in transfer
    creds.key = os.path.expanduser(path)

Hi
I try to do like example but it is not working:

from fabric.api import env
from parallel_sync import rsync
rsync.upload('/tmp/x', '/tmp/y', creds=env)
rsync.download('/tmp/y', '/tmp/z', creds=env)

Are you tested before?

Not seeing a way to add a non standard ssh port, in my case i need to connect to port 2844 on the remote host. I tried modifying the code, but didnt have much luck. Any suggestions? Command works with rsync by itself, but would like to try with parallel_sync.

Essentially, I need to add -e "ssh -p 2844" to the rsync command.

Example:
rsync -avzr -e "ssh -p 2844" - user@host:/local/dir/ /remote/dir/

WARNING:root:Command failed: find "./files/" -type f -name "*"
ERROR:root:find: WARNING: Hard link count is wrong for `./files/' (saw only st_nlink=2 but we already saw 0 subdirectories): this may be a bug in your file system driver.  Automatically turning on find's -noleaf option.  Earlier results may have failed to include directories that should have been searched.

Fix:

        cmds.append('find \"{}\" -noleaf -type {} -name "{}{}"'\
                    .format(start_dir, ftype,\
                            pattern, del_cmd))

Fix:

        cmds.append('find \"{}\" -type {} -name "{}{}"'\
                    .format(start_dir, ftype,\
                            pattern, del_cmd))

Hi,

I'm trying parallel_sync.
The copy looks good but i have no output to see progress.
How can i pass rsync options like --progress --update

Here my launch file:
from parallel_sync import rsync

rsync.copy('/mnt/Backup_BDD/mongodb/', '/data/mongodb/', exclude=['*.pyc'], parallelism=10, extract=False, validate=False)

Thanks

Hi
I can't use DSA key instead of RSA key.
The error is like this:
not valid RSA private key file

When using rsync.copy('/src_foo', '/dst_bar', parallelism=20, validate=True)
I get the following error.

File "/home/ubuntu/.local/lib/python3.6/site-packages/parallel_sync/rsync.py", line 241, in local_copy
if validate and len(srcs) > 0:
NameError: name 'srcs' is not defined

Allow the key to be a string or a list in rsync

        if 'key_filename' in creds:
            key = creds.key_filename
            if isinstance(key, list):
                key = key[0]
            creds.key = os.path.expanduser(key)

module disconnects when file not found, wondering if there is an option/switch I an use to prevent this and "skip" over this condition? Normal rsync does not fail on this type of condition.

⠹ RsyncingTraceback (most recent call last):
  File "./rsync_multi.py", line 65, in <module>
    main()
  File "./rsync_multi.py", line 55, in main
    rsync.copy(src, dest, exclude=[exc], parallelism=10, extract=False, validate=False)
  File "/usr/lib/python2.7/site-packages/parallel_sync/rsync.py", line 206, in copy
    extract=extract, validate=validate)
  File "/usr/lib/python2.7/site-packages/parallel_sync/rsync.py", line 225, in local_copy
    pool.map(_copyfile, paths)
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
    raise self._value
IOError: [Errno 2] No such file or directory: '/mig/path/to/file/com.oracle.tuxedo.tjatmi.jar'

Hello !
Is there a schedule for the migration to Python 3 ?
Thx ;)

The issue is if i give a target path in the wget function like below :

urls = [ "url1/filename1" , "url2/filename2" , "url3/filename3" ]
wget.download('/home/neo/', urls, filenames=[ "filename1" , "filename2" , "filename3" ])

I get the following error :

Exception: The following command failed: cd "./home/neo/" && wget -O "./home/neo/filename2" -t 3 -T 40 -q "url2/fiename2

Note: wget is trying to change directory into the target directory and also again referring to the same target directory in the -O option, due to which the download fails.

The only way i was able to download files was by specifying my target directory as "." . No other paths work.

I get the error

paramiko.ssh_exception.PasswordRequiredException: proivate key is encrypted

How do I fix this?

Using example:

#!/usr/bin/python
from parallel_sync import rsync
rsync.upload('/mnt/disks/10.0.0.200_Z/ServerFolders/Misc/SVG_Files', '/mnt/user/documents/SVG_Files', creds=None)

When using for local folder or mount to another local folder or mount there should be no ssh options

spaces need to be escaped with a backslash for rsync or enclosed in quotes