Comments (17)
Installing coreutils, gnu-sed and gnu-grep resolved this issue for me. Handling this automatically for future users would resolve some pain points.
from bitpocket.
I fixed the illegal option error by changing line 123 to mkdir -p instead of mkdir --parents. But I am still not able to push changes up. it just deletes the local copy.
from bitpocket.
Update, after some experimentation, if I copy files directly into the master folder, they get synced to the slaves, but if I copy files into the slaves, they just get deleted. It appears to only be syncing in one direction.
from bitpocket.
Hi, it seems that some of your commands must have different APIs than what I have seen on my machines. I suspect the issue is that you are not running the GNU coreutils versions of cp/mkdir/rmdir/comm/touch/cat/sleep, although I don't want to completely rule out version incompatibility with bash/rsync/ssh or (even less likely) with grep/sed/kill. All of these are used with various options and we don't have the luxury of having autoconf/automake to figure out which are which.
I do have access to a Mountain Lion machine, but it has most of the the relevant tools (including GNU coreutils) installed through macports. I'd very much like to figure this out, I hate the idea that there might be data loss resulting from this.
Your tests with mkdir certainly brought us in the right direction, but could you perhaps, for completeness sake, post the output of the following commands:
bash --version
rsync --version
ssh -V
grep --version
sed --version
kill --version
cp --version
mkdir --version
rmdir --version
comm --version
touch --version
cat --version
sleep --version
I'll then try to take a look at bitpocket behavior on mountain lion with a set of commands that match these.
from bitpocket.
bash: GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin12)
rsync: rsync version 2.6.9 protocol version 29
ssh: OpenSSH_5.9p1, OpenSSL 0.9.8r 8 Feb 2011
grep: grep (BSD grep) 2.5.1-FreeBSD
none of the other commands generate version output
I am running this command via zsh, could this cause issues?
from bitpocket.
It seems pretty clear that this is caused by BSD versions of core tools (mkdir/sed/...) not behaving in the same way as the gnu versions. mkdir --parent is one example. --version is another (so your commands do not generate version output, probably because they are BSD, not GNU).
Another example:
$ cat rsyncoutput.txt | grep "^-|^d" | /opt/local/libexec/gnubin/sed "s:^\S_\s_\S_\s_\S_\s_\S_\s_:/:"
/.
/bla
/rsyncoutput.txt
$ cat rsyncoutput.txt | grep "^-|^d" | /usr/bin/sed "s:^\S_\s_\S_\s_\S_\s_\S_\s_:/:"
/drwx------ 170 2013/04/16 20:24:56 .
/-rw------- 0 2013/04/16 20:21:49 bla
/-rw------- 23 2013/04/16 20:24:56 rsyncoutput.txt
The second version is definitely not going to work. Too bad.
from bitpocket.
SOLUTIONS:
Immediate: Current version does not work with BSD versions of coreutils, and will cause data loss. You could (on mac) install macports and use that to install GNU coreutils (making sure GNU versions come before BSD versions in $PATH). It is also possible that a previous commit, 9622eac, works better, since that did not rely as heavily on coreutils.
Short-term: Unfortunately I think the only very short-term fix that I'm likely to apply is to test for GNU versions of coreutils and stop execution if they are not present.
Medium-term: I would like to make this robust to BSD, but that will take some work. Can't promise anything about time. I would try to respond quickly to a patch or pull request, though.
from bitpocket.
Just pushed an update that prevents bitpocket from running if it seems that sed is not behaving the way it needs to behave. I think there are ways to bypass the sed commands that cause issues but that requires some testing. Kronusdark, would you be up for doing that if I push a fix? (I'd push it to https://github.com/torfason/bitpocket to verify, before pushing it to the master repo).
from bitpocket.
Sure push away
Sent from my iPhone
On Apr 17, 2013, at 8:41 PM, torfason [email protected] wrote:
Just pushed an update that prevents bitpocket from running if it seems that sed is not behaving the way it needs to behave. I think there are ways to bypass the sed commands that cause issues but that requires some testing. Kronusdark, would you be up for doing that if I push a new version?
—
Reply to this email directly or view it on GitHub.
from bitpocket.
👍
from bitpocket.
Any plans to make this BSD (and therefore Mac) friendly?
from bitpocket.
Actually this is pretty easy to solve.
Just install the gnu coreutils with homebrew. Homebrew will install everything prefixed with a g.
So if you want the GNU sed command on mac, you just have to use gsed.
I'm now in a testing phase. I don't really know if i should make a pull request or a new fork. What do you think @sickill?
from bitpocket.
I did a some work on making bitpocket work without requiring gnu tools, but going that route seemed to need more time than I had (even though that would really be the best solution, since one of the goals of bitpocket is maximum compatibility). So that's why there was no push to the repo and why this issue is still open.
But yes, installing gnu coreutils (and sed/grep - note that sed/grep are not part of gnu coreutils) is a workaround for now.
daseinhorn: What would your fix consist of? Changes that test for the existence of gsed/g* and use it instead of the standard tools if they are available? I'd be all for a pull request if that is what you are thinking.
from bitpocket.
Latest OS X (Mavericks) doesn't have GNU coreutils. I had to install coreutils
and gnu-sed
via hombrew, and run bitpocket using:
PATH="/usr/local/opt/coreutils/libexec/gnubin:/usr/local/opt/gnu-sed/libexec/gnubin:$PATH" bitpocket
Also, checking for sed only still lets me run bitpocket with an incompatible version of mkdir (for when I installed gnu-sed, but not coreutils).
from bitpocket.
This issue clearly shows that bitpocket depends on many external utilities which happen to be highly incompatible across multiple platforms. I think it would be good to implement detection of utility version and/or platform. For example bitpocket script could define functions named awk
, sed
that would call the proper (g)awk
, (g)sed
with proper args. Detection could happen as the first thing in the script and these awk
/sed
function would take actual binary names and proper switch names from variables set in the detection phase.
from bitpocket.
hi @SgtPooki
when you installed these three packages, did you install them such that e.g. the binaries were called "sed" rather than "gsed"?
from bitpocket.
The #61 added support for macOS and was tested against macOS Sierra. Anyone is free to try the new release and offer commentary.
from bitpocket.
Related Issues (20)
- Editing .bitpocket/exclude results in unexpeced deletions & data loss HOT 4
- can rsync daemon be used?
- is bitpocket still maintained ? HOT 5
- Script fails on ubuntu sh HOT 5
- export LC_ALL=C makes script not work properly with international characters HOT 4
- soft links lost on sync
- slave's symlinks deleted on sync HOT 4
- not working if shell is not bash
- server/master data not backed up before deletion HOT 2
- Space in remote path not handled HOT 5
- OSX compatibility issues - El Capitan HOT 4
- New maintainer(s) needed
- Syntax error on old master directory HOT 2
- bitsync fail on backup HOT 5
- Folder synced as empty and cannot find info in the git backups HOT 2
- Interest in fixing shellcheck warnings? HOT 1
- How does bitpocket distinguish between delete/create? HOT 1
- Two different configs? HOT 1
- Syncing local directories with spaces in their names HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bitpocket.