Cannot sync tags about git-sync HOT 11 CLOSED

What version of git-sync? The example you posted doesn't have enough arguments to actually run.

If you look at https://github.com/kubernetes/git-sync/blob/master/test_e2e.sh#L245-L258 you find an example of tag-syncing that is known to work.

from git-sync.

interesting, just tried a different repro and hit what might be this. Investigating.

from git-sync.

Hmm, we're specifically dereferencing tags with ^{}, but it seems there's some conditions under which that returns nothing, rather than the tag. Still looking

from git-sync.

Hmm, my git-fu is weak - I don't know why tags from github remote do not have the ^{} ref, or how to express "always dereference if possible" to git . I thought that was what ^{} meant.

I don't even know how to recreate the state so I can add a test against it.

from git-sync.

Apparently it is a difference between git versions. git version 2.15.3 shows me the right result. git version 2.19.1 fails. Tests run against 2.15.3.

Will have to investigate more.

from git-sync.

Hi @thockin
Thanks for looking at this, here are the additional environment variables, interestingly, the version of git I am using is only 1 minor version off from the one that tests run against. This version was built directly from release 2.0.6, though I'm not sure how to show the version. It may also be relevant to note that using a git hash directly does not show this issue, I have only seen this using tags.

git-sync version 2.0.6
/ # git --version
git version 2.15.2
/ # env | grep GIT
GIT_SYNC_REV=78fd35aa319acf7c7bcf8812b9078d0ac5ef771e
GIT_SYNC_SSH=true
GIT_SYNC_ROOT=/git
GIT_SYNC_DEST=dags
GIT_SYNC_REPO=[email protected]:

from git-sync.

I ran the test that you linked too and I get the same issue, but with more logging

/ # /git-sync --logtostderr --v=5 --rev=0.1 --root=/test2 --repo=[email protected]:--dest="link"
I1106 13:59:03.320489 591 main.go:502] setting up git SSH credentials
I1106 13:59:03.320679 591 main.go:179] starting up: ["/git-sync" "--logtostderr" "--v=5" "--rev=0.1" "--root=/test2" "--repo=[email protected]:" "--dest=link"]
I1106 13:59:03.320936 591 main.go:463] run(""): git clone --no-checkout -b master [email protected]: /test2
I1106 13:59:03.566905 591 main.go:354] cloned [email protected]:
I1106 13:59:03.566943 591 main.go:463] run("/test2"): git rev-list -n1 0.1
I1106 13:59:03.568349 591 main.go:285] syncing to 0.1 (e9f359bb7ccbfe2e912f386373c3bcf45b497fbc)
I1106 13:59:03.568435 591 main.go:463] run("/test2"): git fetch --tags origin master
I1106 13:59:03.798660 591 main.go:463] run("/test2"): git worktree add /test2/rev-e9f359bb7ccbfe2e912f386373c3bcf45b497fbc origin/master
I1106 13:59:03.805037 591 main.go:298] added worktree /test2/rev-e9f359bb7ccbfe2e912f386373c3bcf45b497fbc for origin/master
I1106 13:59:03.805342 591 main.go:463] run("/test2/rev-e9f359bb7ccbfe2e912f386373c3bcf45b497fbc"): git reset --hard e9f359bb7ccbfe2e912f386373c3bcf45b497fbc
I1106 13:59:03.808449 591 main.go:318] reset worktree /test2/rev-e9f359bb7ccbfe2e912f386373c3bcf45b497fbc to e9f359bb7ccbfe2e912f386373c3bcf45b497fbc
I1106 13:59:03.808496 591 main.go:463] run("/test2"): ln -snf rev-e9f359bb7ccbfe2e912f386373c3bcf45b497fbc tmp-link
I1106 13:59:03.809007 591 main.go:257] created symlink tmp-link -> rev-e9f359bb7ccbfe2e912f386373c3bcf45b497fbc
I1106 13:59:03.809089 591 main.go:463] run("/test2"): mv -T tmp-link link
I1106 13:59:03.809711 591 main.go:262] renamed symlink tmp-link to link
I1106 13:59:03.809737 591 main.go:463] run("/test2"): git rev-list -n1 0.1
I1106 13:59:03.810807 591 main.go:211] next sync in 0s
I1106 13:59:03.810843 591 main.go:463] run("/test2/link"): git rev-list -n1 0.1
I1106 13:59:03.812058 591 main.go:463] run("/test2/link"): git ls-remote -q origin refs/tags/0.1^{}
I1106 13:59:04.117890 591 main.go:402] local hash: e9f359bb7ccbfe2e912f386373c3bcf45b497fbc
I1106 13:59:04.117919 591 main.go:403] remote hash:
I1106 13:59:04.117934 591 main.go:405] update required
I1106 13:59:04.117953 591 main.go:285] syncing to 0.1 ()
I1106 13:59:04.117980 591 main.go:463] run("/test2"): git fetch --tags origin master
I1106 13:59:04.325506 591 main.go:463] run("/test2"): git worktree add /test2/rev- origin/master
I1106 13:59:04.331656 591 main.go:298] added worktree /test2/rev- for origin/master
I1106 13:59:04.331793 591 main.go:463] run("/test2/rev-"): git reset --hard
E1106 13:59:04.332645 591 main.go:186] error syncing repo: error running command: exit status 128: "fatal: ambiguous argument '': unknown revision or path not in the working tree.\nUse '--' to separate paths from revisions, like this:\n'git [...] -- [...]'\n"

from git-sync.

Ok, I've been poking around at this for a little while today and it seems like there may be a difference between annotated tags and non-annotated tags.

Creating a tag that is not annotated

$ git tag v5.1 f577a25a7b4e9dd7acc6b522afb15e34a6974851
$ git push --tags 
Total 0 (delta 0), reused 0 (delta 0)
To github.com:....
 * [new tag]         v5.1 -> v5.1

Then

I1106 15:01:31.707969       1 main.go:164] starting up: ["/git-sync"]
I1106 15:01:31.969009       1 main.go:377] update required
I1106 15:01:31.969041       1 main.go:270] syncing to v5.1 ()
E1106 15:01:32.339770       1 main.go:171] error syncing repo: error running command: exit status 128: "fatal: '/git/rev-' already exists\n"

Then a tag on the same commit, except annotated:

$ git tag -a v5.2 f577a25a7b4e9dd7acc6b522afb15e34a6974851
$ git push --tags 
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 164 bytes | 164.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0)
To github.com:....
 * [new tag]         v5.2 -> v5.2

And....

I1106 15:02:07.333007       1 main.go:164] starting up: ["/git-sync"]
I1106 15:02:07.707388       1 main.go:326] cloned [email protected]:...
I1106 15:02:07.708852       1 main.go:270] syncing to v5.2 (f577a25a7b4e9dd7acc6b522afb15e34a6974851)
I1106 15:02:08.038934       1 main.go:283] added worktree /git/rev-f577a25a7b4e9dd7acc6b522afb15e34a6974851 for origin/master
I1106 15:02:08.069686       1 main.go:303] reset worktree /git/rev-f577a25a7b4e9dd7acc6b522afb15e34a6974851 to f577a25a7b4e9dd7acc6b522afb15e34a6974851

Hope this helps, I suspect this may not have come up before because using annotated tags is more common generally. This would not have shown up in the test because it creates an annotated tag. When I added a test that removed the -a and -m from the tag creation, it failed.

./test_e2e.sh 
building: bin/amd64/git-sync
container: e2e/git-sync-amd64:v2.0.6-dirty
test root is /tmp/git-sync-test.23860
Initialized empty Git repository in /tmp/git-sync-test.23860/repo/.git/
testcase head-once: PASS
testcase default-sync: git-sync-9220
PASS
testcase head-sync: git-sync-9220
PASS
testcase branch-sync: git-sync-9220
PASS
testcase tag-sync: git-sync-9220
PASS
testcase tag-sync-no-annotate: FAIL:  file /tmp/git-sync-test.23860/root/link/file does not contain 'tag-sync-no-annotate 2': tag-sync-no-annotate 1
The directory /tmp/git-sync-test.23860 was not removed as it contains log files useful for debugging

And the test case

testcase "tag-sync-no-annotate"
TAG="$TESTCASE"--TAG
# First sync
echo "$TESTCASE 1" > "$REPO"/file
git -C "$REPO" commit -qam "$TESTCASE 1"
git -C "$REPO" tag -f "$TAG" >/dev/null
GIT_SYNC \
    --logtostderr \
    --v=5 \
    --wait=0.1 \
    --repo="$REPO" \
    --rev="$TAG" \
    --root="$ROOT" \
    --dest="link" > "$DIR"/log."$TESTCASE" 2>&1 &
sleep 3
assert_link_exists "$ROOT"/link
assert_file_exists "$ROOT"/link/file
assert_file_eq "$ROOT"/link/file "$TESTCASE 1"
# Add something and move the tag forward
echo "$TESTCASE 2" > "$REPO"/file
git -C "$REPO" commit -qam "$TESTCASE 2"
git -C "$REPO" tag -f "$TAG"  >/dev/null
sleep 3
assert_link_exists "$ROOT"/link
assert_file_exists "$ROOT"/link/file
assert_file_eq "$ROOT"/link/file "$TESTCASE 2"
# Move the tag backward
git -C "$REPO" reset -q --hard HEAD^
git -C "$REPO" tag -f "$TAG"  >/dev/null
sleep 3
assert_link_exists "$ROOT"/link
assert_file_exists "$ROOT"/link/file
assert_file_eq "$ROOT"/link/file "$TESTCASE 1"
# Add something after the tag
echo "$TESTCASE 3" > "$REPO"/file
git -C "$REPO" commit -qam "$TESTCASE 3"
sleep 3
assert_link_exists "$ROOT"/link
assert_file_exists "$ROOT"/link/file
assert_file_eq "$ROOT"/link/file "$TESTCASE 1"
# Wrap up
remove_sync_container
wait
pass

Thanks again for looking at this @thockin

from git-sync.

Yep, I think that is the difference -- all the tests use annotated. Working on git-magic to ls-remote and peel a tag, otherwise may just switch to rev-parse and use tag hashes. It seems to be semantically correct for what we need, just one-step removed from commit hashes.

from git-sync.

PR is opened, if you want to try it.

from git-sync.

I've just tried it and it works for me.

Without PR:

I1106 17:26:11.509726       1 main.go:164] starting up: ["/git-sync"]
I1106 17:26:11.848234       1 main.go:377] update required
I1106 17:26:11.848277       1 main.go:270] syncing to 0.2 ()
E1106 17:26:12.138937       1 main.go:171] error syncing repo: error running command: exit status 128: "fatal: '/git/rev-' already exists\n"

With PR

I1106 17:26:29.416826       1 main.go:179] starting up: ["/git-sync"]
I1106 17:26:29.693399       1 main.go:354] cloned [email protected]:...
I1106 17:26:29.697606       1 main.go:285] syncing to 0.2 (e9f359bb7ccbfe2e912f386373c3bcf45b497fbc)
I1106 17:26:29.939205       1 main.go:298] added worktree /git/rev-e9f359bb7ccbfe2e912f386373c3bcf45b497fbc for origin/master
I1106 17:26:29.944443       1 main.go:318] reset worktree /git/rev-e9f359bb7ccbfe2e912f386373c3bcf45b497fbc to e9f359bb7ccbfe2e912f386373c3bcf45b497fbc

from git-sync.