wei / git-sync Goto Github PK

View Code? Open in Web Editor NEW

248.0 10.0 120.0 23 KB

🔃 A GitHub Action for syncing between two independent repositories using force push

Dockerfile 12.52% Shell 87.48%

github-action sync fork action hacktoberfest

git-sync's Introduction

Git Sync

A GitHub Action for syncing between two independent repositories using force push.

Features

Sync branches between two GitHub repositories
Sync branches to/from a remote repository
GitHub action can be triggered on a timer or on push
To sync with current repository, please checkout Github Repo Sync

Usage

Always make a full backup of your repo (git clone --mirror) before using this action.

GitHub Actions

# .github/workflows/git-sync.yml

on: push
jobs:
  git-sync:
    runs-on: ubuntu-latest
    steps:
      - name: git-sync
        uses: wei/git-sync@v3
        with:
          source_repo: "source_org/repository"
          source_branch: "main"
          destination_repo: "destination_org/repository"
          destination_branch: "main"
          ssh_private_key: ${{ secrets.SSH_PRIVATE_KEY }} # optional
          source_ssh_private_key: ${{ secrets.SOURCE_SSH_PRIVATE_KEY }} # optional, will override `SSH_PRIVATE_KEY`
          destination_ssh_private_key: ${{ secrets.DESTINATION_SSH_PRIVATE_KEY }} # optional, will override `SSH_PRIVATE_KEY`

Using shorthand

You can use GitHub repo shorthand like username/repository.

Using ssh

The ssh_private_key, or source_ssh_private_key and destination_ssh_private_key must be supplied if using ssh clone urls.

source_repo: "[email protected]:username/repository.git"

source_repo: "[email protected]:username/repository.git"

Using https

The ssh_private_key, source_ssh_private_key and destination_ssh_private_key can be omitted if using authenticated https urls.

source_repo: "https://username:[email protected]/username/repository.git"

Set up deploy keys

You only need to set up deploy keys if repository is private and ssh clone url is used.

Either generate different ssh keys for both source and destination repositories or use the same one for both, leave passphrase empty (note that GitHub deploy keys must be unique for each repository)

$ ssh-keygen -t rsa -b 4096 -C "[email protected]"

In GitHub, either:
- add the unique public keys (key_name.pub) to Repo Settings > Deploy keys for each repository respectively and allow write access for the destination repository
or
- add the single public key (key_name.pub) to Personal Settings > SSH keys
Add the private key(s) to Repo > Settings > Secrets for the repository containing the action (SSH_PRIVATE_KEY, or SOURCE_SSH_PRIVATE_KEY and DESTINATION_SSH_PRIVATE_KEY)

Advanced: Sync all branches

To Sync all branches from source to destination, use source_branch: "refs/remotes/source/*" and destination_branch: "refs/heads/*". But be careful, branches with the same name including master will be overwritten.

source_branch: "refs/remotes/source/*"
destination_branch: "refs/heads/*"

Advanced: Sync all tags

To Sync all tags from source to destination, use source_branch: "refs/tags/*" and destination_branch: "refs/tags/*". But be careful, tags with the same name will be overwritten.

source_branch: "refs/tags/*"
destination_branch: "refs/tags/*"

Docker

$ docker run --rm -e "SSH_PRIVATE_KEY=$(cat ~/.ssh/id_rsa)" $(docker build -q .) \
  $SOURCE_REPO $SOURCE_BRANCH $DESTINATION_REPO $DESTINATION_BRANCH

Author

Wei He [email protected]

License

MIT

git-sync's People

Contributors

Stargazers

Watchers

Forkers

wade1990 castlabs dlangheiter-tgm jim-tech rasodu kingsd041 nshen melissamcewen softodev saikatsaha9896 jdrossl cortexlogic manzke nomadfoodtruckapp hblankenship anthchirp ungless jaswooliex odentech fadenb lkurz shakti-garg cdasolutions evayo copyit homerjam frikke cdlaimin 3615yeye fairyworld connectionmaster codesnas 2pisoftware rory-z dumpmemory hellixit summer-ospp xsp1343001800 samjacoby nikola-zifra-udrafter xword liucaidong bensuperpc sloppyjuicy karst-development wm1123 erasmuswill liangjiancang orchana-astellas actionsamples cjh0613 maximumqualityfoods halleyassist tiepies csci1300-startingcomputing sainaptic annihilatorrrr tst32 vardhandevalla dimitarnestorov bluetwistnz necna zimbres yum-audio digital-detox arteapartedelarte sig-alanyan ofundefined sesukaku jintan-pki bhcleek ufahrrad rmcarias ftos-forks lemonadestar-lordsea anibmurthy meinvtuzai daizihan233 rivy leo-mansini sudarshann mcx luoisz barseghyanartur superstruct-tech 00mjk moeloli eailoo davidsdevel ausaci tiacsys gwuniversity outlier-org lr7skquo diseq valtech-sd posthog dugshnay galdanwing thinmy

git-sync's Issues

destination path 'source' already exists and is not an empty directory

When synchronizing from master to master, the following error is reported:
fatal: destination path 'source' already exists and is not an empty directory.
fatal: 'source' does not appear to be a git repository
fatal: Could not read from remote repository.

This error starts from v1.1.3

destination_repo doesn't accept `org/repository` as the value

Describe the bug
The configuration source_repo accepts org/repository as the value, but destination_repo does not.

To Reproduce
Create a config which passes org/repository as the value for destination address.

Example config

name: git-sync

on:
  # Triggers the workflow on push events but only for the main branch
  push:
    branches: [ main ]

jobs:
  git-sync:
    runs-on: ubuntu-latest

    steps:
      - name: git-sync
        uses: wei/git-sync@v3
        with:
          source_repo: "source_org/repository"
          source_branch: "main"
          destination_repo: "destination_org/repository" 
          destination_branch: "main"
          source_ssh_private_key: ${{ secrets.SOURCE_SSH_PRIVATE_KEY }}
          destination_ssh_private_key:  ${{ secrets.DESTINATION_SSH_PRIVATE_KEY }}

Tested with:

source_repo: "trivikr/git-sync-test"
destination_repo: "trivikr/private-git-sync-test"
example GitHub Action run: https://github.com/trivikr/git-sync-test/runs/1778016093?check_suite_focus=true

Example GitHub Action run output

Cloning into '/root/source'...
[email protected]:trivikr/git-sync-test.git:main
DESTINATION=trivikr/private-git-sync-test.git:main
Warning: Permanently added 'github.com,140.82.113.4' (RSA) to the list of known hosts.
Warning: Permanently added the RSA host key for IP address '140.82.112.4' to the list of known hosts.
* main                a60328d [source/main] fix: remove check for secrets.GIT_SYNC_DESTINATION_REPO
  remotes/source/HEAD -> source/main
  remotes/source/main a60328d fix: remove check for secrets.GIT_SYNC_DESTINATION_REPO
fatal: could not read Username for 'github.com': No such device or address

Output: The GitHub Action run fails with fatal: could not read Username for 'github.com': No such device or address

Issue: DESTINATION is trivikr/private-git-sync-test.git:main instead of [email protected]:trivikr/git-sync-test.git:main

Expected behavior
The git sync is successful between "source_org/repository" and "destination_org/repository" when ""destination_org/repository" is passed in configuration destination_repo

Screenshots

Screenshot

Additional context
This issue was identified while testing suggestion in fix #25 (comment)

Problems scheduling git sync

Hi @wei Thanks for the tips so far. Per your recommendation at repo-sync/github-sync#18 (comment)

I got this sync to work by creating a "taskrunner" repo that syncs from the private source repo to my private destination repo...but it works only when I push changes to the taskrunner repo...even though I have a "every 3 mins" schedule defined below in my yml file.
I saw the same issue with https://github.com/repo-sync/github-sync/ where the schedule does not run (but it does run on push). What am I doing wrong here? Could this be a bug, and does this work for you? Thanks..

# File: .github/workflows/git-sync.yml

name: Git Sync # from https://github.com/wei/git-sync
on:
  push:
  schedule:
  - cron: "*/3 * * * *"  # runs every 3 mins (for now)

jobs:
  git-sync:
    runs-on: ubuntu-latest
    steps:
    - name: git-sync
      uses: wei/git-sync@v2
      with:
        source_repo: "https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS }}@github.com/xxxxxxxxxx/yyyyyyyyyyy.git"
        source_branch: "refs/remotes/source/*"
        destination_repo: "https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS }}@github.com/connecteev/yyyyyyyyyyy.git"
        destination_branch: "refs/heads/*"

[Bug] failing validation check on GitHub repo slug

If source or destination repo is "https://github.com/shakti-garg/datahub", the whole script should fail as it is neither a repo slug nor full clone url.

Reason:
In git-sync.sh, the following condition is failing

if ! echo $SOURCE_REPO | grep '.git'

as grep is using a special regex character '.'

fatal: Could not read from remote repository.

Hi sir,

i found your nice action. But I cant get it to work for my case. I get an error if I work with the secret.

#This is a basic workflow to help you get started with Actions

name: Sync Repo

#Controls when the action will run. Triggers the workflow on push or pull request
#events but only for the master branch
on:
  push:
    branches:
    - dev
    - qa
    - prod
#A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  #This workflow contains a single job called "build"
  repo-sync:
    runs-on: ubuntu-latest
    steps:
    - name: repo-sync
      uses: wei/git-sync@v2
      env:
        SOURCE_REPO: "interdiscount/tealium"
        SOURCE_BRANCH: "refs/remotes/source/*"
        DESTINATION_REPO: "coopanalytics/tealium-lib-msp-id"
        DESTINATION_BRANCH: "refs/heads/*"
        SSH_PRIVATE_KEY: ${{ secrets.REPO_SYNC }}
      with:
        args: $SOURCE_REPO $SOURCE_BRANCH $DESTINATION_REPO $DESTINATION_BRANCH $SSH_PRIVATE_KEY

#Log of action run

2020-04-24T08:25:15.5651492Z ##[section]Starting: Request a runner to run this job
2020-04-24T08:25:16.0907380Z Requesting a hosted runner in current repository's account/organization with labels: 'ubuntu-latest', require runner match: True
2020-04-24T08:25:16.2345147Z Labels matched hosted runners has been found, waiting for one of them get assigned for this job.
2020-04-24T08:25:16.2741967Z ##[section]Finishing: Request a runner to run this job
2020-04-24T08:25:22.3945090Z Current runner version: '2.169.1'
2020-04-24T08:25:22.3966966Z ##[group]Operating System
2020-04-24T08:25:22.3967510Z Ubuntu
2020-04-24T08:25:22.3967679Z 18.04.4
2020-04-24T08:25:22.3967830Z LTS
2020-04-24T08:25:22.3967937Z ##[endgroup]
2020-04-24T08:25:22.3968093Z ##[group]Virtual Environment
2020-04-24T08:25:22.3968285Z Environment: ubuntu-18.04
2020-04-24T08:25:22.3968443Z Version: 20200406.2
2020-04-24T08:25:22.3968696Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu18/20200406.2/images/linux/Ubuntu1804-README.md
2020-04-24T08:25:22.3969078Z ##[endgroup]
2020-04-24T08:25:22.3969991Z Prepare workflow directory
2020-04-24T08:25:22.4156473Z Prepare all required actions
2020-04-24T08:25:22.4166930Z Download action repository 'wei/git-sync@v2'
2020-04-24T08:25:23.4150786Z Build container for action use: '/home/runner/work/_actions/wei/git-sync/v2/Dockerfile'.
2020-04-24T08:25:23.4208279Z ##[command]/usr/bin/docker build -t c27d31:7d33ac6c1fbc447fa667f04458006758 "/home/runner/work/_actions/wei/git-sync/v2"
2020-04-24T08:25:24.3745887Z Sending build context to Docker daemon  9.216kB
2020-04-24T08:25:24.3746078Z 
2020-04-24T08:25:24.3746856Z Step 1/7 : FROM alpine
2020-04-24T08:25:24.3747058Z latest: Pulling from library/alpine
2020-04-24T08:25:24.3747174Z cbdbe7a5bc2a: Pulling fs layer
2020-04-24T08:25:24.3922710Z cbdbe7a5bc2a: Verifying Checksum
2020-04-24T08:25:24.3922864Z cbdbe7a5bc2a: Download complete
2020-04-24T08:25:25.3738318Z cbdbe7a5bc2a: Pull complete
2020-04-24T08:25:25.3742035Z Digest: sha256:9a839e63dad54c3a6d1834e29692c8492d93f90c59c978c1ed79109ea4fb9a54
2020-04-24T08:25:25.3742795Z Status: Downloaded newer image for alpine:latest
2020-04-24T08:25:25.3743142Z  ---> f70734b6a266
2020-04-24T08:25:25.3743509Z Step 2/7 : LABEL "repository"="http://github.com/wei/git-sync"
2020-04-24T08:25:25.3743913Z  ---> Running in 7c69e0396e6e
2020-04-24T08:25:26.0139744Z Removing intermediate container 7c69e0396e6e
2020-04-24T08:25:26.0141603Z  ---> f2287f64e2f0
2020-04-24T08:25:26.0143876Z Step 3/7 : LABEL "homepage"="http://github.com/wei/git-sync"
2020-04-24T08:25:26.0324226Z  ---> Running in 2391f8da4f2f
2020-04-24T08:25:27.0078720Z Removing intermediate container 2391f8da4f2f
2020-04-24T08:25:27.0143232Z  ---> a0bd902c4822
2020-04-24T08:25:27.0143811Z Step 4/7 : LABEL "maintainer"="Wei He <[email protected]>"
2020-04-24T08:25:27.0241015Z  ---> Running in cfccf5168ebe
2020-04-24T08:25:28.0080753Z Removing intermediate container cfccf5168ebe
2020-04-24T08:25:28.0081466Z  ---> ca42eb4ad615
2020-04-24T08:25:28.0082140Z Step 5/7 : RUN apk add --no-cache git openssh-client &&   echo "StrictHostKeyChecking no" >> /etc/ssh/ssh_config
2020-04-24T08:25:28.0217245Z  ---> Running in aa19f7179d27
2020-04-24T08:25:28.3689232Z fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
2020-04-24T08:25:28.4167364Z fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
2020-04-24T08:25:28.5054524Z (1/11) Installing ca-certificates (20191127-r1)
2020-04-24T08:25:28.5199934Z (2/11) Installing nghttp2-libs (1.40.0-r0)
2020-04-24T08:25:28.5235498Z (3/11) Installing libcurl (7.67.0-r0)
2020-04-24T08:25:28.5299268Z (4/11) Installing expat (2.2.9-r1)
2020-04-24T08:25:28.5328934Z (5/11) Installing pcre2 (10.34-r1)
2020-04-24T08:25:28.5403070Z (6/11) Installing git (2.24.3-r0)
2020-04-24T08:25:28.6869096Z (7/11) Installing openssh-keygen (8.1_p1-r0)
2020-04-24T08:25:28.6916452Z (8/11) Installing ncurses-terminfo-base (6.1_p20200118-r3)
2020-04-24T08:25:28.6970066Z (9/11) Installing ncurses-libs (6.1_p20200118-r3)
2020-04-24T08:25:28.7042880Z (10/11) Installing libedit (20191211.3.1-r0)
2020-04-24T08:25:28.7095740Z (11/11) Installing openssh-client (8.1_p1-r0)
2020-04-24T08:25:28.7377793Z Executing busybox-1.31.1-r9.trigger
2020-04-24T08:25:28.7409516Z Executing ca-certificates-20191127-r1.trigger
2020-04-24T08:25:28.7851807Z OK: 26 MiB in 25 packages
2020-04-24T08:25:30.3041790Z Removing intermediate container aa19f7179d27
2020-04-24T08:25:30.3041980Z  ---> c676909e7f0a
2020-04-24T08:25:30.3042473Z Step 6/7 : ADD *.sh /
2020-04-24T08:25:31.0086625Z  ---> d349200676db
2020-04-24T08:25:31.0087282Z Step 7/7 : ENTRYPOINT ["/entrypoint.sh"]
2020-04-24T08:25:31.0259463Z  ---> Running in 8657b7b08c18
2020-04-24T08:25:32.0098965Z Removing intermediate container 8657b7b08c18
2020-04-24T08:25:32.0099525Z  ---> f57cd4870129
2020-04-24T08:25:32.0099667Z Successfully built f57cd4870129
2020-04-24T08:25:32.0138128Z Successfully tagged c27d31:7d33ac6c1fbc447fa667f04458006758
2020-04-24T08:25:32.0276289Z ##[group]Run wei/git-sync@v2
2020-04-24T08:25:32.0276563Z with:
2020-04-24T08:25:32.0276691Z   args: $SOURCE_REPO $SOURCE_BRANCH $DESTINATION_REPO $DESTINATION_BRANCH $SSH_PRIVATE_KEY
2020-04-24T08:25:32.0276856Z env:
2020-04-24T08:25:32.0276971Z   SOURCE_REPO: interdiscount/tealium
2020-04-24T08:25:32.0277045Z   SOURCE_BRANCH: refs/remotes/source/*
2020-04-24T08:25:32.0277169Z   DESTINATION_REPO: coopanalytics/tealium-lib-msp-id
2020-04-24T08:25:32.0277283Z   DESTINATION_BRANCH: refs/heads/*
2020-04-24T08:25:32.0277996Z   SSH_PRIVATE_KEY: ***

2020-04-24T08:25:32.0278131Z ##[endgroup]
2020-04-24T08:25:32.0310117Z ##[command]/usr/bin/docker run --name c27d317d33ac6c1fbc447fa667f04458006758_16168c --label c27d31 --workdir /github/workspace --rm -e SOURCE_REPO -e SOURCE_BRANCH -e DESTINATION_REPO -e DESTINATION_BRANCH -e SSH_PRIVATE_KEY -e INPUT_ARGS -e INPUT_SOURCE_REPO -e INPUT_SOURCE_BRANCH -e INPUT_DESTINATION_REPO -e INPUT_DESTINATION_BRANCH -e INPUT_SSH_PRIVATE_KEY -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/tealium/tealium":"/github/workspace" c27d31:7d33ac6c1fbc447fa667f04458006758  "" "" "" ""
2020-04-24T08:25:32.3003575Z [email protected]:.git:
2020-04-24T08:25:32.3034204Z [email protected]:.git:
2020-04-24T08:25:32.3066332Z Cloning into '/root/source'...
2020-04-24T08:25:32.3714554Z Warning: Permanently added 'github.com,140.82.114.4' (RSA) to the list of known hosts.
2020-04-24T08:25:32.4313457Z [email protected]: Permission denied (publickey).
2020-04-24T08:25:32.4313724Z fatal: Could not read from remote repository.
2020-04-24T08:25:32.4313786Z 
2020-04-24T08:25:32.4313911Z Please make sure you have the correct access rights
2020-04-24T08:25:32.4314035Z and the repository exists.
2020-04-24T08:25:32.4315069Z fatal: not a git repository (or any parent up to mount point /github)
2020-04-24T08:25:32.4317916Z Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2020-04-24T08:25:32.5834265Z Cleaning up orphan processes

Can it be used to sync github and gitlab?

how to sync GitHub and GitLab? like I want to take a backup of GitHub to GitLab.

Trying to understand variable configuration

Hi, here's a snippet of my implementation of this action:
- name: repo-sync
uses: wei/git-sync@v1
env:
SOURCE_REPO: "[email protected]:hopeonesource/hopeonesource.org.git"
SOURCE_BRANCH: ${ GITHUB_REF:11 }
DESTINATION_REPO: "codeserver.dev.[redacted]@codeserver.dev.[redacted].drush.in:2222/~/repository.git"
DESTINATION_BRANCH: ${ GITHUB_REF:11 }
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY_PANTHEON_REMOTE }}
with:
args: $SOURCE_REPO $SOURCE_BRANCH $DESTINATION_REPO $DESTINATION_BRANCH

The secret exists, however I am getting both a Permission Denied (public key) message as well as the following:

SOURCE=[email protected]:repo/repo.git:${
DESTINATION=[email protected]:GITHUB_REF:11.git:}
Warning: Permanently added 'github.com,140.82.114.4' (RSA) to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
error: src refspec ${ does not match any
error: failed to push some refs to '[email protected]:GITHUB_REF:11.git'
##[error]Docker run failed with exit code 1

What am I doing wrong? It's tough to find documentation on how env vars are supposed to be referenced.

Use GITHUB_TOKEN instead of creating ssh?

Hello,

If the source branch is the current branch, can the GITHUB_TOKEN be used instead of creating an SSH key to allow reading the source repository?

avoid to run action on destination repo

I don't want to run the action on destiny repository when the sync happens
that is possible?

Sync all branches AND all tags?

I'm trying to sync both all branches as well as all tags, but the yml doesnt accept two with clauses. Any ideas?

# File: .github/workflows/git-sync.yml

name: Git Sync # from https://github.com/wei/git-sync
on:
  push:
  schedule:
  - cron: "*/3 * * * *"  # runs every 3 mins (for now)

jobs:
  git-sync:
    runs-on: ubuntu-latest
    steps:
    - name: git-sync
      uses: wei/git-sync@v2
      with:
        source_repo: "https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS }}@github.com/xxxxxxxxxx/yyyyyyyyyyy.git"
        source_branch: "refs/remotes/source/*"
        destination_repo: "https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS }}@github.com/connecteev/yyyyyyyyyyy.git"
        destination_branch: "refs/heads/*"

      with:
        source_repo: "https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS }}@github.com/xxxxxxxxxx/yyyyyyyyyyy.git"
        source_branch: "refs/tags/*"
        destination_repo: "https://connecteev:${{ secrets.MY_PERSONAL_ACCESS_TOKEN_FROM_GITHUB_SETTINGS }}@github.com/connecteev/yyyyyyyyyyy.git"
        destination_branch: "refs/tags/*"

".git" in repo url is just a convention. It should not be used to ID a source as full URL source.

I am trying to use this action to push one of my repository to Dokku.
Dokku uses git SSH end point. But their URL don't contain '.git' in it. Example

git remote add dokku [email protected]:ruby-getting-started
git push dokku master

Try to understand - Advanced: Sync all branches

To Sync all branches from source to destination, use

source_branch: "refs/remotes/source/"
and
destination_branch: "refs/heads/"
But be careful, branches with the same name including master will be overwritten.

So In my case

SOURCE REPO:
BRANCHES X,Y,Z

DESTINATION REPO:
BRANCHES X,Y,Z

In my case a HEAD branch gets added to the destination repo - correct?

# This is a basic workflow to help you get started with Actions

name: Sync Repo

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
  push:
    branches:
    - dev
    - qa
    - prod
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  repo-sync:
    runs-on: ubuntu-latest
    steps:
    - name: repo-sync
      uses: wei/git-sync@v2
      with:
        source_repo: "https://${{ secrets.USER_NAME }}:${{ secrets.REPO_SYNC_TOKEN }}@github.com/interdiscount/tealium.git"
        source_branch: "refs/remotes/source/*"
        destination_repo: "https://${{ secrets.USER_NAME }}:${{ secrets.REPO_SYNC_TOKEN }}@github.com/coopanalytics/tealium-lib-msp-id.git"
        destination_branch: "refs/heads/*"

[Question] Is there a way to sync pull requests?

Hey. Thanks for this action.
I want to sync Github's Pull Request into Gitlab's Merge Requests.
Is there a way to make it?

Thank you!

Syncing specific files

Hello, I'd like to keep some files of my repo synced with some other specific files from another repo (that I don't necessarily own). Is that possible? Thanks

The git push output shows destination repository address even if it is provided as a GitHub secret

Is your feature request related to a problem? Please describe.
The git push output shows destination repository address even if it is provided as a GitHub secret

Tested with repo: https://github.com/trivikr/git-sync-test
Example GitHub Action test run: https://github.com/trivikr/git-sync-test/runs/1773895924?check_suite_focus=true

Steps to reproduce:

Set up public keys and private keys as per README.
Set up destination_repo in GitHub secret GIT_SYNC_DESTINATION_REPO.

Repro git-sync config

name: git-sync

on:
  # Triggers the workflow on push events but only for the main branch
  push:
    branches: [ main ]

jobs:
  git-sync:
    runs-on: ubuntu-latest

    steps:
      - name: git-sync
        uses: wei/git-sync@v3
        with:
          source_repo: "trivikr/git-sync-test"
          source_branch: "main"
          destination_repo: ${{ secrets.GIT_SYNC_DESTINATION_REPO }} 
          destination_branch: "main"
          source_ssh_private_key: ${{ secrets.SOURCE_SSH_PRIVATE_KEY }}
          destination_ssh_private_key:  ${{ secrets.DESTINATION_SSH_PRIVATE_KEY }}

GitHub Action test run output

[email protected]:trivikr/git-sync-test.git:main
DESTINATION=***:main
Cloning into '/root/source'...
Warning: Permanently added 'github.com,140.82.113.4' (RSA) to the list of known hosts.
Warning: Permanently added the RSA host key for IP address '140.82.112.4' to the list of known hosts.
* main                0ebf7d8 [source/main] chore: allow manually running git-sync workflow
  remotes/source/HEAD -> source/main
  remotes/source/main 0ebf7d8 chore: allow manually running git-sync workflow
Warning: Permanently added the RSA host key for IP address '140.82.114.3' to the list of known hosts.
To github.com:trivikr/private-git-sync-test.git
   b27fdec..0ebf7d8  main -> main

Note that private git repository name is hidden when DESTINATION is printed:

DESTINATION=***:main

However, it's shown in git output.

To github.com:trivikr/private-git-sync-test.git
   b27fdec..0ebf7d8  main -> main

Describe the solution you'd like
Hide the destination repository name in GitHub Action logs.

submodules

how can I also sync submodules?

use 'img' instead of docker

Hi Wei,

Thank you so much for developing this action, it's a very useful action 💯 . I have some problems with using docker. We are using 'img' for building docker images. My question is :

is it possible to use 'img' instead of docker
Is there any possible copy particular folder instead of the full repo

Kind Regards,
Sathish

Help neede to set up my action

Sorry for my abyssal incompetence with 'Actions', this is my code:

name: Sync to public website shriveling.github.io

# Controls when the action will run. 
on:
  push:
    branches: [ gh-pages ]
  pull_request:
    branches: [ gh-pages ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:
  
jobs:
  git-sync:
    runs-on: ubuntu-latest
    steps:
      - name: git-sync
        uses: wei/git-sync@v3
        with:
          source_repo: " theworldisnotflat/shriveling_world "
          source_branch: "gh-pages"
          destination_repo: "shriveling/shriveling.github.io"
          destination_branch: "gh-pages"
          ssh_private_key: ${{ secrets.SSH_PRIVATE_KEY_RSA }} # optional

and the resulting error after run.
I want to sync a branch between two repositories
Is ssh_key necessary?

Run wei/git-sync@v3
  with:
    source_repo:  theworldisnotflat/shriveling_world 
    source_branch: gh-pages
    destination_repo: shriveling/shriveling.github.io
    destination_branch: gh-pages
    ssh_private_key: ***
  
/usr/bin/docker run --name e4872fbb9d37d34a2f8f47885462a6ea4d_31f8ae --label 5588e4 --workdir /github/workspace --rm -e INPUT_SOURCE_REPO -e INPUT_SOURCE_BRANCH -e INPUT_DESTINATION_REPO -e INPUT_DESTINATION_BRANCH -e INPUT_SSH_PRIVATE_KEY -e INPUT_SOURCE_SSH_PRIVATE_KEY -e INPUT_DESTINATION_SSH_PRIVATE_KEY -e SSH_PRIVATE_KEY -e SOURCE_SSH_PRIVATE_KEY -e DESTINATION_SSH_PRIVATE_KEY -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/shriveling_world/shriveling_world":"/github/workspace" 5588e4:872fbb9d37d34a2f8f47885462a6ea4d  " theworldisnotflat/shriveling_world " "gh-pages" "shriveling/shriveling.github.io" "gh-pages"
[email protected]:theworldisnotflat/shriveling_world.git:gh-pages
Cloning into '/root/source'...
[email protected]:shriveling/shriveling.github.io.git:gh-pages
Warning: Permanently added 'github.com,140.82.114.4' (RSA) to the list of known hosts.
Load key "/root/.ssh/id_rsa": invalid format
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: not a git repository (or any parent up to mount point /github)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

feature: exclude files/directories from being pushed

Hi,

I suggest to add a new feature that allows us to synchronize the repository by excluding files and/or directories.

Reasons for this feature could be multiple. In my case I don't want to sync the .github folder cause it makes no sense to have it on my custom github instance that does allow execution of workflow based on public workflow.

That could looks like the following :

exclude_dir:
  - .github
exclude_file:
  - file_to_not_be_synced

Regards.

HTTP Basic auth when trying to push from github to gitlab?

Hi there.
I'm using gitlab, which is protected by HTTP Basic auth.
Is there any way to set up credentials for that?

Thank you

[Question] Why does it wipe out all my commits?

From my understanding, this worked, similarly to the GitHub Bot "Pull", except, that it did not have to be in the same fork network, and could be a private repo, but I set it up successfully, and ran it, but (Now luckily it was not my main fork, but a clone of it.) it cleared all of my commits, and made it identical to the source repo, I thought, like the Pull bot, it would make a PR that merges the new (since cloning) commits, of the source repo, if there were any...

Ignore .github/workflows folder

If the source repo doesn't include the exact same sync action workflow file, the first sync overwrites it.

Please expose a flag to disable syncing the .github/workflows directory.

Possible to sync issues/pull requests/wiki?

Hey and thanks for your work. Is it possible to sync issues/pre/wiki/...?

Use a different Docker image

Hi Wei,

First and foremost, thank you so much for developing this action because it has been useful to me and my team's activities. Just a question, would it be possible for us to define a different Docker image? Like make it configurable? Due to Docker limits in place and we are moving A LOT of repositories from 1 scm to another, we keep on hitting the Docker limit threshold. We thought of taking advantage of our Artifactory instance to bypass that threshold however I am not sure how we could define that in your action.

Hope to hear from you soon!

Regards,
Elixa

Problem using different branches

This is my current configuration:

name: Mirroring repository

on:
  push:
    branches:
      - master
      - staging

jobs:
  sync_repos:
    runs-on: ubuntu-18.04
    steps:
    - name: repo-sync
      uses: wei/[email protected]
      env:
        SOURCE_REPO: <secret_source_repo>
        DESTINATION_REPO:  <secret_destination_repo>
        SSH_PRIVATE_KEY:  <secret_private_key>
      with:
        args: $SOURCE_REPO ${GITHUB_REF##*/} $DESTINATION_REPO ${GITHUB_REF##*/}

This works perfectly when I push to master branch, however, for any other branches it always gives me the following error, It doesn't matter if I have created the new branch in the other repo or not.

error: src refspec staging does not match any error: failed to push some refs to '[email protected]:***.git'

Any thoughts?

Two-way Sync

Hi,

I was just wondering if there is a way to get the action to sync in both directions? I have a private and public version of a repository that I'm hoping to keep linked, and currently have it syncing between source: private/public and destination: public/main branches, but if there is two-way sync, I suspect this is creating an issue when the actions main.yml file is being copied from the private as the source and destination branches should now be reversed.

Thanks,

Ability to sync to non Github repository

Currently only allowed to sync to Github repo to another Github repo.
Can we allow syncing of Github repo to any other repo?
For example push a Github repo branch to Bitbucket repo branch.

Add automatic testing

This will allow features to be merged faster

Create repository 'wei/git-sync-downstream'
Create a new Github action in 'wei/git-sync' : push-to-downstream
- Automatically push branch(only the branch that gets update) to 'wei/git-sync-downstream'

Not totally sure that this will work though.

Sync folders

Hi, I would like sync some of folders to avoid overwrite .gitignore or workflow folder in the remote repo.

Is it possible?

Thanks!

fatal: 'HEAD' is not a valid branch name.

I have issues, when i sync my repos, git-sync create HEAD branch, when i pull all repos with action/checkout, i have this issue
:

  /usr/bin/git checkout --progress --force -B HEAD refs/remotes/origin/HEAD
  Error: fatal: 'HEAD' is not a valid branch name.
  Error: The process '/usr/bin/git' failed with exit code 128

Config file:
https://github.com/Bensuperpc/repo-sync/blob/main/.github/workflows/repo-sync.yml

I would like to sync the whole repository but without creating a HEAD branch

[Feature Request] Purge branches/tags in destination repository

For example, a user may want to push to GitLab simply to trigger a around of testing in GitLab CI, in which case it'd be nice if it's possible to clean up remote branches/tags as well so that users don't need to do this manually.