Giter Club home page Giter Club logo

mirror-to-gitea's Introduction

Automatically Mirror Github Repo To Your Gitea Server

Badges

image pulls

Description

This script automatically mirrors the repositories from a github-user or github-organization to your gitea server. Once started, it will create a mirrored repository under a given token for a gitea user, completely automatically.

Example: A github user github-user has public repositories dotfiles and zsh-config. Starting the script with a gitea token for the account gitea-user will create the following mirrored repositories:

  • github.com/github-user/dotfiles โ†’ your-gitea.url/gitea-user/dotfiles
  • github.com/github-user/zsh-config โ†’ your-gitea.url/gitea-user/zsh-config

The mirror settings are default by your gitea instance.

It is also possible to mirror private repos, which can be configred here in #parameters. When mirroring private repos, they will be created as private repos on your gitea server.

Prerequisites

  • A github user or organization with repositories
  • Configured Gitea instance up and running
  • User for Gitea with generated token (Settings -> Applications -> Generate New Token)
  • Docker or Docker Compose

Docker Run

docker run \
 -d \
 --restart always \
 -e GITHUB_USERNAME=github-user \
 -e GITEA_URL=https://your-gitea.url \
 -e GITEA_TOKEN=please-exchange-with-token \
 jaedle/mirror-to-gitea:latest

This will a spin up a docker container which will run forever, mirroring all your repositories once every hour to your gitea server.

Docker Compose

version: "3.3"
services:
    mirror-to-gitea:
        image: jaedle/mirror-to-gitea:latest
        restart: always
        environment:
          - GITHUB_USERNAME=github-user
          - GITEA_URL=https://your-gitea.url
          - GITEA_TOKEN=please-exchange-with-token
          #- GITHUB_TOKEN=please-exchange-with-token # Optional, set to mirror private repos
          #- MIRROR_PRIVATE_REPOSITORIES=true # Optional, set to mirror private repos
          # - DELAY=3600 # Optional, set to change the delay between checks (in seconds)
        container_name: mirror-to-gitea

Building from Source

Prerequisites

  • NodeJS
  • NPM

Build

npm install

If errors occur, try deleting the package-lock.json file and run npm install again.

Build Docker Image

docker build -t mirror-to-gitea .

Run With NodeJS

export GITHUB_USERNAME=github-user
export GITEA_URL=https://your-gitea.url
export GITEA_TOKEN=please-exchange-with-token
node src/index.js

Also export GITHUB_TOKEN and MIRROR_PRIVATE_REPOSITORIES if you want to mirror private repos, or DELAY if you want to change the delay between checks.

Run With Docker

In the above Docker run command, replace jaedle/mirror-to-gitea:latest with mirror-to-gitea. In your Docker Compose file, replace jaedle/mirror-to-gitea:latest with build: .. Then run docker compose build and docker compose up -d.

Parameters

Required

  • GITHUB_USERNAME: The name of your user or organization which public repos should be mirrored
  • GITEA_URL: The url of your gitea server
  • GITEA_TOKEN: The token for your gitea user (Settings -> Applications -> Generate New Token)

Optional

  • GITHUB_TOKEN: GitHub personal access token. Attention: if this is set, the token will be transmitted to your specified Gitea instance!
  • MIRROR_PRIVATE_REPOSITORIES: If set to true, your private GitHub repositories will also be mirrored to gitea. The GITHUB_TOKEN parameter must be set for this to work.
  • DELAY: How often to check for new repositories in seconds. Default is 3600 (1 hour).

Things to do

  • Refactoring
  • Think about how to test
  • Better logging
  • Add gitlab support
  • And so on..

mirror-to-gitea's People

Contributors

agustinmista avatar dependabot[bot] avatar derhuerst avatar jaedle avatar masterevarior avatar nelsondane avatar nilp0inter avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

mirror-to-gitea's Issues

Issue after initial sync

I tried this out and so far I've loved it. Yesterday it seemed to be doing fine. However today I noticed when I created a new repo, it synced the initial repo but stopped syncing after that point. I have the delay variable set to 1800 seconds,

DELAY=1800

However, after the initial commit, it hasn't pushed anymore updates to the repo from Github to Gitea.

I've attempted to restart the docker container, and even shut it off and re-launch it after verifying the docker-compose.yml was alright.

Any ideas of where to look? Hopefully it's just a small thing, because I love the idea of this app, and it's vital to the setup I have. Am I losing my mind or is it supposed to keep mirroring each time the delay expires? I swear I saw it doing multiple commits.

Edit: Something I noticed. If I go into the repo settings, I noticed that each one of the repos that were mirrored ended up having the mirror settings set. However, for the timer, it's set to 8h0m0s. Whereas my pull time in the docker yml is set to 30 minutes.

Is the pull time for docker only how often it looks for new repos? And then 8 hours to sync for each one? Or is this a bug?

Undo mirrors

Is it possible to remove all repository mirrors at once rather than needing to manually remove every repository mirror?

Feature: Ability to skip forks

It would be lovely with a feature to be able to skip mirroring repositories which are forks of other github repositories.

When forking a repository to submit a small patch or fix it doesn't really make much sense to mirror the repository to my local gitea installation, as the fork is unnecessary as soon as the fix has been merged upstream.

Therefore it would be very nice with an option to skip mirroring forks, and only mirror repositories that I have created myself.

Bad Credentials

Hey,

I have my docker-compose.yml which has my Gitea / Github credentials. However, when I review the docker logs for mirror-to-gitea, I keep getting an error about bad credentials, and then also a syntax error

/app/node_modules/@octokit/request/dist-node/index.js:86
      const error = new requestError.RequestError(toErrorMessage(data), status, {
Node.js v18.14.2
Starting to create mirrors...
/app/node_modules/@octokit/request/dist-node/index.js:86
      const error = new requestError.RequestError(toErrorMessage(data), status, {
                    ^

RequestError [HttpError]: Bad credentials
    at /app/node_modules/@octokit/request/dist-node/index.js:86:21
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.next (/app/node_modules/@octokit/plugin-paginate-rest/dist-node/index.js:112:28)
    at async getGithubRepositories (/app/src/index.js:11:39)
    at async main (/app/src/index.js:121:30) {
  status: 401,
  response: {
    url: 'https://api.github.com/users/aetherinox/repos',
    status: 401,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
      connection: 'close',
      'content-length': '80',
      'content-security-policy': "default-src 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Mon, 13 Nov 2023 22:18:12 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'GitHub.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': 'C65A:7056:C11DB9:18F414A:6552A0A3',
      'x-ratelimit-limit': '60',
      'x-ratelimit-remaining': '30',
      'x-ratelimit-reset': '1699916753',
      'x-ratelimit-resource': 'core',
      'x-ratelimit-used': '30',
      'x-xss-protection': '0'
    },
    data: {
      message: 'Bad credentials',
      documentation_url: 'https://docs.github.com/rest'
    }
  },
  request: {
    method: 'GET',
    url: 'https://api.github.com/users/aetherinox/repos',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': 'octokit-rest.js/18.12.0 octokit-core.js/3.6.0 Node.js/18.14.2 (linux; x64)',
      authorization: 'token [REDACTED]'
    },
    request: { hook: [Function: bound bound register] }
  }
}

Node.js v18.14.2

I've checked those credentials over and over, and they appear correct. I'm even using them for another service. So I'm not sure what I'm missing here.

It was running fine before, and I noticed I created a new repo on Github but it never moved over to Gitea. So I checked docker and the process kept getting restarted every 5 or so seconds, and that's what popped up in the logs.

I even created a fresh Personal Access Token just for mirror-to-gitea, and same error.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.