Giter Club home page Giter Club logo

automated-large-org-repo-discovery's Introduction

Export All Organization Repositories

This script is designed to export all repositories from a specified GitHub organization using REST API and pagination.

It was inspired by the GitHub Community discussion "How To Use Pagination With GitHub's API #69826".

Usage

Run the script in your terminal and when prompted, enter the name of the organization you wish to query::

bash run_discovery.sh
Please enter the organization name:
My-Super-Cool-ORG

Requirements

  • Ensure you have a GitHub API token set in your environment:
export TOKEN=ghp_**********

Visit GitHub's documentation on Rate Limits to choose the best authentication method for your use case.

  • The script requires the following minimum scopes: repo, read:org.

Important Notes

  • The delay between API calls is hardcoded to 6 seconds to comply with rate limits.
  • Page progress is printed to the screen during discovery.
  • This script makes use of the jq command-line JSON processor to parse API responses. Ensure it is installed on your system.

Script Details

The script performs the following steps:

  1. Checks if the GitHub API token is set in the TOKEN environment variable.
  2. Prompts the user to enter the organization's name.
  3. Constructs the API URL for querying the organization's repositories.
  4. Iterates through all pages of the API response, adding repository names to an array.
  5. Checks for the presence of a "next page" in the API response to continue pagination.
  6. Prints the number of discovered repositories and location of output for use in next scripts.

Example Output

$ bash run_discovery.sh 
Please enter the organization name: 
My-Super-Cool-ORG

Discovering Repo Listing Standby: 

link: <https://api.github.com/organizations/123456789/repos?per_page=100&page=2>; rel="next", <https://api.github.com/organizations/123456789/repos?per_page=100&page=2>; rel="last"

Discovering Repo Listing Standby: 

link: <https://api.github.com/organizations/123456789/repos?per_page=100&page=1>; rel="prev", <https://api.github.com/organizations/123456789/repos?per_page=100&page=1>; rel="first"

Discovered Repository Count: 142
Repo Listing located in: /tmp/2024-01-20_10-15_discovered_repositories.tmp

License

This script is released under MIT License.

automated-large-org-repo-discovery's People

Contributors

appatalks avatar

Stargazers

 avatar  avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.