githubexporter / github-exporter Goto Github PK

View Code? Open in Web Editor NEW

390.0 11.0 113.0 2.09 MB

:octocat: Prometheus exporter for github metrics

License: MIT License

Go 95.86% Dockerfile 1.76% Makefile 0.36% Shell 2.02%

prometheus-exporter metrics prometheus exporter monitoring github

github-exporter's People

Contributors

Stargazers

Watchers

Forkers

hairyhenderson surabhikarthi pashupathi mikequentel linlynn lukecav shahriarb centro1993 sinhaashish seanmcguigan seancrwhite spencerkight wisely147 nkinkade kanishkn13 ayushin78 tkuhrt nodox etsangsplk ezc cweagans spawn1978 rodrigozrusso kkirsche msiuts jlegrone grofers oconnormi cyberswat alexellis enmobile kpachhai switchboardop gopherofawesomeness guusvw x-wf prabhakarms89 pberthonneau steinbrueckri asotirov0 iamrare metabol lodoyun gorkem acourdavault polohex dsmith73 enigmae chambo-e jiyeonkkk nhardt dominicmarino phoban01 adamslack rolaya gsanchezgavier zwb-github ercpereda contractshark avanier devopsmakers fionaliao jayadeepnagaraja j6s abhay-krishna rheehot ndadev dgeorgievski sokoloffa thefreshuk fernandrone rgeyer andrea-mariadb-2 grafana mattmattox supporttools yotpoltd checksum0 soloradish benri-io aly76 yuriolisa rkildea1 filecoin-project kyrdteam golang108 chartleyit zcharym paxosglobal shahondin1624 galbirk sridevipmese-git srideviorg1 100daysofkubernetes cloud-native-security marctc marcialrs upenn-githealth rhopping73 no-acl

github-exporter's Issues

docker build fails

docker version 17.06.0-ce

lynn@lynn-ThinkCentre-M92p:~/github-exporter$ sudo docker build -t github-exporter:1.0 .
Sending build context to Docker daemon 114.7kB
Step 1/13 : FROM golang:1.8.0-alpine as build
---> bcb935bbf1da
Step 2/13 : LABEL maintainer "Infinity Works"
---> Using cache
---> 3a629a30802d
Step 3/13 : RUN apk --update add ca-certificates && apk --update add --virtual build-deps git
---> Running in d054829967c2
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.5/main: temporary error (try again later)
WARNING: Ignoring APKINDEX.c51f8f92.tar.gz: No such file or directory
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.5/community: temporary error (try again later)
WARNING: Ignoring APKINDEX.d09172fd.tar.gz: No such file or directory
OK: 5 MiB in 12 packages
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.5/main: temporary error (try again later)
WARNING: Ignoring APKINDEX.c51f8f92.tar.gz: No such file or directory
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.5/community: temporary error (try again later)
WARNING: Ignoring APKINDEX.d09172fd.tar.gz: No such file or directory
ERROR: unsatisfiable constraints:
build-deps-0:
masked in: cache
satisfies: world[build-deps]
git (missing):
required by:
The command '/bin/sh -c apk --update add ca-certificates && apk --update add --virtual build-deps git' returned a non-zero code: 2

Latest tag seems borked with authentication

With latest:

time="2021-08-03T15:05:59Z" level=info msg="Fetching https://api.github.com/orgs/repos?per_page=100 \n"
time="2021-08-03T15:06:00Z" level=error msg="Error scraping API, Error: Error: Received 404 status from Github API, ensure the repsository URL is correct. If it's a privare repository, also check the oauth token is correct"
time="2021-08-03T15:06:18Z" level=info msg="Fetching https://api.github.com/orgs/repos?per_page=100 \n"
time="2021-08-03T15:06:18Z" level=error msg="Error scraping API, Error: Error: Received 404 status from Github API, ensure the repsository URL is correct. If it's a privare repository, also check the oauth token is correct"

instead with 2.0.5-alpha is working fine.

I've set both ORGS and GITHUB_TOKEN

Feature request - list open pull requests

Hi everybody,
thanks for the great work! Do you think it is possible to also track the amount of open pull requests?

KeyError subscribers_count

Hi,

it seems that this PR introduced a bug cf the trace below it's maybe not always returned.

2017-5-2 18:11:59Traceback (most recent call last):
2017-5-2 18:11:59  File "github_exporter.py", line 131, in <module>
2017-5-2 18:11:59    REGISTRY.register(GitHubCollector())
2017-5-2 18:11:59  File "/usr/local/lib/python3.5/site-packages/prometheus_client/core.py", line 50, in register
2017-5-2 18:11:59    names = self._get_names(collector)
2017-5-2 18:11:59  File "/usr/local/lib/python3.5/site-packages/prometheus_client/core.py", line 86, in _get_names
2017-5-2 18:11:59    for metric in desc_func():
2017-5-2 18:11:59  File "github_exporter.py", line 43, in collect
2017-5-2 18:11:59    self._collect_org_metrics(gauges, metrics)
2017-5-2 18:11:59  File "github_exporter.py", line 59, in _collect_org_metrics
2017-5-2 18:11:59    self._add_metrics(gauges, metrics, repo)
2017-5-2 18:11:59  File "github_exporter.py", line 119, in _add_metrics
2017-5-2 18:11:59    gauges[metric].add_metric([response_json['name'], response_json['owner']['login'], str(response_json['private']).lower()], value=response_json[field])
2017-5-2 18:11:59KeyError: 'subscribers_count'```

Feature Request - Specify Github Org and have repo's populate automagically.

You should be able to specify individual repo's, as well as organisations. Would ensure that repositories are added automatically without having to update the monitoring.

Image Vulnerabilities

I am hoping to understand the team's timeline on a new official release that will resolve the following CVEs listed for the current version (1.1.0) of github-exporter:

▶ docker scout cves githubexporter/github-exporter:1.1.0
    i New version 1.8.0 available (installed version is 1.5.0) at https://github.com/docker/scout-cli
    ✓ SBOM of image already cached, 35 packages indexed
    ✗ Detected 3 vulnerable packages with a total of 35 vulnerabilities


## Overview

                    │             Analyzed Image
────────────────────┼─────────────────────────────────────────
  Target            │  githubexporter/github-exporter:1.1.0
    digest          │  77c7ebf3bc78
    platform        │ linux/amd64
    vulnerabilities │    2C    14H    11M     0L     8?
    size            │ 11 MB
    packages        │ 35


## Packages and Vulnerabilities

   2C    13H     7M     0L     6?  stdlib 1.19.5
pkg:golang/[email protected]

    ✗ CRITICAL CVE-2023-24540
      https://scout.docker.com/v/CVE-2023-24540
      Affected range : <1.19.9
      Fixed version  : 1.19.9

    ✗ CRITICAL CVE-2023-24538
      https://scout.docker.com/v/CVE-2023-24538
      Affected range : <1.19.8
      Fixed version  : 1.19.8

    ✗ HIGH CVE-2023-29403
      https://scout.docker.com/v/CVE-2023-29403
      Affected range : <1.19.10
      Fixed version  : 1.19.10

    ✗ HIGH CVE-2023-45287
      https://scout.docker.com/v/CVE-2023-45287
      Affected range : <1.20.0
      Fixed version  : 1.20.0

    ✗ HIGH CVE-2023-45283
      https://scout.docker.com/v/CVE-2023-45283
      Affected range : <1.20.11
      Fixed version  : 1.20.11

    ✗ HIGH CVE-2023-39325
      https://scout.docker.com/v/CVE-2023-39325
      Affected range : <1.20.10
      Fixed version  : 1.20.10

    ✗ HIGH CVE-2023-24537
      https://scout.docker.com/v/CVE-2023-24537
      Affected range : <1.19.8
      Fixed version  : 1.19.8

    ✗ HIGH CVE-2023-24536
      https://scout.docker.com/v/CVE-2023-24536
      Affected range : <1.19.8
      Fixed version  : 1.19.8

    ✗ HIGH CVE-2023-24534
      https://scout.docker.com/v/CVE-2023-24534
      Affected range : <1.19.8
      Fixed version  : 1.19.8

    ✗ HIGH CVE-2022-41725
      https://scout.docker.com/v/CVE-2022-41725
      Affected range : <1.19.6
      Fixed version  : 1.19.6

    ✗ HIGH CVE-2022-41724
      https://scout.docker.com/v/CVE-2022-41724
      Affected range : <1.19.6
      Fixed version  : 1.19.6

    ✗ HIGH CVE-2022-41723
      https://scout.docker.com/v/CVE-2022-41723
      Affected range : <1.19.6
      Fixed version  : 1.19.6

    ✗ HIGH CVE-2022-41722
      https://scout.docker.com/v/CVE-2022-41722
      Affected range : <1.19.6
      Fixed version  : 1.19.6

    ✗ HIGH CVE-2023-29400
      https://scout.docker.com/v/CVE-2023-29400
      Affected range : <1.19.9
      Fixed version  : 1.19.9

    ✗ HIGH CVE-2023-24539
      https://scout.docker.com/v/CVE-2023-24539
      Affected range : <1.19.9
      Fixed version  : 1.19.9

    ✗ MEDIUM CVE-2023-29406
      https://scout.docker.com/v/CVE-2023-29406
      Affected range : <1.19.11
      Fixed version  : 1.19.11

    ✗ MEDIUM CVE-2023-39319
      https://scout.docker.com/v/CVE-2023-39319
      Affected range : <1.20.8
      Fixed version  : 1.20.8

    ✗ MEDIUM CVE-2023-39318
      https://scout.docker.com/v/CVE-2023-39318
      Affected range : <1.20.8
      Fixed version  : 1.20.8

    ✗ MEDIUM CVE-2023-45284
      https://scout.docker.com/v/CVE-2023-45284
      Affected range : <1.20.11
      Fixed version  : 1.20.11

    ✗ MEDIUM CVE-2023-39326
      https://scout.docker.com/v/CVE-2023-39326
      Affected range : <1.20.12
      Fixed version  : 1.20.12

    ✗ MEDIUM CVE-2023-29409
      https://scout.docker.com/v/CVE-2023-29409
      Affected range : <1.19.12
      Fixed version  : 1.19.12

    ✗ MEDIUM CVE-2023-24532
      https://scout.docker.com/v/CVE-2023-24532
      Affected range : <1.19.7
      Fixed version  : 1.19.7

    ✗ UNSPECIFIED CVE-2024-24785
      https://scout.docker.com/v/CVE-2024-24785
      Affected range : <1.21.8
      Fixed version  : 1.21.8

    ✗ UNSPECIFIED CVE-2024-24784
      https://scout.docker.com/v/CVE-2024-24784
      Affected range : <1.21.8
      Fixed version  : 1.21.8

    ✗ UNSPECIFIED CVE-2024-24783
      https://scout.docker.com/v/CVE-2024-24783
      Affected range : <1.21.8
      Fixed version  : 1.21.8

    ✗ UNSPECIFIED CVE-2023-45290
      https://scout.docker.com/v/CVE-2023-45290
      Affected range : <1.21.8
      Fixed version  : 1.21.8

    ✗ UNSPECIFIED CVE-2023-45289
      https://scout.docker.com/v/CVE-2023-45289
      Affected range : <1.21.8
      Fixed version  : 1.21.8

    ✗ UNSPECIFIED CVE-2023-45288
      https://scout.docker.com/v/CVE-2023-45288
      Affected range : <1.21.9
      Fixed version  : 1.21.9


   0C     1H     3M     0L     2?  openssl 3.0.10-r0
pkg:apk/alpine/[email protected]?os_name=alpine&os_version=3.17

    ✗ HIGH CVE-2023-5363
      https://scout.docker.com/v/CVE-2023-5363
      Affected range : <3.0.12-r0
      Fixed version  : 3.0.12-r0

    ✗ MEDIUM CVE-2023-6129
      https://scout.docker.com/v/CVE-2023-6129
      Affected range : <3.0.12-r2
      Fixed version  : 3.0.12-r2

    ✗ MEDIUM CVE-2024-0727
      https://scout.docker.com/v/CVE-2024-0727
      Affected range : <3.0.12-r4
      Fixed version  : 3.0.12-r4

    ✗ MEDIUM CVE-2023-5678
      https://scout.docker.com/v/CVE-2023-5678
      Affected range : <3.0.12-r1
      Fixed version  : 3.0.12-r1

    ✗ UNSPECIFIED CVE-2024-2511
      https://scout.docker.com/v/CVE-2024-2511
      Affected range : <3.0.12-r5
      Fixed version  : 3.0.12-r5

    ✗ UNSPECIFIED CVE-2023-6237
      https://scout.docker.com/v/CVE-2023-6237
      Affected range : <3.0.12-r3
      Fixed version  : 3.0.12-r3


   0C     0H     1M     0L  google.golang.org/protobuf 1.28.1
pkg:golang/google.golang.org/[email protected]

    ✗ MEDIUM CVE-2024-24786 [Loop with Unreachable Exit Condition ('Infinite Loop')]
      https://scout.docker.com/v/CVE-2024-24786
      Affected range : <1.33.0
      Fixed version  : 1.33.0



35 vulnerabilities found in 3 packages
  UNSPECIFIED  8
  LOW          0
  MEDIUM       11
  HIGH         14
  CRITICAL     2

Rate limit checks fail when there is no rate limiting

I am trying to use this exporter against a GitHub Enterprise instance. This server does not have rate limiting enabled, which appears to be causing the following error when checking for a rate limit: Error gathering Data from remote API: strconv.ParseFloat: parsing "": invalid syntax.

Set an interval or limit the github api requests

Hi,

I am running this docker setup in a production server with almost 50+ Repositories. After running for few hours our IP is getting blocked by GitHub API rate limit policy. while checking the log also we can find the request always trying to call the APIs. So is there any configuration and env parameter to set an interval between each requests and any way to limit the API requests.

Thanks.

how to export private repo metrics

LOG_LEVEL isn't working

The documentation says:

LOG_LEVEL The level of logging the exporter will run with, defaults to debug

I tried with differend values, and I can't see any difference.
I also notice that log.SetLevel(log.<some log level>) hasn't been used in any place in the code
https://github.com/infinityworks/github-exporter/search?q=SetLevel

Is there something I'm don't understanding?

Not able see metrics

I am not able to get metrices and metrices url loading continuously
below is my docker command i used to up conatiner.

i have EC2 instance created with nececcery port open
And installed docker on it and ran below command

docker run -d --restart=always -p 9171:9171 -e REPOS="prometheus-grafana, devops-xyz" -e GITHUB_TOKEN="***" githubexporter/github-exporter

then when i hit :9171/metrics its keep on loading

Add Tests

Come on...

Current Project State?

Hey all,

I'd like to get some more features added to the Grafana Agent which wraps around this exporter, however the number of outstanding PR's and issues is making me question if this project is still being actively developed?

The additional metrics I would like to see are:

Issue state (at the moment, it only returns "open" issues. We'd like to track open vs closed vs reopened over time)
Contributors, ideally with the number of PR's they've approved on the repo etc
Enhanced PR data such as checks passed, approvals required vs approvals granted, and merged vs awaiting merge

Given that the last commit to master was 248088c nearly 12 months ago, can someone let me know the status of the project so the Grafana team can work out where they want to go with this?

Thanks in advance!

Exporter is leaking github token to the logs

The exporter logs the github token to the logs.
That is an issue if you logs end in elastic search or similar tools

time="2020-10-21T15:35:10Z" level=info msg="No targets specified. Only rate limit endpoint will be scraped"
{"APIToken":"**********","APITokenEnv":"***********","APITokenFile":"","APIURL":"https://api.github.com","BaseConfig":{},"Organisations":"","Repositories":"","TargetURLs":[],"Users":"","level":"info","msg":"Starting Exporter","time":"2020-10-21T15:35:10Z"}

exporter to Enterprise GitHub

Hi Team,

I would like to know whether its possible to connect this github exporter to Enterprise GitHub. If yes, then may I know the commended steps for that?

With Regards,
Aparna Reji

Add `account_name` label to the metrics

I would like to monitor the rate limit for different github accounts in the same prometheus.
I can't find a way to do it; even if I deploy multiple instance of the exporter, the metrics will collide.
My propouse is to add the label account_name to the metrics, then metrics from differents accounts can be differentiated.
So, we'll end up with something like
github_rate_limit{account_name=ercpereda}

Can you please vendor dependencies or use go modules

Hi, I am working on packaging this project for Gentoo.

it looks like it imports many external packages. However, they are not vendored and go modules [1] are not used.

Would you please consider vendoring your dependencies or using Go modules [1]?

This would make building this exporter much easier for packagers and allow reproducible builds.

Thanks much.

[1] https://github.com/golang/go/wiki/Modules

Github Exporter Is Down

I have used portainer for executing my docker compose file which has the github exporter.

github-exporter: tty: true stdin_open: true expose: - 9171 ports: - 9171:9171 image: infinityworks/github-exporter:latest environment: - REPOS=............. - GITHUB_TOKEN=....................

But on prometheus I can see that it is down.

w

Open bugs by team member?

I didn't see this granularity in github_repo_open_issues so I am assuming you don't have it exposed yet.

But it would be nice to export some stats about open bugs per team member.
https://www.reddit.com/r/github/comments/as88u2/is_there_a_ui_or_tool_to_help_track_open_bugs/

Is it possible to support multiple repos or support the whole organization?

Spelling errors

There are a few spelling mistakes

Will open PR to fix

Typo in code

https://github.com/infinityworksltd/github-exporter/blob/master/exporter/http.go#L69

Recieved => Received

ORGS is not working

https://github.com/infinityworks/github-exporter/blob/284088c21e7d796e9b04674c1e2f00e73d9372e3/config/config.go#L52-L55

The first argument of strings.Split should be orgs instead of repos.

Number of repositories is limited to 100

I am trying to scrape an organisation with more than 100 repos. I am getting only first 100.

Watchers Count is 0

So... V3 API is somewhat strange.

watchers_count actually just shows how many people star the repo
subscribers_count used to show the watchers, seemingly it's been removed more recently from the organisation view and is being deprecated.

Ideally we could find the detail in the V3 app and fix the JSON capture on this app. Alternatively though we could move to the V4 API, this entails interaction with graphql though so would require reasonable amount of refactoring.

Please consider releasing latest master

There have been multiple commits since last release 2.0.5-alpha...master

De-Duplication between repo's and orgs

If we specify both an Org and a repo, don't bother getting the info twice, cut down on API calls.

arm64 image support

Suggestion for key metrics

Hi,

I used this exporter a year or two ago, but revisiting it now I can see a few things missing which I'd consider essential info:

# commits
# contributors

Would you be interested in adding this or accepting a PR?

Alex

Set LISTEN_PORT in the Prometheus assigned port (9171) by default

Hi currently the default port for the LISTEN_PORT variable is 8080 and should be 9171

When running the exporter outside the container LISTEN_PORT needs to be set and that could be avoided. Setting this property to 9171 by default would be the expected behavior since that is the port reserved for this exporter.

GitHub Enterprise without rate-limiting will fail

https://github.com/infinityworks/github-exporter/blob/284088c21e7d796e9b04674c1e2f00e73d9372e3/exporter/gather.go#L67-L70

The above line will return error thus causing the collection of metrics to stop when the target is a private GitHub Enterprise server with no rate-limiting.

I am not sure if this is an intended behavior as otherwise this exporter can be used to collect metrics for repositories in GitHub Enterprise even if rate-limiting is disabled.

I think with only error logs will be a better way and return nil as error can let the exporter to proceed to show already collected repository metrics. If this is intended for reducing the risk of overloading the target server, getRates should be called before gatherData so as to reduce total number of requests.

https://github.com/infinityworks/github-exporter/blob/284088c21e7d796e9b04674c1e2f00e73d9372e3/exporter/prometheus.go#L23-L36

request to support number of commits

can we also support this metric?

Bug - Ignore invalid repos/orgs, log out and continue

As above, currently the script fails poorly.

exporter: missing github_repo_release_downloads metrics

Hey! 👋 I was trying out this project and don't see github_repo_release_downloads being exported. I'm not sure if this is due to the updated API for releases / downloads or something else.

I'm running this image:

Image ID:       docker-pullable://infinityworks/github-exporter@sha256:d89edcc0cd16310984ac8afbf5a32be0067b2aa76a972fff5c07ab5800fb6dd9

Missing private repo stats?

I'm providing a GITHUB_TOKEN, but it's only showing my org's public repos - is this supported?

Thanks!

Feature Request - Add additional metrics

Current metrics are basic, add in at least:

Last commit timestamp
Issues closed (we already have open)
Downloads
Releases

Panic when token has no access to outside collaborators

{"level":"info","msg":"Initialising capture of metrics","time":"2021-09-29T07:52:59Z"}
{"level":"info","msg":"Gathering metrics for GitHub Org cloudesire","time":"2021-09-29T07:52:59Z"}
{"level":"error","msg":"Error listing members by org: GET https://api.github.com/orgs/cloudesire/outside_collaborators?per_page=100: 403 You must be an owner of this organization to list outside collaborators. []","time":"2021-09-29T07:53:05Z"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x8a1718]

goroutine 15 [running]:
github.com/infinityworks/github-exporter/exporter.(*Exporter).Collect(0xc00011e000, 0xc000053140)
	/go/src/github.com/infinityworks/github-exporter/exporter/gather.go:56 +0x1e8
github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
	/go/src/github.com/infinityworks/github-exporter/vendor/github.com/prometheus/client_golang/prometheus/registry.go:444 +0x19d
created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
	/go/src/github.com/infinityworks/github-exporter/vendor/github.com/prometheus/client_golang/prometheus/registry.go:536 +0xe36

github_repo_pull_request_count only returns result from one page

We have a repository that returns more than 1 page of result for the /pull endpoint. The data that's gathered for scraping is much smaller, and equal to the number of PRs returned by the last page of results. This line seem to be the culprit. Same thing for getReleases. Before I submit a PR and make it store results from all pages, was there a reason it was done this way?

Suggestion for exception handling

It is a great work and nice exporter.
I have one suggestion. Please consider, if it is valuable one.

I tried this with out using git token. It was working fine at that time. After several trials it didn't get the result. After further trial and error method I realized that the reason for this issue was "github_rate_limit". My limit was exceeded after several trials (without git-token it is about 60)
If we used this with git-token it will be about 5000 call. I tried the same with git-token, then it is working fine.

It will be great, If you can include this exception ("Limit exceeded") in this code.

Thanks and regards,
Jijo

change stdout print to log

Hi, I found that there is a print to the stdout here this cause some verbosity in the default behavior of the exporter.
Would be ok to change this to a log.Debug so it can be controlled with the logs verbose mode?

Making a docker hub release with a precise version.

Could you make a release on Docker hub: https://hub.docker.com/r/infinityworks/github-exporter/tags using the latest only cannot guarantee that we will have a pinned version of this software. Semver could be a nice versioning schema: https://semver.org/

In the meantime, we will use SHA256 of the docker image as a pinned version.

Incorrect `github_repo_pull_request_count` when more than one page of pull requests exists

Description

I've discovered an issue with how github_repo_pull_request_count is calculated in the exporter. It seems that the count only reflects the last page of pull requests when multiple pages are present due to GitHub's API pagination. As a result:

For repositories with fewer than 30 open pull requests, github_repo_pull_request_count accurately reports the correct number.
For repositories with more than 30 open pull requests, github_repo_pull_request_count incorrectly reports only the count from the last page. For example, if there are 36 open pull requests, it reports 6.

This discrepancy leads to inaccurate metrics for repositories with a high number of pull requests.

Steps to Reproduce

Use the exporter on a repository with fewer than 30 open pull requests and note the correct count.
Use the exporter on a repository with more than 30 open pull requests (e.g., 36).
Observe that github_repo_pull_request_count only reflects the number of pull requests on the last page (e.g., 6 instead of 36).

Expected Behavior

github_repo_pull_request_count should accurately report the total number of open pull requests in a repository, regardless of how many pages of pull requests there are.

Actual Behavior

github_repo_pull_request_count only reports the number of pull requests on the last page due to not accounting for GitHub's API pagination.

Possible Solution / Workaround

I've applied a temporary workaround that modifies the request to fetch pull requests with a higher per_page limit, aiming to reduce the impact of pagination. However, this is not a scalable solution, especially for repositories with pull requests exceeding the per_page limit. Here's the patch:

diff --git a/exporter/gather.go b/exporter/gather.go
index 7a72db2..de73e5c 100644
--- a/exporter/gather.go
+++ b/exporter/gather.go
@@ -111,7 +111,7 @@ func getReleases(e *Exporter, url string, data *[]Release) {
 func getPRs(e *Exporter, url string, data *[]Pull) {
  i := strings.Index(url, "?")
  baseURL := url[:i]
- pullsURL := baseURL + "/pulls"
+ pullsURL := baseURL + "/pulls?per_page=100"
  pullsResponse, err := asyncHTTPGets([]string{pullsURL}, e.APIToken())
  
  if err != nil {

This workaround increases the per_page parameter to 100, but a more robust solution that iterates through all pages to calculate the total count is needed.

Additional Context

Another thing is that the test for checking github_repo_pull_request_count is also not working i guess because if dont make a real http request.

Metric was collected before with the same name and label values

Hello,

I am currently using this exporter on a K8s Cluster in combination with Grafana, and it seems I cannot get any metrics out of the exporter:

An error has occurred during metrics collection:

5 error(s) occurred:
* collected metric github_repo_stars label:<name:"private" value:"true" > label:<REDACTED> gauge:<value:1 >  was collected before with the same name and label values
* collected metric github_repo_forks label:<name:"private" value:"true" > label:<REDACTED> gauge:<value:1 >  was collected before with the same name and label values
* collected metric github_repo_open_issues label:<name:"private" value:"true" > label:<REDACTED > gauge:<value:0 >  was collected before with the same name and label values
* collected metric github_repo_watchers label:<name:"private" value:"true" > label:<REDACTED> gauge:<value:0 >  was collected before with the same name and label values
* collected metric github_repo_size_kb label:<name:"private" value:"true" > label:<REDACTED > gauge:<value:2512 >  was collected before with the same name and label values

From the logs I can see that the Exporter has connected successfully to Github and all metrics can be fetched, but the /metrics endpoint only gives these error-messages. Tried it via curl, same response. No Data has been collected by Prometheus before.

Whilst researching this behavious, I found the same bug in another Project: gettyimages/chronos_exporter#2

How to scan multiple repos and return added values?

Wrong number of watchers

I quickly tested this exporter and found that the number of watchers is wrong.
Stars is ok, but gitea has 133 watchers. 😉

github_repo_watchers{private="false",repo="gitea",user="go-gitea"} 2527.0
github_repo_stars{private="false",repo="gitea",user="go-gitea"} 2527.0

Feature Request - Make use of conditional requests

https://developer.github.com/v3/#conditional-requests

We should do this, it should help us dramatically reduce the number of API calls.

Pull Requests and Issues seem incorrect in develop release

Hey there. First, thank you for your work on this exporter. As maintainer as a bunch of exporters myself, I appreciate what y'all are doing!

I've built from the latest pre-release (Development Release - 02) and detect what appears to be incorrect PR/issues data. When setting the following parameters via ENV:

USERS=DRuggeri
OPTIONAL_METRICS=pulls

I get the following (reduced) results:

# TYPE github_repo_open_issues gauge
github_repo_open_issues{archived="false",fork="false",language="",license="mit",private="false",repo="OSSClass",user="DRuggeri"} 2
github_repo_open_issues{archived="false",fork="false",language="C",license="other",private="false",repo="crystalfontz-lcd-cmd",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="false",language="Dockerfile",license="",private="false",repo="devkits",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="false",language="Go",license="apache-2.0",private="false",repo="alertmanager_gotify_bridge",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="false",language="Go",license="other",private="false",repo="bind_query_exporter",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="false",language="Go",license="other",private="false",repo="dhcpd_leases_exporter",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="false",language="Go",license="other",private="false",repo="dhcpdleasesreader",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="false",language="Go",license="other",private="false",repo="netgear_client",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="false",language="Go",license="other",private="false",repo="netgear_exporter",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="false",language="Go",license="other",private="false",repo="nut_exporter",user="DRuggeri"} 0
github_repo_open_issues{archived="false",fork="true",language="JavaScript",license="mit",private="false",repo="jsvectormap",user="DRuggeri"} 0
# HELP github_repo_pull_request_count Total number of pull requests for given repository
# TYPE github_repo_pull_request_count gauge
github_repo_pull_request_count{archived="false",fork="false",language="",license="mit",private="false",repo="OSSClass",user="DRuggeri"} 2
github_repo_pull_request_count{archived="false",fork="false",language="C",license="other",private="false",repo="crystalfontz-lcd-cmd",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="false",language="Dockerfile",license="",private="false",repo="devkits",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="false",language="Go",license="apache-2.0",private="false",repo="alertmanager_gotify_bridge",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="false",language="Go",license="other",private="false",repo="bind_query_exporter",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="false",language="Go",license="other",private="false",repo="dhcpd_leases_exporter",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="false",language="Go",license="other",private="false",repo="dhcpdleasesreader",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="false",language="Go",license="other",private="false",repo="netgear_client",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="false",language="Go",license="other",private="false",repo="netgear_exporter",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="false",language="Go",license="other",private="false",repo="nut_exporter",user="DRuggeri"} 0
github_repo_pull_request_count{archived="false",fork="true",language="JavaScript",license="mit",private="false",repo="jsvectormap",user="DRuggeri"} 0

As can be confirmed by viewing the OSSClass repository, there are no issues... but there ARE two pull requests.

This functionality also seems broken in the current stable release. That version shows NO data for pull requests... but shows open issues as the number of pulls.

Not clear message when Github token is wrong

It doesn't collect the correct labels and metrics and it ends failing with message:

... was collected before with the same name and label values

And only if you have two or more repositories.

Enable github app authentication

The exporter does not enable api authentication using github app.
So, the monitoring of github apps metrics is not enabled.

GitHub App has higher rate limit (15,000 vs 5,000 api calls).

I will PR this feature and reference this issue.