nchah / github-traffic-stats Goto Github PK
View Code? Open in Web Editor NEWGet statistics on web traffic to your GitHub repositories.
Home Page: https://pypi.org/project/github-traffic-stats/
Get statistics on web traffic to your GitHub repositories.
Home Page: https://pypi.org/project/github-traffic-stats/
@nchah I am having credential problems with the package. when I run the package as below I get the following response (i have tried manual pw entry and the concatenated version):
pipenv run gts 'sampoearas' 'ALL' 'save_csv'
Password: [Correct Password]
Bad Credentials
Code done.
I also have tried running it with my organization and get the same error
pipenv run gts 'sampoearas' 'ALL' 'save_csv' -o 'araslabs'
After some investigating, I can get the API to pass back a 200 just using my credentials on a /user page, and I can get a direct call to my organization: https://api.github.com/users/araslabs/repos?per_page=100
Any ideas what I'm doing wrong? I can generate a public key if that will help with the debugging.
Thanks for the help,
@sampoearas
I'm trying to log GitHub traffic to a dockerized PostgreSQL database with this service definition in docker-compose:
postgres:
image: postgres
environment:
- POSTGRES_DB=github-traffic-stats
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=<magic password>
volumes:
- ./volumes/postgres/data:/var/lib/postgresql/data
restart: always
I'm running github-traffic-stats in a docker container as well, with this service definition in docker-compose:
github-traffic-stats:
build: ./services/github-traffic-stats
depends_on:
- postgres
environment:
- GITHUB_USER=gieldemeulemeester
- GITHUB_TOKEN=<magic token>
- POSTGRES_HOST=postgres
- POSTGRES_PORT=5432
- POSTGRES_DB=github-traffic-stats
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=<magic password>
restart: unless-stopped
The environment variables above are used for the entrypoint of the container:
python main.py $GITHUB_USER:$GITHUB_TOKEN "ALL" "set_db" --host $POSTGRES_HOST:$POSTGRES_PORT --db-user $POSTGRES_USER:$POSTGRES_PASSWORD --db-name $POSTGRES_DB
However, github-traffic-stats throws an exception relation "repo_overview" does not exist
.
Traceback (most recent call last):
File "main.py", line 406, in main
err_msg = repos_json.json().get('message')
AttributeError: 'list' object has no attribute 'get'During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 467, in
main()
File "main.py", line 436, in main
store_db(db_config, repo, traffic_response, 'views')
File "main.py", line 258, in store_db
__repo_overview_insert(cur, repo, response_type, json_response)
File "main.py", line 273, in __repo_overview_insert
cur.execute(check_count % (repo, response_type))
psycopg2.ProgrammingError: relation "repo_overview" does not exist
LINE 1: SELECT COUNT(*) FROM repo_overview WHERE create_timestamp=DA...
Do I need to initialize the database in some way (create tables, ...)? What am I doing wrong?
Hi,
I hope gts is meant to be run also without a end-user ready to type a password at each run.
The syntax documented as username:password no longer works.
The code main() has:
pw = getpass.getpass('Password:')
I suggest to replace it with:
if sys.stdin.isatty():
pw = getpass.getpass('Password:')
else:
print 'Password:'
pw = sys.stdin.readline().rstrip()
This way it'll be possible to use:
{ sleep 1; cat secret.nchah ; } | gts 'nchah' 'github-traffic-stats' 'save_csv'
Otherwise I'm open to any other suggestion to avoid a user typing the password at each request.
Thanks in advance!
https://developer.github.com/changes/2020-02-14-deprecating-password-auth/
If the token auth can be added, it'd be appreciated.
: D
Hi ,
I have only traffic permission to one of the repisitory but dont have the repository permission, in that case how can i get the traffic-stats for that repository. could you pleae help?
Thanks for your help in advance
BRs, Gowrisankar
I tried reading stats for an org with many repos. The output stops after a chunk of repos. I haven't looked deeper into it. Is this a well-known limit, a bug in the traffic API or with the Python code?
It would be awesome if clones and referrers would be tracked as well, as specified in the official api.
I've tried copy-pasting the traffic
-sections and modifying them to also track clones, but my understanding of json and the involved datastructures in the py-script seem to be too limited.
In the graphs/traffic section there is also a fourth graph with "Popular content" data.
How about adding it in github-traffic-stats?
I am trying to get traffic info for a repository I manage, and decided to test the code on one of my own repos first. I am trying to run via:
gts dmarshall_bing dad_jokes
I get the following message:
(base) C:\Users\dpm42>gts dmarshall-bing dad_jokes
Password:
Must have push access to repository
Code done.
This is regardless of whether I enter the correct or incorrect password for my account. Is there something wrong with my syntax? I looked into the code and the server is returning a 403 error from the request call.
Thanks!
This is a great code and I have an idea to use it for other purposes, but I do not see a LICENSE file.
When setting up a personal token for Github access there's a large set of permissions that we could set.
For this script to work, which are the minimal (strictly necessary) permissions we need to provide to the access token?
Is there a difference if a repo is personal or part of an organization?
Here's what happens when I try to run it:
โ ~ gts 'tom-doerr' commonroad_batch_processing save_csv
fatal: not a git repository (or any of the parent directories): .git
Is it necessary to run it inside the corresponding git repository?
I get the following error when I run gts 'swaminator' 'nikhil' 'save_csv'
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/swaminator/nikhil/traffic/views (Caused by SSLError(SSLError(1, u'[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)'),))
Any idea why?
By default all files are written the local path. It would be nice for an option to specify an output directory. This will also make it easier to work with the merge-csv.sh
script.
I can submit a PR for a fix.
Hi,
Is there a way to not display the repo that have 0 view/clone?
Thanks in advance,
export to a database, e.g. sql would be awesome.
someone has done something similar already, but in MS Visual Studio: https://github.com/Microsoft/GitHubTelemetryParsor
Hello! Only a little detail, in the Readme file the link to psycopg2 isn't working.
Regards
I already installed requests package.
But, for example, when I executed python github-traffic-stats.py '29988122' 'ALL' 'save_csv' and provided password, I got Bad credentials response.
On the repo traffic page, I can view them without any issue.
Does this mean that I need to give permission to github api or what?
Thank you for your project!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.