Integrate GitHub authentication with tokens for all requests about slickstack HOT 5 CLOSED

Some examples in the documentation use a user/password too, not much information on using only the token by itself but hopefully it works with e.g. wget --header="Authorization: token <OAUTH-TOKEN>"

Ref: https://superuser.com/questions/735452/
Ref: https://stackoverflow.com/questions/63601394/

According to Gregor Martynus, a GitHub employee, it still works:

curl --header "Authorization: token d64761df071c2bf517ceb063b279432ed2f89c62" \
     https://api.github.com/repos/octokit/core.js/releases/latest

Ref: https://dev.to/gr2m/github-api-authentication-personal-access-tokens-53kd

from slickstack.

This hasn't been tested yet, but I've created a new bash alias function ss_wget_github for testing:

## ss_wget ##
function ss_wget {
    command wget --no-check-certificate --no-cache --no-cookies --quiet --inet4-only --tries=30 --timeout=300 --waitretry=15 -O "$@"
}

## ss_wget_github ##
function ss_wget_github {
    command wget --header "Authorization: token ${GITHUB_TOKEN}" --no-check-certificate --no-cache --no-cookies --quiet --inet4-only --tries=30 --timeout=300 --waitretry=15 -O "$@"
}

This function is now in ss-functions here:
https://github.com/littlebizzy/slickstack/blob/master/bash/ss-functions.txt

And the GITHUB_TOKEN option exists in ss-config here:
https://github.com/littlebizzy/slickstack/blob/master/bash/ss-config-sample.txt

from slickstack.

The bigger question seems to be whether this even applies to "raw" files being downloaded... and the answer to that is unclear, even from GitHub employees:

I spoke with our engineering team and learnt that there's a limit of 5000 requests per hour per IP address. Additionally, due to internal routing and caching, that 5000 figure isn't going to be exact. We may accept more but it's sometimes possible that we'll accept less too.

As was pointed out to me, if you're at risk of hitting this limit, then you're probably doing something wrong and there's a better way to obtain or even store the file.

Ref: github/docs#8031 (comment)

That GitHub Issue is now over 1+ year old, with no final confirmation from the GitHub employees. So it's possible that SlickStack has experienced timeout/denied problems in the past because of this ambiguity... however, I suspect that including tokens in our wget requests might not even achieve anything, since we request "raw" files... in which case, to make SlickStack more future proof, we might need to consider:

1. route requests via the GitHub API instead of "raw" or at least make this optional
2. decrease the amount of requests that SlickStack makes to GitHub
3. include the token key in all wget requests if it helps

Also see:
https://stackoverflow.com/questions/66522261/github-limit-on-public-repositories

from slickstack.

TLDR on this, apparently some of the excessive GitHub API calls were caused by this variable in ss-functions which we have now commented out for now:

# GITLAB_PRESTASHOP_ZIP=$(curl -s https://api.github.com/repos/PrestaShop/PrestaShop/releases/latest | grep browser_download_url | cut -d '"' -f 4)

And while it's true we could integrate an API key into wget requests, after further testing and research it doesn't seem necessary and would actually complicate the functionality of SlickStack at this point.

There was also a scare last month where GitHub was caching raw files for up to 24 hours instead of the previous short window of less than a few minutes, so this is still a long-term concern we need to keep an eye on, but for now all these concerns seem to be resolved.

I'm going to remove ss_wget_github function and GITHUB_TOKEN variable for the time being.

from slickstack.

Ref: 2b6324b

Ref: 6579d7f

from slickstack.