[Bug][github plugin] Unexpected "Bad Credentials" when accessing github connection proxy about incubator-devlake HOT 7 OPEN

this functionality is affected as well (it is not able to list repositories)

from incubator-devlake.

Hi there,

Thanks for reaching out! Let's address the two issues you mentioned:

1. Expired Token:

Unfortunately, due to GitHub's design, tokens cannot be refreshed. The best course of action is to regenerate a new token with the appropriate permissions for your needs.

2. OOM (Out-of-Memory) Error:

To help diagnose the OOM error, could you please provide some additional information:

Crash Logs: Were there any logs indicating the crash, such as "panic" messages, around the time of the error?
Container Resources: How much memory and CPU have you allocated to the container running the process?
Pipeline Context: Did the OOM occur while running a pipeline? If so, were there any specific workflows or tasks in progress during the breakdown?

from incubator-devlake.

Hi @klesh, thanks for reaching back :D

I think we can focus solely on problem 1 on this issue if that's okay!

Unfortunately, due to GitHub's design, tokens cannot be refreshed. The best course of action is to regenerate a new token with the appropriate permissions for your needs.

We are using a github app, so I think the tokens should be re-generated with the app's secret key, which does not expire, right? These tokens only live for 1 hour if I am not mistaken.. so we just wanted to make sure they are being refreshed.. from what it looks, we might have caught a situation where it's not 😥

thanks!

from incubator-devlake.

@pimguilherme
I apologize for the misunderstanding. I wasn't aware of the GitHub App authorization process and initially thought you were referring to a Personal Access Token (PAT).

I've reviewed the code, and it appears the access token is regenerated at the start of each collection. This logic hasn't changed since version 0.17.

You mentioned the token expiring after an hour, which could be the issue. It's possible that previously collections completed within that timeframe. However, with growing data volumes, that might no longer be the case.

from incubator-devlake.

Hi @klesh, thanks for the feedback! I apologize for the delay

We do have issues with large repositories that fail because they need to access github after the token has already expired :(

And I am not sure this is the same mechanics, but we also have problems loading data from the Github connection proxy (ie https://devlake/api/plugins/github/connections/3/proxy/rest/repos/my-org/my-repo), how does that token get refreshed?

Thanks again!!

from incubator-devlake.

@pimguilherme Yup, you are correct, the token for Github App should be refreshed when expires automatically.

The proxy endpoint is the perfect proof of the problem because the token is requested once when ApiClient gets instantiated and the proxy helper caches it so the token expires one hour later.

It might take a while to fix.

from incubator-devlake.

Do you have the status code of the Bad credentials response?

from incubator-devlake.