Giter Club home page Giter Club logo

Comments (7)

pimguilherme avatar pimguilherme commented on June 25, 2024

this functionality is affected as well (it is not able to list repositories)

image

from incubator-devlake.

klesh avatar klesh commented on June 25, 2024

Hi there,

Thanks for reaching out! Let's address the two issues you mentioned:

  1. Expired Token:

Unfortunately, due to GitHub's design, tokens cannot be refreshed. The best course of action is to regenerate a new token with the appropriate permissions for your needs.

  1. OOM (Out-of-Memory) Error:

To help diagnose the OOM error, could you please provide some additional information:

Crash Logs: Were there any logs indicating the crash, such as "panic" messages, around the time of the error?
Container Resources: How much memory and CPU have you allocated to the container running the process?
Pipeline Context: Did the OOM occur while running a pipeline? If so, were there any specific workflows or tasks in progress during the breakdown?

from incubator-devlake.

pimguilherme avatar pimguilherme commented on June 25, 2024

Hi @klesh, thanks for reaching back :D

I think we can focus solely on problem 1 on this issue if that's okay!

Unfortunately, due to GitHub's design, tokens cannot be refreshed. The best course of action is to regenerate a new token with the appropriate permissions for your needs.

We are using a github app, so I think the tokens should be re-generated with the app's secret key, which does not expire, right? These tokens only live for 1 hour if I am not mistaken.. so we just wanted to make sure they are being refreshed.. from what it looks, we might have caught a situation where it's not 😥

thanks!

from incubator-devlake.

klesh avatar klesh commented on June 25, 2024

@pimguilherme
I apologize for the misunderstanding. I wasn't aware of the GitHub App authorization process and initially thought you were referring to a Personal Access Token (PAT).

I've reviewed the code, and it appears the access token is regenerated at the start of each collection. This logic hasn't changed since version 0.17.

You mentioned the token expiring after an hour, which could be the issue. It's possible that previously collections completed within that timeframe. However, with growing data volumes, that might no longer be the case.

from incubator-devlake.

pimguilherme avatar pimguilherme commented on June 25, 2024

Hi @klesh, thanks for the feedback! I apologize for the delay

We do have issues with large repositories that fail because they need to access github after the token has already expired :(

And I am not sure this is the same mechanics, but we also have problems loading data from the Github connection proxy (ie https://devlake/api/plugins/github/connections/3/proxy/rest/repos/my-org/my-repo), how does that token get refreshed?

Thanks again!!

from incubator-devlake.

klesh avatar klesh commented on June 25, 2024

@pimguilherme Yup, you are correct, the token for Github App should be refreshed when expires automatically.

The proxy endpoint is the perfect proof of the problem because the token is requested once when ApiClient gets instantiated and the proxy helper caches it so the token expires one hour later.

It might take a while to fix.

from incubator-devlake.

klesh avatar klesh commented on June 25, 2024

Do you have the status code of the Bad credentials response?

from incubator-devlake.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.