Comments (7)
To ensure that the Bitbucket Server data collection task in DevLake continues even if one pull request is corrupted, you can configure the Sync Policy to "Skip Failed Tasks." This setting allows the data collection process to skip over any failed tasks, preventing the entire pipeline from failing and ensuring that you do not lose all Bitbucket metrics for the repository.
Here is the relevant section from the document:
### Step 2.3 - Set the Sync Policy
There are three settings for Sync Policy:
- Data Time Range: You can select the time range of the data you wish to collect. The default is set to the past six months.
- Sync Frequency: You can choose how often you would like to sync your data in this step by selecting a sync frequency option or enter a cron code to specify your preferred schedule.
- Skip Failed Tasks: sometime a few tasks may fail in a long pipeline; you can choose to skip them to avoid spending more time in running the pipeline all over again.
Additionally, the ConvertPullRequests
function in the pr_convertor.go
file is designed to handle errors gracefully. The api.NewDataConverter
method in the ConvertPullRequests
function is set up to manage errors effectively, allowing the data collection process to continue even if some pull requests are not found or are corrupted.
By enabling the "Skip Failed Tasks" option and leveraging the error handling in the ConvertPullRequests
function, you can ensure that the data collection process continues even if some tasks fail [1][2].
To continue talking to Dosu, mention @dosu.
from incubator-devlake.
this is similar to 7258
from incubator-devlake.
@sstojak1
There is an option in project's sync config, you can have a try.
from incubator-devlake.
I believe it is fixed already, will be available in the coming rc
version which should be ready in a couple of days.
from incubator-devlake.
@d4x1 This option is already on for all our projects.
Here a single task is falling since one PR is corrupt in Bitbucket. As a result, other repository information won't be collected.
@klesh
Are you referring to 7577?
7577 is connected with PR commits and it's handling 40X error status. This error is with 500 status and it's connected with PR activities.
If it's ok, we can do it like @abeizn did for commits but handle 500 + body message - com.atlassian.bitbucket.scm.CommandFailedException
?
What do you think?
from incubator-devlake.
Ahh.. 500 errors? I am not sure, 500 represents Server Internal Errors, It might suggest that the server is corrupted or down, in this case, it is hard to say if it is appropriate to skip the PR.
It makes more sense to fix the 500 errors on the bitbucket server rather than ignoring them on the devlake end.
from incubator-devlake.
You're correct. Deciding whether to skip something based on the message content will be challenging. Resolving the ticket...
from incubator-devlake.
Related Issues (20)
- [Bug][devLake:UI] Error: Request failed with status code 502 , while login to devLake HOT 10
- [Bug][Devlake] unexpected end of JSON when adding more than 31 repositories as data scope to Azure-DevOps Connection HOT 2
- [Bug][Backend] fatal error: concurrent map writes
- [Bug][gitextractor] Error running pipeline beta5 HOT 9
- [Bug][Github] Pull Requests not being updated HOT 1
- [Bug][Sonarcube] Data too long for column 'component' at row 12 HOT 1
- [Bug][Github] Table 'pull_request_assignees' & 'pull_request_reviewers' has no user data from Github HOT 7
- Question about RDS Devlake HOT 21
- [Question][DORA] About the logic of ‘Median Lead Time for Changes‘ HOT 10
- [Feature][Azure DevOps Go] Add TFVC Repository Support in DevLake Plugin for Azure DevOps Go
- [Question]Getting file info HOT 13
- [Bug][TimeAfter] Bug title The timeAfter parameter in the DevLake sync API isn't functioning as expected. HOT 2
- [Feature][Config UI] Support favorites in project list
- [Bug][CircleCI] Regex Matches Workflows but Fails to Match Jobs HOT 9
- [Feature][Webhook] Allow cleanup for (deployment/incident) data collected by a webhook
- [Question] Including the gitextractor and customize plugin in new GHE blueprint HOT 3
- [Question]How do I modify this so that it records file names that were updated during git commits/pull requests HOT 5
- [Bug][Framework] fields are empty in table `_devlake_subtasks`
- [Bug][Grafana] Monthly Deployment SQL Filter HOT 1
- Unable to access config-ui after resetting the admin password in Grafana HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from incubator-devlake.