Comments (4)
Hah! Not looking as yet, just getting back from an Alaska trip.
I will be in Idaho Falls in a week or two if anybody wants to get a 🍺!
from civet.
Yeah, this would happen occasionally and was pretty annoying.
I was not a big fan of keeping state on the server since it would need to be stored in the database and be constantly checked. I was going to go with your assumption that we don't have malicious clients and so they can take care of themselves.
My psuedo plan was to mark most of the initial git operations as "No Fail" as that is where most of these sort of problems happen (network, keys, git server down, etc). I think the only git operation that should be a real failure is if the merge fails. The "No Fail" operations would exit with a certain code or touch a file in the filesystem (if the git exit code is wanted). The client would check this and if something marked "No Fail" failed, the client would add that job id to a list of jobs not to ask for, then tell the server to reschedule the job. This list of failed jobs would be only held in the client. The list would probably be specific to each git server. If github goes down it shouldn't affect doing gitlab jobs. If too many are added to the list then it stop doing jobs for that server. Could also have timeouts for each job (ie if it gets a "No Fail" but then starts working again, the initial "No Fail" gets expired).
I thought it would be far better to restart clients that automatically stopped than it is to go through and invalidate a huge number of jobs. This isn't obviously the most robust fix but it would probably be straight forward to implement and ease the annoyance on this relatively rare problem.
Anyway, obviously never got around to implementing this so there might be problems with this approach.
from civet.
@brianmoose - Are you looking for a job yet? 😄
We are hiring, yours would come with a decent raise
from civet.
from civet.
Related Issues (20)
- Move CIVET off the INL network HOT 5
- Show most current N events rather than N table rows HOT 2
- Provide a quick way of updating submodules HOT 2
- Dynamic priorities
- Remove spaces from filenames/directories of CIVET downloads HOT 3
- Add ability to store coverage and benchmark results in database HOT 1
- Graceful error on > 3000 file github PR
- Better algorithm for collecting information from large output HOT 1
- Different build_user for same 'repository' attribute causes all recipes to be inactive
- moose modules unit testing not occurring HOT 6
- Sort clients by inacive in client listing
- More Graphs
- Enhanced scheduling capabilities HOT 4
- Management of the DB HOT 3
- Support the concatenation of multiple PR post messages
- Add a third (or any higher) level of submodule cloning HOT 1
- Cache job get
- Catch invalid json in `GitAPI._response_json`
- Add `--cleanup-command`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from civet.