Comments (2)
mcinquil: Seen previous experiences and actual workflow, the idea is to have a component (eg: GetOutputLite) which simply poll through a dao the bl_runningjob table for finished jobs, collecting those by task/usercrendential, and passes to the working subprocess which simply calls the BossLite (then Air) API to perform the getoutput (in case of succeeded jobs) or the postmortem (in case of aborted jobs).
So, I am building a prototype with this structure, that seems to work fine:
-> GetOutputLite
|
|-> GetOutputPoller
|
|-> OutputWorker
|-> OutputWorker
.
.
|-> OutputWorker
I think this is fast and allows to have something simple enough to not have much overhead.
from crabserver.
mcinquil: Just need to review and commit the patch attached to the ticket.
from crabserver.
Related Issues (20)
- #813: Test dev CRABClient using test2 REST instance and CMSSW_13_0_2 CMSSW release HOT 8
- checktaperecall - sometimes fails without clear cause HOT 4
- Improve PyPI images building process HOT 2
- #814: Test dev CRABClient using test2 REST instance and CMSSW_13_0_2 CMSSW release HOT 7
- RUCIO_Transfers should store also scope in filetransfersdb HOT 2
- rationalize configurations and common functions
- adapt Publisher_rucio to have rucio scope:name in tm_dbs_blockname HOT 2
- fix /opt/rucio/etc/rucio.cfg in pypi contaiiners
- [PyPI] Use local timezone for all images HOT 2
- avoid exiting container on command error HOT 4
- Change entrypoint of TW process to simple binary script
- keep tmp directory in TW only for 6 hours
- #815: Test prod CRABClient using test12 REST instance and CMSSW_13_0_2 CMSSW release HOT 7
- make text select via mouse for copy/paste work in pypi container
- stop extending TapeRecall rule when data is on disk HOT 3
- new DN for cmscrab service account
- FTS_Transfer does not handle external connection and bookkeeping properly, caused some files to be stale in the state "SUBMITTED" HOT 7
- incorrect handling of partial dataset
- Change entrypoint of Publisher process to simple binary script
- #816: Test prod CRABClient using preprod REST instance and CMSSW_13_0_2 CMSSW release HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crabserver.