re-deign of submission component structure about crab2 HOT 9 CLOSED

Comment by farinafa on Tue Feb 26 09:30:54 2008

First implementation completed. Very simple debugging performed using fake messages injected in the system by hand.

Already integrates BossLite layer and API for SE abstraction.
Need some testing for the interaction with the other submission component. The submitting threads business logic needs to be validated in detail.

from crab2.

Comment by farinafa on Wed Feb 27 10:22:09 2008

The first code is in cvs. It needs coarse-grain tests and fixes.

from crab2.

Comment by farinafa on Mon Mar 10 12:34:46 2008

Large part of the submission chain for new tasks and resubmission covered.

Still some open points on the way for the basic usecase:

• Relative paths for the SE interface check (now absolute client paths are serialized)
• Error using Session objects by using the JobState APIs
• Scheduler Name: on jobs / task? where should be stored the field?
• align with the very last release of the SA direct submission code

from crab2.

Comment by farinafa on Tue Mar 11 12:47:36 2008

Some changes added to boss lite in order to solve serialization/deserialization issues of runningJobs.

The touched files is BossLiteAPI.py (26056 Bytes Mar 11 18:46)

from crab2.

Comment by farinafa on Wed Mar 12 13:16:08 2008

Solved the show stopper related to the BossLite XML deserialization of tasks on server-side.
Still some issues when summoning the actual schedulers.

The submission code is now aligned w.r.t. the SA version, also the BossLite alignment solved many issues reported earlier.

The problem with the Session interaction during the jobs registration is fixed (we need to check if it is really thread-safe, as with SafeSession it doesnt seem to work).

from crab2.

Comment by farinafa on Thu Mar 13 10:44:10 2008

First successful submissions of production jobs. Still some problems with the arguments quotes for the analysis tasks.

from crab2.

Comment by farinafa on Mon Mar 17 10:39:05 2008

Solved both the problem on the quotes for arguments and for the absolute paths in sandboxes (also the check now works properly for local SEs, soon we will move to surl IDs)

from crab2.

Comment by farinafa on Mon Mar 31 08:43:44 2008

Some final comments before closing the task.

Dashboard integration is on-going.

Task split in groups of 200 jobs at most is on the way from the creation level

Probabli we will need some sort of dynamic load balancing and cut-off features to manage tasks with >5000 jobs and other less pathological cases

from crab2.

Closed by spiga on Wed Apr 2 11:36:23 2008

from crab2.