CPU/wallclock requirements in JDL about crab2 HOT 4 CLOSED

Comment by afanfani on Thu Mar 20 07:28:03 2008

The way to get the current MaxCPU/WallClockTime values at sites is:

lcg-info --list-ce --vo cms --attrs MaxCPUTime,MaxWCTime

From that one can see:
a) there are few sites that are not publishing
at all these information (*) and they will have to be corrected.
b) the queues settings are:
- short queues are those with <=2h
- a variety of medium and long queues from 3h to 160h

The possibilities to introduce in CRAB could be:
A) Default CPUtime to slightly more than 2h so that the short queues are skipped and ask users to specify in crab.cfg the cpu_time/wallclock_time :
- reducing the default if their jobs lasts less than 2h
- increasing the default if their job lats more than 8-10h
B) Default CPUtime to more than 10h and ask the users to specify in crab.cfg the cpu_time/wallclock_time if their jobs last less.

I would probably go for A) since B) seems to much restrictive and I hope users does have jobs lasting much less than 10hours.

(*)

• CE: ce1-egee.srce.hr:2119/jobmanager-sge-prod
  • MaxCPUTime 0
  • MaxWCTime 0
• CE: gw-4.ccc.ucl.ac.uk:2119/jobmanager-lcgpbs-lcg
  • MaxCPUTime 0
  • MaxWCTime 0
• CE: i101.hpc2n.umu.se:2119/jobmanager-lcgpbs-ngrid
  • MaxCPUTime 0
  • MaxWCTime 9060
• CE: ikaros4.inp.demokritos.gr:2119/jobmanager-condor-ikaros
  • MaxCPUTime 0
  • MaxWCTime 0

from crab2.

Comment by afanfani on Thu Mar 20 15:08:22 2008

I've applied a default of slightly more than 2hours (130minutes)
to the GlueCEPolicyMaxCPUTime (since that's the more restrictive parameter wrt to GlueCEPolicyMaxWallClockTime).

The implementation it's really trivial, i.e.
replacing in SchedulerGrid.py
self.EDG_cpu_time = cfg_params.get('EDG.max_cpu_time',None)
with
self.EDG_cpu_time = cfg_params.get('EDG.max_cpu_time', '130')

I've applied it both to CRAB_2_1_1_pre2 branch and the HEAD .

Now what is missing is to have a clear FAQ for the users
notifying the default change and asking them as good practice to
change the default if they know their job will last less than 2 hours or more than 8-10hours.

from crab2.

Comment by afanfani on Fri Mar 21 07:11:47 2008

I forgot to mention that GGUS ticket for sites not publishing
the CPU time and Wallcklock time have been opened:

https://gus.fzk.de/pages/ticket_details.php?ticket=34371
https://gus.fzk.de/pages/ticket_details.php?ticket=34372
https://gus.fzk.de/pages/ticket_details.php?ticket=34373

from crab2.

Closed by spiga on Wed Apr 2 11:37:12 2008

from crab2.