AsyncStageout should be a component about crabserver HOT 8 CLOSED

riahi: The component is written in WMAgent style and ready to be commited. I expect is fine to check the code in cvs. That is fine? Any constraints about the agent naming convention? The following is the workflow:

1. FileTransferPoller queries the couchDB docs to get needed information
2. It loops on users to call the ftscpWrapper for them. It gives to ftsWrapper processes (working in parallel): mapping of pfn sources and destinations, userProxy, destination site, list of destination pfn
3. Each ftscpWapper process cleans the storage space removing the destination pfn got
4. create ftscp copy jobs splitting by source the mapping of pfn sources and destination in files (since different sources should be in different copy jobs to be transferred using ftscp command)
5. Call ftscp command for the copy job files created
6. remove the pfn for succeeded transfers

The following is what I see as next steps :

• Overload the mark_good method to remove succeeded transfers from couchDB (discuss couchDB fields to use)
• Discuss from where the user proxy can be got

from crabserver.

metson: Hi,
The code should go into SVN, can we do that next week after the MC code fest? Will that hold you up on other work? I'm not sure by what you mean by steps 2 and 4 - creating the input for ftscp was done via a view and list - have you changed this? I'd like to discuss that further before the code goes into version controll.

Can you open tickets up for the two issues you mention at the end of your comment?
Cheers
Simon

from crabserver.

riahi: Hi,

For the commit I'll wait your green light since you are the boss :-)

For the step 2, the input for ftscp wrapper is done via view and list: mapping of pfn sources and destinations, destination site and list of destination pfn are got from view and list.

For the step 4, due to the fact that different sources can't use the same FTS server, for e.g:

source X and destination Y uses fts server Z
source T and destination Y can't use the fts server Z

That is why the jobs should be split into different copy job files sorted by source. So, each ftscpWrapper process will create needed copy jobs file before calling ftscp command.

I will open a ticket for the two isseus.

Cheers
Hassen

from crabserver.

riahi: Patch to review and commit is enclosed to the ticket.

from crabserver.

metson: (In 8891) Default config file to fix ticket 95

From: Hassen Riahi [email protected]

from crabserver.

metson: (In 8892) Poller class to fix ticket 95

From: Hassen Riahi [email protected]

from crabserver.

metson: (In 8893) Main component class to fix ticket 95

From: Hassen Riahi [email protected]

from crabserver.

metson: (In 8894) Worker class to fix ticket 95

From: Hassen Riahi [email protected]

from crabserver.