LoadDummyData class for AsyncStageout about crabserver HOT 7 CLOSED

riahi: Patch to review is enclosed to the ticket.

Cheers
Hassen

from crabserver.

metson: The patch seems to add apply_tfc and get_tfc_rules twice to the code, why is that? Are they needed? This should be adding dummy LFN's into the database, not creating files for transfer (that's quite a separate thing), so as is I don't think this patch should go in.

from crabserver.

riahi: apply_tfc and get_tfc_rules twice are not needed. Between I'm not seeing the interest adding only dummy LFN's into the database. To test the work done by this component, files are needed in the storage source and therefore trying to transfer them using the component. I write this class to only test the work done by this component and thus it is fine with me also if this patch will not be in. I removed duplicated methods in this class. You can just close the ticket if you are still convinced that is useless.

from crabserver.

metson: I think there's two tests we need:

1. a disconnected test that is purely a metadata operation (e.g. what LoadDummyData does).
2. an integration test that actually transfers some reasonably sized files (TransferDummyData or similar) which is what you've got in the patch.

Both tests are needed, 1. helps us work out scaling limits of the code, since we can use a dummy transfer backend that always works, or works with some stochastic profile e.g. we can use 1. to work out a theoretical limit on the system. 2. is necessary to test that we've hooked up FTS right, but it's not really of interest for scale testing, since then you're scale testing FTS not our code.

Does that make sense? If you split out your patch into a new file I'll be happy to commit it.

from crabserver.

riahi: yeah it makes sens...I'll the patch later in this ticket.

from crabserver.

riahi: Please review. The patch enclosed to the ticket.

from crabserver.

metson: (In 10776) Outcome of October code fest
Proper base class for plugins for LFNSourceDuplicator
Return plain text from the ftscp list function (helpful for manual debugging)
TransferDaemon uses call() instead of run() for the ftscp worker
Fix mark_good/failed methods and write parse_ftscp_results
Add user name in local DB + convert storage name to CMS name + manage startkey in couch query
Add since attribute to query source DBs + add phedexApi attribute in parent class to convert to CMS name in plugins
Instatiate WMFactory in the contructor + loading plugin object from cache
Add TransferDummyData class
Other things

fixes #604, #349, #338

from crabserver.