Tier0 and Rucio Subscriptions about t0 HOT 3 CLOSED

Two options here.

We could leave things as they are (with the custodial site setting in the Tier0 config and placement rules for each output type hard-coded in Tier0 code and then just replace the current per-dataset PhEDEx subscriptions with per-block Rucio rules. This has the advantage that setting the rules, tracking workflow completion and removing the rules (i.e. the completeTier0 data lifetime management) is handled in one place.

Second option is that the custodial assignment of primary datasets to T1 and the placement rules for different Tier0 outputs are kept in Rucio itself. That's cleaner in terms of managing these rules, but it raises some technical concerns. We would need to have Rucio automatically and reliably be able to detect what type of data it deals with. While that's easy for primary datasets, what about different PromptReco output (RECO, AOD, AlcaReco Skims, Physics Skims) ? And what about keeping PromptReco and ReReco seperate? Are parsing rules for the /A/B/C CMS dataset name enough or would the Tier0 injection need to 'tag' the data with a type maybe?

from t0.

Take a look at the metadata listed here: https://github.com/dmwm/CMSRucio/wiki/Draft-Rucio-data-management-plan#other-metadata

To keep RECO and PromptRECO separate I think we could use prod_step or phys_group. There may be other metadata fields one could use too.

So the Tier0 could set these fields which could be used to distinguish which Rucio subscriptions apply

from t0.

After consideration, it was decided not to use Rucio Subscriptions, due to the fact that Tier0 needs to be able to easily modify destination sites for every dataset. Instead, the PhEDEx procedure was adapted to Rucio (WMCore PR10006). The procedure have been tested and successfully used in production. I will close this issue.

from t0.