cern-ph-cmg / cmg-cmssw Goto Github PK
View Code? Open in Web Editor NEWCERN CMG Tools repository. Installation instructions available on the twiki:
Home Page: https://twiki.cern.ch/twiki/bin/viewauth/CMS/CMGToolsReleasesExperimental
CERN CMG Tools repository. Installation instructions available on the twiki:
Home Page: https://twiki.cern.ch/twiki/bin/viewauth/CMS/CMGToolsReleasesExperimental
Hi Yuta,
Could you transfer this sample.
Thanks
/ZJetsToNuNu_HT-100To200_13TeV-madgraph/RunIISpring15DR74-Asympt25ns_MCRUN2_74_V9-v1/MINIAODSIM
Ciao.
I added some instruction to check the timing of our analysis workflow
https://twiki.cern.ch/twiki/bin/viewauth/CMS/SUSYCMGfwk#Checking_your_timing
EveryOne is encouraged to post here the results of the test.
Maria
Hi @mdunser ,
could you please transfer the following dataset:
/WJetsToLNu_TuneCUETP8M1_8TeV-amcatnloFXFX-pythia8/Summer12DR53X-PU_S10_TuneCUETP8M1_START53_V19-v3/AODSIM
Cheers,
Jan K. and Markus Stoye
I am on branch heppy_80X, as recommended for Moriond 2017 analyses (https://twiki.cern.ch/twiki/bin/viewauth/CMS/CMGToolsReleasesExperimental)
When setting the option "smearJets = True" for the JetAnalyzer the code no longer runs and reports the following error (note the second to last file in the trace):
Traceback (most recent call last):
File "/afs/cern.ch/work/c/cbeiraod/Stop4Body/CMSSW_8_0_25/python/PhysicsTools/HeppyCore/framework/heppy_loop.py", line 35, in runLoopAsync
loop = runLoop( comp, outDir, copy.copy(sys.modules[configName].config), options)
File "/afs/cern.ch/work/c/cbeiraod/Stop4Body/CMSSW_8_0_25/python/PhysicsTools/HeppyCore/framework/heppy_loop.py", line 61, in runLoop
loop.loop()
File "/afs/cern.ch/work/c/cbeiraod/Stop4Body/CMSSW_8_0_25/python/PhysicsTools/HeppyCore/framework/looper.py", line 198, in loop
self.process( iEv )
File "/afs/cern.ch/work/c/cbeiraod/Stop4Body/CMSSW_8_0_25/python/PhysicsTools/HeppyCore/framework/looper.py", line 256, in process
ret = analyzer.process( self.event )
File "/afs/cern.ch/work/c/cbeiraod/Stop4Body/CMSSW_8_0_25/python/PhysicsTools/Heppy/analyzers/objects/JetAnalyzer.py", line 184, in process
self.smearJets(event, allJets)
File "/afs/cern.ch/work/c/cbeiraod/Stop4Body/CMSSW_8_0_25/python/PhysicsTools/Heppy/analyzers/objects/JetAnalyzer.py", line 468, in smearJets
gen = jet.mcJet
File "/afs/cern.ch/work/c/cbeiraod/Stop4Body/CMSSW_8_0_25/python/PhysicsTools/Heppy/physicsobjects/PhysicsObject.py", line 28, in __getattr__
return getattr(self.physObj, name)
AttributeError: 'pat::Jet' object has no attribute 'mcJet'
I have traced this problem to the fact that not all jets have a matched genJet. The cause can be easily observed by looking at the following lines of code:
self.matchJets(event, [ j for j in allJets if j.pt()>self.cfg_ana.jetPt ]) # To match only jets above chosen threshold
if getattr(self.cfg_ana, 'smearJets', False):
self.smearJets(event, allJets)
From here: JetAnalyzer.py lines 182-184
To fix, it should be fairly straightforward, either match all jets or smear only the jets that were matched. The intended behavior is not clear and choosing between the two options is not as straightforward. The two fixes, below:
self.matchJets(event, allJets)
if getattr(self.cfg_ana, 'smearJets', False):
self.smearJets(event, allJets)
self.matchJets(event, [ j for j in allJets if j.pt()>self.cfg_ana.jetPt ])
if getattr(self.cfg_ana, 'smearJets', False):
self.smearJets(event, [ j for j in allJets if j.pt()>self.cfg_ana.jetPt ])
I would be happy to implement the fix and make a pull request if the desired behavior can be made clear.
Hi Yuta,
Could you please transfer this sample for the PF paper studies?
/QCD_Pt-15to20_EMEnriched_TuneCUETP8M1_13TeV_pythia8/RunIISpring15DR74-AsymptNoPUReco_MCRUN2_74_V9A-v1/GEN-SIM-RECO
Many thanks,
Colin
Dear All
This is Mohsen and i recently find out about your framework and would like to use is for our top analysis. i went through your instruction to run it via
heppy_batch.py -o output_Directory -r /store/cmst3/user/XXXXyourusenameXXXX/test_13Jan16 run_susyXXX_cfg.py -b 'bsub -q 8nh -u pippo123 -o std_output.txt -J JobName < batchScript.sh'
but it seems it can not recognize the eos storage. could you please let me know if this is an issue or i should set something else to know the eos area?
creating root://eoscms.cern.ch//eos/cms/store/group/phys_top/test
Command (['/afs/cern.ch/project/eos/installation/pro/bin/eos.select', 'ls', '/eos/cms/store/group/phys_top/test']) failed with return code: 2
Unable to stat /eos/cms/store/group/phys_top/test; No such file or directory (errc=2) (No such file or directory)
Best
Mohsen
Hello Yuta,
could you transfer these two sample?
/QCDuubar_Pt-15to3000_TuneZ2star_Flat_13TeV_pythia6/RunIISpring15DR74-AsymptNoPU_MCRUN2_74_V9A-v1/AODSIM
/QCDuubar_Pt-15to3000_TuneZ2star_Flat_13TeV_pythia6/RunIISpring15DR74-AsymptNoPU_MCRUN2_74_V9A-v1/MINIAODSIM
thanks,
cristina
NB: I've started a new thread for the new MC campaign.
Hi,
I was just trying to run on a fastSim signal sample using the 7_4_12 branch in 7_4_12, i.e with the preprocessor etc.
The preprocessor runs fine, but then heppy itself gives this error:
Exception:
Cannot find collection with:
type = edm::TriggerResults
label = ('TriggerResults', '', 'PAT') or ('TriggerResults', '', 'RECO')
The full heppy log is here.
This seems related to the eventFlagAnayzer, which expects to have triggerResults either in PAT or in RECO format. But this is absent for FastSim and therefore causes the crash.
Is there a way to keep the event Flag branches inside the tuple, but to ignore reading them from FastSim, i.e. when they're absent? We're anyway not going to use the event flags for Signals, but it would be nice to have consistent tuples for Full and FastSim.
Thanks,
Artur
Hi @mdunser ,
We would like to request to transfer the following dataset to CERN:
/SingleMuHighPt/Run2015E-PromptReco-v1/AOD - size: 4.6TB
It is a pp data set from the 5 TeV run. This dataset belongs to the pp budget (not HIN) it however is only available at T2_US_Vanderbilt (HIN dedicated T2). Vanderbilt cannot be used currently due to the ongoing pPb run. We (TOP PAG) need the dataset to finalize the ttbar cross sections at 5 TeV.
Marta
In the python analysis framework, CMGDB queries to get the filenames appear to be cached while DAS ones don't.
This should be fixed.
Also, we probably should fetch only the datasets that we're running on.
Hi Sezen,
Could we have these three samples of rare backgrounds transferred?
/TBZToLL_4F_TuneZ2star_8TeV-madgraph-tauola/Summer12_DR53X-PU_S10_START53_V7C-v1/AODSIM
/WpWpqq_8TeV-madgraph/Summer12_DR53X-PU_S10_START53_V7A-v1/AODSIM
/WmWmqq_8TeV-madgraph/Summer12_DR53X-PU_S10_START53_V7A-v1/AODSIM
Thanks,
Giovanni
Hi
Was the HEEP electron Id [1] by chance already implemented into the cmg framework or is somebody working on the implementation at the moment?
Thanks
Myriam
[1] https://twiki.cern.ch/twiki/bin/view/CMS/HEEPElectronIdentificationRun2
Ciao,
I think that it would make sense to store only the sign of the gen weight
instead of its absolute value
https://github.com/CERN-PH-CMG/cmg-cmssw/blob/CMGTools-from-CMSSW_7_4_7/PhysicsTools/Heppy/python/analyzers/core/AutoFillTreeProducer.py#L127
since only the former is a meaningful quantity, see for instance
https://hypernews.cern.ch/HyperNews/CMS/get/generators/2679/1.html
Also the PDF weights could be normalized to the absolute value of the gen weight.
Finally, are the scale variation weights stored? They could also be useful, I am not sure I found them but maybe they are somewhere else.
What do you think?
Hello there,
it's this time of the year again where our local buffer is full and we can no longer process any requests made. Therefore, I plan on deleting a bunch of datasets very soon.
The full list is below, but let me give you a summary first:
I will be deleting all requests that are more than one year old. Those are 74X and 76X MC samples and some data samples from Run2015D (some PromptReco, some ReReco). All these samples are listed in List 1 below. This amounts to roughly 20 TB, so it will not be enough space for the coming weeks, let alone when datataking starts.
All datasets from 2016 which are PromptReco, and are not RunH. Remember that every sample before Run2016H was rerecoed (23Sep2016 rereco). None of the 23Sep2016, or 03Feb2017 samples will be deleted. If accepted, this will free up ~100 TB, so we should have some room to add some rereco and re-miniaod samples which are requested. All the datasets in List 2 are affected.
So. As always: please let me know if anyone has very strong opinions about keeping these datasets. Please do so until tomorrow, as I plan on sending the request for deletion tomorrow ~early afternoon.
Best,
-marc
/DoubleEG/Run2015D-16Dec2015-v2/MINIAOD
/DoubleEG/Run2015D-PromptReco-v4/MINIAOD
/HTMHT/Run2015D-PromptReco-v4/MINIAOD
/JetHT/Run2015D-PromptReco-v4/MINIAOD
/MET/Run2015D-PromptReco-v4/MINIAOD
/MuonEG/Run2015D-16Dec2015-v1/MINIAOD
/SingleElectron/Run2015D-16Dec2015-v1/MINIAOD
/SingleElectron/Run2015D-PromptReco-v4/MINIAOD
/SingleMuon/Run2015D-05Oct2015-v1/MINIAOD
/SingleMuon/Run2015D-16Dec2015-v1/MINIAOD
/SingleMuon/Run2015D-PromptReco-v4/MINIAOD
/Tau/Run2015D-16Dec2015-v1/MINIAOD
/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15DR76-25nsFlat10to25TSG_HCALDebug_76X_mcRun2_asymptotic_v11-v1/AODSIM
/GJets_HT-40To100_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v3/MINIAODSIM
/QCD_HT500to700_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/QCD_Pt-15to20_EMEnriched_TuneCUETP8M1_13TeV_pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/SMS-T1tttt_mGluino-1050_mLSP-50to775_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-FastAsympt25ns_74X_mcRun2_asymptotic_v2-v1/MINIAODSIM
/SMS-T1tttt_mGluino-1050to1075_mLSP-650to850_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-FastAsympt25ns_74X_mcRun2_asymptotic_v2-v1/MINIAODSIM
/SMS-T1tttt_mGluino-1225to1250_mLSP-1to1025_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-FastAsympt25ns_74X_mcRun2_asymptotic_v2-v1/MINIAODSIM
/SMS-T1tttt_mGluino-1550to1575_mLSP-500to1175_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-FastAsympt25ns_74X_mcRun2_asymptotic_v2-v1/MINIAODSIM
/SMS-T1tttt_mGluino-600to625_mLSP-250to375_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-FastAsympt25ns_74X_mcRun2_asymptotic_v2-v1/MINIAODSIM
/SMS-T5qqqqVV_mGluino-1000To1075_mLSP-1To950_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-FastAsympt25ns_74X_mcRun2_asymptotic_v2-v1/MINIAODSIM
/SMS-T5qqqqVV_mGluino-1300To1375_mLSP-1to1250_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-FastAsympt25ns_74X_mcRun2_asymptotic_v2-v1/MINIAODSIM
/SMS-T5tttt_degen_mGl-600to800_mLSP-0to625_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-FastAsympt25ns_74X_mcRun2_asymptotic_v2-v1/MINIAODSIM
/ST_t-channel_4f_leptonDecays_13TeV-amcatnlo-pythia8_TuneCUETP8M1/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/ST_t-channel_antitop_4f_leptonDecays_13TeV-powheg-pythia8_TuneCUETP8M1/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/ST_tW_antitop_5f_inclusiveDecays_13TeV-powheg-pythia8_TuneCUETP8M1/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/SingleK0L_P-1to2000_Expo_13TeV_ExpoRandomPGun/RunIISpring15DR74-AsymptNoPUReco_MCRUN2_74_V9A-v1/GEN-SIM-RECO
/SinglePhoton_E-0to100_Flat_13TeV_FlatRandomEGun/RunIISpring15DR74-AsymptNoPUReco_MCRUN2_74_V9A-v1/GEN-SIM-RECO
/TTJets_SingleLeptFromTbar_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15MiniAODv2-74X_mcRun2_asymptotic_v2_ext1-v1/MINIAODSIM
/W3JetsToLNu_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/WJetsToLNu_HT-100To200_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12_ext1-v1/MINIAODSIM
/WJetsToLNu_HT-200To400_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/WJetsToLNu_HT-600To800_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/WJetsToLNu_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/WWTo1L1Nu2Q_13TeV_amcatnloFXFX_madspin_pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/WWTo2L2Nu_13TeV-powheg/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/WZ_TuneCUETP8M1_13TeV-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/ZNuNuGJets_MonoPhoton_PtG-130_TuneCUETP8M1_13TeV-madgraph/RunIIFall15MiniAODv1-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/ZToMuMu_NNPDF30_13TeV-powheg_M_400_800/RunIISpring15DR74-Asympt25ns_MCRUN2_74_V9-v1/MINIAODSIM
/ZToMuMu_NNPDF30_13TeV-powheg_M_50_120/RunIISpring15DR74-Asympt25ns_MCRUN2_74_V9-v1/MINIAODSIM
/ZprimeToTauTau_M-5000_Tune4C_13TeV-pythia8/Phys14DR-PU20bx25_tsg_PHYS14_25_V1-v1/AODSIM
/DoubleEG/Run2015D-PromptReco-v4/MINIAOD
/DoubleEG/Run2016B-PromptReco-v2/MINIAOD
/DoubleEG/Run2016C-PromptReco-v2/MINIAOD
/DoubleEG/Run2016D-PromptReco-v2/MINIAOD
/DoubleEG/Run2016E-PromptReco-v2/MINIAOD
/DoubleEG/Run2016F-PromptReco-v1/MINIAOD
/DoubleEG/Run2016G-PromptReco-v1/MINIAOD
/DoubleMuon/Run2016B-PromptReco-v2/MINIAOD
/DoubleMuon/Run2016C-PromptReco-v2/MINIAOD
/DoubleMuon/Run2016D-PromptReco-v2/MINIAOD
/DoubleMuon/Run2016E-PromptReco-v2/MINIAOD
/DoubleMuon/Run2016F-PromptReco-v1/MINIAOD
/DoubleMuon/Run2016G-PromptReco-v1/MINIAOD
/DoubleMuonLowMass/Run2016B-PromptReco-v2/MINIAOD
/DoubleMuonLowMass/Run2016C-PromptReco-v2/MINIAOD
/DoubleMuonLowMass/Run2016D-PromptReco-v2/MINIAOD
/DoubleMuonLowMass/Run2016E-PromptReco-v2/MINIAOD
/DoubleMuonLowMass/Run2016F-PromptReco-v1/MINIAOD
/DoubleMuonLowMass/Run2016G-PromptReco-v1/MINIAOD
/HTMHT/Run2015D-PromptReco-v4/MINIAOD
/HTMHT/Run2016B-PromptReco-v2/MINIAOD
/HTMHT/Run2016C-PromptReco-v2/MINIAOD
/HTMHT/Run2016D-PromptReco-v2/MINIAOD
/HTMHT/Run2016E-PromptReco-v2/MINIAOD
/HTMHT/Run2016F-PromptReco-v1/MINIAOD
/HTMHT/Run2016G-PromptReco-v1/MINIAOD
/JetHT/Run2015D-PromptReco-v4/MINIAOD
/JetHT/Run2016B-PromptReco-v2/MINIAOD
/JetHT/Run2016C-PromptReco-v2/MINIAOD
/JetHT/Run2016D-PromptReco-v2/MINIAOD
/JetHT/Run2016E-PromptReco-v2/MINIAOD
/JetHT/Run2016F-PromptReco-v1/MINIAOD
/JetHT/Run2016G-PromptReco-v1/MINIAOD
/MET/Run2015D-PromptReco-v4/MINIAOD
/MET/Run2016B-PromptReco-v2/MINIAOD
/MET/Run2016C-PromptReco-v2/MINIAOD
/MET/Run2016D-PromptReco-v2/MINIAOD
/MET/Run2016E-PromptReco-v2/MINIAOD
/MET/Run2016F-PromptReco-v1/MINIAOD
/MET/Run2016G-PromptReco-v1/MINIAOD
/MuonEG/Run2016B-PromptReco-v2/MINIAOD
/MuonEG/Run2016C-PromptReco-v2/MINIAOD
/MuonEG/Run2016D-PromptReco-v2/MINIAOD
/MuonEG/Run2016E-PromptReco-v2/MINIAOD
/MuonEG/Run2016F-PromptReco-v1/MINIAOD
/MuonEG/Run2016G-PromptReco-v1/MINIAOD
/SingleElectron/Run2015D-PromptReco-v4/MINIAOD
/SingleElectron/Run2016B-PromptReco-v2/MINIAOD
/SingleElectron/Run2016C-PromptReco-v2/MINIAOD
/SingleElectron/Run2016D-PromptReco-v2/MINIAOD
/SingleElectron/Run2016E-PromptReco-v2/MINIAOD
/SingleElectron/Run2016F-PromptReco-v1/MINIAOD
/SingleElectron/Run2016G-PromptReco-v1/MINIAOD
/SingleMuon/Run2015D-PromptReco-v4/MINIAOD
/SingleMuon/Run2016B-PromptReco-v2/MINIAOD
/SingleMuon/Run2016C-PromptReco-v2/MINIAOD
/SingleMuon/Run2016D-PromptReco-v2/MINIAOD
/SingleMuon/Run2016E-PromptReco-v2/MINIAOD
/SingleMuon/Run2016F-PromptReco-v1/MINIAOD
/SingleMuon/Run2016G-PromptReco-v1/MINIAOD
/SinglePhoton/Run2016B-PromptReco-v2/MINIAOD
/SinglePhoton/Run2016C-PromptReco-v2/MINIAOD
/SinglePhoton/Run2016D-PromptReco-v2/MINIAOD
/SinglePhoton/Run2016E-PromptReco-v2/MINIAOD
/SinglePhoton/Run2016F-PromptReco-v1/MINIAOD
/SinglePhoton/Run2016G-PromptReco-v1/MINIAOD
/Tau/Run2016B-PromptReco-v2/MINIAOD
/Tau/Run2016C-PromptReco-v2/MINIAOD
/Tau/Run2016D-PromptReco-v2/MINIAOD
/Tau/Run2016E-PromptReco-v2/MINIAOD
/Tau/Run2016F-PromptReco-v1/MINIAOD
/Tau/Run2016G-PromptReco-v1/MINIAOD
I came across this problem when I tried to produce some ntuple, using either splitFactor > 1
or fineSplitFactor > 1
.
This produces a number of Chunks, but then heppy_check.py
returns me something like
[manzoni@lxplus0006 trash]$ heppy_check.py *
HiggsTTHInclusive125_Chunk0 cannot find number of processed events
...
list of bad directories:
['HiggsTTHInclusive125_Chunk0',
...]
All the Chunks are bad.
This boils down to the fact that the log.txt
file which is produced for each Chunk is always empty.
Everything happens in looper.py
By default, the log messages with severity lower than 'warning' are not written into log.txt
.
A bunch of messages, including this for instance, are not saved.
However, heppy_check.py
parses log.txt
and infers the number of processed events from that line. In case it doesn't fint the number of processed events it labels the given Chunk as bad.
This in not the proper behaviour, I believe.
Two possible solutions:
I went for 2. in my fork, but @steggema told me that this was debated in the past and this solution is probably already discarded.
EDIT: I am aligned to cgm-central CMGTools-from-CMSSW_7_2_3 branch
Somebody should take care that any development on lepton IDs in 7.0.X is propagated to the CMGTools python classes.
Currently, what's implemented is:
As a sensible starting point for CSA14 we probably want at least to have at least also the Run 1 Cut-based, and to have for both IDs the choice between 7.0.X shower shapes and 5.3.X shower shapes (the "full5x5" ones)
Hi all,
I realized that the home directory of the cmgtools is currently > 90% full.
There seems to be lots of log files crated by the production.
Please delete unnecessary files if you don't need anymore.
Regards,
Yuta
Hi Sezen,
Would it be possible to transfer the following samples?
/TT_Tune4C_13TeV-pythia8-tauola/Fall13dr-tsg_PU40bx25_POSTLS162_V2-v1/GEN-SIM-RAW
/DYJetsToLL_M-50_13TeV-pythia6/Fall13dr-tsg_PU40bx25_POSTLS162_V2-v1/GEN-SIM-RAW
The request is a bit unusual as these are GEN-SIM-RAW samples, but the samples aren't particularly large.
Thanks,
Jan
We have classes and dictionaries for a lot of fancy composite types (e.g. DiMuonDiElectron) many of which are probably unused: which we could clean up to reduce the clutter, save compilation time, library size
Can people tell what they are using? my limited understanding is that ZZ and ttH use none, tau tau probably uses the cmg::DiTau<X,Y> objects, with X,Y being leptons, possibly hadronic susy might use DiObjects with PFJets, BaseJets or Hemispheres.
Does anybody e.g. use the W's (pairs lepton+met) ?
Dear Sezen,
I'm starting here a new post that we can use for the requests of new samples to be transferred to produce the PFAOD. I want to test the new version of the production tools Giovanni just committed (PFAOD, cmgTuple, final-trees with the python framework), so could you please transfer for me the sample:
/TTWWJets_8TeV-madgraph/Summer12_DR53X-PU_S10_START53_V7A-v1/AODSIM/
Thanks,
Cristina
Hi Giovanni,
Good idea to have a separate issue - I made a new one.
/ZZTo4L_Tune4C_13TeV-powheg-pythia8/Phys14DR-PU20bx25_PHYS14_25_V1-v1/MINIAODSIM
/GluGluToHToZZTo4L_M-125_13TeV-powheg-pythia6/Phys14DR-PU20bx25_tsg_PHYS14_25_V1-v1/MINIAODSIM
/GluGluToHToZZTo4L_M-125_13TeV-powheg-pythia6/Phys14DR-PU40bx25_PHYS14_25_V1-v1/MINIAODSIM
The request is here,
https://cmsweb.cern.ch/phedex/prod/Request::View?request=429839
Yuta
Hi Sezen,
Would it be possible to transfer following post-LS1 samples?
I would like to use these samples for the PFAOD production.
/DYJetsToLL_M-50_13TeV-pythia6/Fall13dr-tsg_PU20bx25_POSTLS162_V2-v1/AODSIM
/DYJetsToLL_M-50_13TeV-pythia6/Fall13dr-tsg_PU40bx50_POSTLS162_V1-v1/AODSIM
/DYJetsToLL_M-50_13TeV-pythia6/Fall13dr-tsg_PU40bx25_POSTLS162_V2-v1/AODSIM
/DYJetsToLL_M-50_13TeV-pythia6/Fall13dr-tsg_PU80bx50_POSTLS162_V1-v1/AODSIM
/VBF_HToTauTau_M-125_13TeV-powheg-pythia6/Fall13dr-tsg_PU20bx25_POSTLS162_V2-v1/AODSIM
/VBF_HToTauTau_M-125_13TeV-powheg-pythia6/Fall13dr-tsg_PU40bx25_POSTLS162_V2-v1/AODSIM
/VBF_HToTauTau_M-125_13TeV-powheg-pythia6/Fall13dr-tsg_PU40bx50_POSTLS162_V2-v1/AODSIM
/VBF_HToTauTau_M-125_13TeV-powheg-pythia6/Fall13dr-tsg_PU80bx50_POSTLS162_V1-v2/AODSIM
/QCD_Pt-15to3000_Tune4C_Flat_13TeV_pythia8/Fall13dr-NoPileUp_POSTLS162_V1-v1/AODSIM
/QCD_Pt-15to3000_Tune4C_Flat_13TeV_pythia8/Fall13dr-PU20bx25_POSTLS162_V2-v1/AODSIM
/QCD_Pt-15to3000_Tune4C_Flat_13TeV_pythia8/Fall13dr-PU40bx25_POSTLS162_V2-v1/AODSIM
/QCD_Pt-15to3000_Tune4C_Flat_13TeV_pythia8/Fall13dr-PU40bx50_POSTLS162_V1-v1/AODSIM
/QCD_Pt-15to3000_Tune4C_Flat_13TeV_pythia8/Fall13dr-PU80bx50_POSTLS162_V1-v1/AODSIM
Thanks, Yuta
Hi Marc,
Can we get the following important background samples back to CERN that have just been removed?
/TT_TuneCUETP8M2T4_13TeV-powheg-pythia8/RunIISummer16MiniAODv2-PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6-v1/MINIAODSIM
/DY1JetsToLL_M-50_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISummer16MiniAODv2-PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6-v1/MINIAODSIM
/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISummer16MiniAODv2-PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6_ext2-v1/MINIAODSIM
/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISummer16MiniAODv2-PUMoriond17_80X_mcRun2_asymptotic_2016_TrancheIV_v6_ext1-v2/MINIAODSIM
Thanks,
Jan
Dear experts,
I used cmg package on our local site.(outside CERN)
On CMGTools / Production / python / eostools.py and dataset.py,
These tools need to import cmsIO module.
However, I can not find cmsIO.py from cmg package.
Where is cmsIO.py module?
Please, let me know this.
Thanks,
Regards,
Ciao.
This thread will be used by different analyzers to request tags of the code from time to time .
Hi,
I am going to produce the PFAOD for these samples:
/SMS-T2DegenerateStop_2J_mStop-250to300_mLSP-170to290_TuneZ2star_8TeV-madgraph-tauolapp/Summer12-START53_V19_FSIM_PU_S12-v1/AODSIM
/SMS-T2DegenerateStop_2J_mStop-100to150_mLSP-20to140_TuneZ2star_8TeV-madgraph-tauolapp/Summer12-START53_V19_FSIM_PU_S12-v1/AODSIM
/SMS-T2DegenerateStop_2J_mStop-325to375_mLSP-245to365_TuneZ2star_8TeV-madgraph-tauolapp/Summer12-START53_V19_FSIM_PU_S12-v1/AODSIM
/SMS-T2DegenerateStop_2J_mStop-175to225_mLSP-95to215_TuneZ2star_8TeV-madgraph-tauolapp/Summer12-START53_V19_FSIM_PU_S12-v1/AODSIM
(Please use this thread to tell the others if you are using the cmgtools account to make productions)
Cristina
In 7.0.X CMGTools we have only the isolation variables that are statically provided in the RECO dataformats or that are added during miniAOD production in the PAT dataformat.
Eventually we'll have to integrate the recipes to compute these isolations with the new pileup mitigation strategies, and in case of electrons the proper veto of footprint PF Candidates (according to Josh B. this is not yet done in the RECO nor probably in the PAT)
We could possibly want to support the feature to read from anywhere via xrootd as per
https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookXrootdService
It's not fast, and one shouldn't abuse of the network it by running too many jobs reading from remote, but it can be useful for testing before samples are available at cern, or even for processing small samples (e.g. rare backgrounds or exotic signals)
Hi @mdunser,
can you please transfer the following datasets?
/QCD_Pt-30to50_MuEnrichedPt5_TuneCUETP8M1_13TeV_pythia8/RunIISpring16MiniAODv1-PUSpring16RAWAODSIM_80X_mcRun2_asymptotic_2016_v3-v1/MINIAODSIM
/QCD_Pt-170to300_MuEnrichedPt5_TuneCUETP8M1_13TeV_pythia8/RunIISpring16MiniAODv2-PUSpring16RAWAODSIM_80X_mcRun2_asymptotic_2016_miniAODv2_v0-v1/MINIAODSIM
It appears that the cmgtools user is getting only very few batch slots: at the moment we're getting less than ~200 running jobs, which seems low even for an ordinary user.
Does anybody have a way to check if we still have the proper priorities set in the batch system?
Giovanni
Dear all, while moving most of the cpu to lxplus6, CERN has extended the computing centre with capacity installed at the Wigner Data Centre in Budapest [*], and the new resources are added transparently for the users, so jobs can go to Hungary.
There is a known CPU efficiency issue for I/O intensive jobs running at Wigner and reading data from EOS in Meyrin.
I am not aware of ways to avoid this with bsub command specifications.
The complete thread can be found here
https://savannah.cern.ch/support/?142241#comment13
Ciao
Luca
[*]This will affect only SLC6 jobs, because there are no SLC5 worker nodes in Wigner
Hi,
I'd like to request the transfer of the Powheg+Herwig ttbar sample:
/TT_CT10_AUET2_8TeV-powheg-herwig/Summer12_DR53X-PU_S10_START53_V19-v1/AODSIM
Thanks, Filip
Hi Sezen,
Could you request the Phedex transfer of:
/MET/Run2012A-22Jan2013-v1/AOD
/MET/Run2012B-22Jan2013-v1/AOD
/MET/Run2012C-22Jan2013-v1/AOD
/METParked/Run2012D-22Jan2013-v1/AOD
Regards,
Markus
We are migrating to github for issue tracking.
We still depend on savannah for the publication of the CMG samples. We decided to stop publish information to savannah, and to rely on the sole CMG DB to keep track of our samples.
Somebody should take care of removing the corresponding code from the CMGTools/Production package.
Dear all,
in light of the changing data formats and analysis behavior introduced by MINIAOD and also just as a spring-cleaning, we are starting a rather extensive clean-up and re-distribution campaign in the cmg-managed and controlled space on eos. Please read the following very carefully if you have strong opinions about keeping some specific data from run-1.
Have a look at this twiki for details and the files containing the deletion data:
—> https://twiki.cern.ch/twiki/bin/view/CMS/CMGCleanupSpring2016 <—
Two areas are affected by this:
this directory contains all PFAODs that were produced for 7 and 8 TeV. From what I can tell, no 13 TeV data is put there, nor are there many recently made files. The plan is to delete all files that are in the directories in this path which were modified anytime between 2011 and (including) 2013, newer ones will remain in place for now.
this directory contains most ntuples which were derived from the above mentioned PFAODs. The plan here is to delete all ntuples except those which are produced with CMGTools version V5_18. (Note that this is a tentative plan and will most likely change because certain analysis have their final ntuples in a different CMGTools version). We will ask for the archiving of the nutples of version V5_18 (and other version as needed), meaning that they will end up in CASTOR.
Please have a look into the two aforementioned files and read the twiki. Respond with details to me, this email, the github issue, or the cmg-cmgtools e-groupto if you would like to keep any of the directories that are marked with DELETE.
Please do this as soon as possible, and no later than Thursday the 21st of April 2016 for the group space and no later than Thursday the 28th of April 2016 for the user space. Deletion will start the day after.
This cleaning will free up some several hundreds of TB of diskspace, some of which will be used for the enlargement of the phedex managed local space, which should alleviate some of the issues we’re having with the relatively restrictive quota of 60 TB there.
Best,
-marc
I'm adding a task here so that we can keep a log of what is to be archived and of what has been archived and can be deleted.
To my knowledge @gitytakahas has already started archiving V5.6 (8TeV only), V5.8, V5.9, V5.10, V5.12, V5.13 and V5.14, but I don't know if the archival is complete and if the integrity of the copy has been checked (i.e. that all files have been successfully copied and none is corrupted)
Of these, I'd ask to keep on eos V5.10 for another few weeks as we're still using it for a few susy fake rate studies, though as soon as we'll be done producing the PFAOD and V5.18 CMGTuples of the QCD samples in the buffer then for us V5.10 can go too.
Could you please transfer this sample:
/SMS-T1bbbb_mGluino-1500_mLSP-100_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIISpring15DR74-Asympt25ns_MCRUN2_74_V9-v1/MINIAODSIM
Thanks,
Batool
Here is a post from Tae Jeong on savannah, see here: https://savannah.cern.ch/task/index.php?41323
Hi,
Just to give you a quick status on this,
what I need is the reference to gen particles so that I can track down jet gen-constituents which point to gen particles.
But what I found in "genParticlesPruned" is that the link was broken between genParticles and jet gen-constituent. So even though I include B-hadrons in "genParticlesPruned", I can not get gen constituents since there is no link.
I could use dR(jet, B-hadron) but this is not the way we did in the previous analysis and we would like to keep the same method.
I couldn't find a good solution so far. It looks like I need to modify "GenParticlePruner" producer in order to keep the reference. (I don't know if this can solve the problem.)
But currently we are happy with previous production (V5_13_0 and V5_12_00_44X, hopefully we can finalize our analysis with these versions). So as long as these samples are not removed, we don't need to produce pat-samples again with new release.
But for sure, it would be great if we can solve above issue just in case. I will try to take a look at more later. Any idea would be welcome.
Regards,
Taejeong
I'm now working on reviving the python tutorial (important for new people joining the group):
https://twiki.cern.ch/twiki/bin/viewauth/CMS/CMGToolsPythonAnalysis
Hi all,
As we have discussed in today's meeting, we will consider to archive / remove !(53X) samples.
The list of the samples to be affected is as followings.
https://twiki.cern.ch/twiki/bin/view/Main/Cmgtools
Please have a careful look and reply to this thread, if you find any samples to be kept.
The deadline is 5 Dec.
Cheers,
Yuta
In 7.0.X miniAODs we have the standard E/gamma POG collection of photons so it could be good to add the support for it in the python analysis framework (and get e.g. photon ids and isolations)
Also, if we want in 7.0.X the FSR photon definition a la H->4l, it should be implemented starting from the Packed PF Candidates (and the ecal energy of the pat::Muons)
This issue is open for the requests of the RunIISpring15MiniAODv2 miniAOD
The AttributeError
occurs at L449 in the method smearJets()
because some jets don't have the attribute mcJet
.
The attribute 'mcJet' is attached to jet at L443 in the method matchJets()
.
At L169, matchJets()
called with only selected jets but, at L171, smearJets()
is called with all jets.
Therefore, some jets don't have mcJet
and causes the AttributeError
.
People should check if they can use the newer trigger tools in this version so that we can drop the cmgTriggerObjectSel collection (saves 5kb/ev on data)
There's a new python module for applying trigger filters by path name
https://github.com/CERN-PH-CMG/cmg-cmssw/blob/CMG_PAT_V5_18_from-CMSSW_5_3_14/CMGTools/RootTools/python/analyzers/triggerBitFilter.py
It takes a list of triggers and optionally a list of triggers to veto. Trigger names can end in "" , as usual. They can also be prefixed by a run range (e.g. "1-203302:HLT_Mu18_v")
There's a new light trigger object that records the prescale information, it can be added to the CMGTuple by adding
process.outcmg.outputCommands.append('keep *_cmgTriggerPrescales_*_*')
Ciao,
I experienced a problem with cmsStageWithFailover.py
It doesn't catch an error while trying to copy the produced file to EOS
if eos is not installed in the node.
Quoting from STDOUT:
sending the job directory back
Destination directory /store/cmst3/user/cmgtools/CMG/SingleMu/Run2011B-12Oct2013-v1/AOD/PAT_CMG_V5_18_0 does not exist.
/pool/spool/1400199225.529427782.shell: line 31: eos: command not found
Maybe we could grep for "command not found" in the STDOUT
as additional check in the production tools.
Ciao
Luca
Nobody's using the stringbitsets, and the stringbitsets definitions don't correspond anymore to the official POG recommendations.
We decided to remove them.
To do:
Hi Yuta,
could you please transfer these datasets?
/WWTo2L2Nu_13TeV-powheg/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM
/DoubleEG/Run2015D-16Dec2015-v2/MINIAOD
/MuonEG/Run2015D-16Dec2015-v1/MINIAOD
Thanks,
Marco
As said in today's meeting, in the current code there is the possibility of producing a slimmer version of the full PF Candidates compressed using 16 bits per variable.
You can add them by modifying the cfg to produce the cmgtuple
CMGTools/Common/prod/PATCMG_cfg.py
and adding
process.outcmg.outputCommands.append('keep cmgPackedCandidates_cmgCandidates_*_*')
Somebody should check if these compressed candidates are good enough for jet substructure, met or whatever other purpose the pf candidates were serving in the cmg tuple.
Hi all,
I'm starting a new issue for data-transfers of Run2016. Many of the datasets, despite being subscribed originally to CERN are being deleted fairly rapidly.
After some interaction with CompOps it has become clear that there is no longer any guarantee that any dataset of any run-period in 2016 will remain at the CERN T2. Computing resources are hitting their absolute limit pretty much everywhere, so whatever workflow that used to work in 2015 is no longer guaranteed to work in 2016.
For users of heppy and heppy_batch that means that they should have a look into using one of two options:
useAAA which copies the root files locally to /tmp via xrd and runs on them from there. In my experience this works well, though it sometimes takes many resubmissions for it to finally run through. This requires setting the environment variable X509_USER_PROXY to an empty file that exists.
crab3 works well, with some adaptions to the run-config and the crab/ directory. If you plan on running loads of data now or in the future, this might be worthwhile checking out. In my experience this work well.
It is, of course, still possible to transfer datasets to our local buffer, but be advised that this may take up to a week or more from your request on this thread to the dataset finally being present at CERN, and in addition the buffer has a limited size, so there is no way we can store all data in it just for the sake of it.
Best,
-m
Hi,
While running CMG (CMGTools-from-CMSSW_7_2_3) to make ntuples we noticed that the maximum speeds we're getting are around 30-50 events/second. During Run 1 the ntuplising stage usually ran at O1000s events/second. Do you know what could be causing such a slower speed and how it could be optimised? We’ve already exported the complex variable (alphaT) calculation to a C module.
Best,
Matthew
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.