BranchesTags

Latest commit

History

50 Commits

Folders and files

NameName
Last commit message
Last commit date
run362321
run362321
run381594
run381594
run398183
run398183
run402360
run402360
README.md
README.md

Repository files navigation

DQM-Integration

Repository to store input files for the CMSSW DQM/Integration package.

Streamer Files

For the moment this repository is used to store streamer files that are needed as input for the unitTest.

Currently the unitTest that make use of input streamer files are:

  • the onlinebeammonitor_dqm_sourceclient_cfg.py client, it reads the streamDQM (prepared such that only the hltOnlineBeamSpot is present in the file) streamer files regenerated from run 381594 (from 2024E pp run OMS link): 
    run381594/run381594_ls1000_streamDQM_pid1388480.dat
run381594/run381594_ls1000_streamDQM_pid1388480.jsn
  • the beamhlt_dqm_sourceclient-live_cfg.py client, it reads the streamDQMOnlineBeamspot streamer files regenerated from run 381594 (from 2024E pp run OMS link): 
    run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.dat
run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.jsn
  • the ecalgpu_dqm_sourceclient-live_cfg.py, hcalgpu_dqm_sourceclient-live_cfg.py, pixelgpu_dqm_sourceclient-live_cfg.py and pfgpu_dqm_sourceclient-live_cfg.py read the streamDQMGPUvsCPU streamer files regenerated from run 398183 (from Run2025G pp run OMS link): 
    run398183_ls0142_streamDQMGPUvsCPU_pid3119228.dat
run398183_ls0142_streamDQMGPUvsCPU_pid3119228.jsn
  • the sistrip_approx_dqm_sourceclient-live_cfg.py reads the streamDQM streamer files regenerated from run 362321 (from 2022 HI run, OMS link, though they have been re-HLT'ed, see for more details at CMSHLT-2884): 
    run362321/run362321_ls0231_streamHIDQM_pid276864.dat
run362321/run362321_ls0231_streamHIDQM_pid276864.jsn
  • the scouting_dqm_sourceclient-live_cfg.py reads the streamDQMOnlineScouting streamer files generated from run 398183 (from Run2025G pp run, OMS link): 
    run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.dat
run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.jsn
  • the ngt_dqm_sourceclient-live_cfg.py reads the streamDQMTestDataScouting streamer files generated from run 402360 (from Run2026B pp run, OMS link): 
    run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.dat
run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.jsn

Recipe to regenerate Streamer files (when streamer layout gets broken)

In repsonse to issue cms-sw/cmssw#45224, streamer files have been regenerated in a release that contains cms-sw/cmssw#44978 (in this case CMSSW_14_0_9_MULTIARCHS.

Recipe for streamDQM

This was done using the following script for the pp data:

#!/bin/bash -ex
RUNNUMBER=381594
LUMISECTION=1000
# cmsrel CMSSW_14_0_9_MULTIARCHS# cd CMSSW_14_0_9_MULTIARCHS/src# cmsenv# scram b
INPUTFILE=root://eoscms.cern.ch//store/express/Run2024E/ExpressPhysics/FEVT/Express-v1/000/381/594/00000/1e2c895f-a250-45be-a7ff-ee95e636a6e9.root
rm -rf run${RUNNUMBER}*# run on 300 events of LS 1000, with 300 events per input file
convertToRaw -f 300 -l 300 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --runNumber "${RUNNUMBER}">"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log

Recipe for streamHIDQM

While the following script for the HIon data:

#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-03-03-2300# cd CMSSW_16_1_X_2026-03-03-2300/src# cmsenv# scram b# run 362321, LSs 231-232
RUNNUMBER=362321
LUMISECTION=231
INPUTFILE=root://eoscms.cern.ch//eos/cms/store/user/cmsbuild//store/hidata/HIRun2022A/HITestRaw0/RAW/v1/000/362/321/00000/f467ee64-fc64-47a6-9d8a-7ca73ebca2bd.root
HLTMENU=/dev/CMSSW_16_0_0/HIon/V31
rm -rf run${RUNNUMBER}*# run on 100 events of LS 231, with 100 events per input file
convertToRaw -f 100 -l 100 -r ${RUNNUMBER}:${LUMISECTION} -s rawDataRepacker -o . -- "${INPUTFILE}"
tmpfile=tmp.py
hltConfigFromDB --configName "${HLTMENU}">"${tmpfile}"
sed -i 's|process = cms.Process( "HLT" )|from Configuration.Eras.Era_Run3_cff import Run3\nprocess = cms.Process( "HLT", Run3 )|g'"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")#process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# override the GlobalTag, connection string and pfnPrefixfrom Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTagprocess.GlobalTag = customiseGlobalTag( process.GlobalTag, globaltag = "160X_dataRun3_HLT_v1", conditions = "L1Menu_CollisionsHeavyIons2025_v1_0_3_xml,L1TUtmTriggerMenuRcd,frontier://FrontierProd/CMS_CONDITIONS,,9999-12-31 23:59:59.000")# run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLTfrom HLTrigger.Configuration.CustomConfigs import L1REPACKprocess = L1REPACK(process, "uGT")# to run without any HLT prescalesdel process.PrescaleService# # to run using the same HLT prescales as used online in LS 231# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamHIDQM_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}

Recipe for streamDQMOnlineScouting

The streamer files for the streamDQMOnlineScouting were prepared using the scouting specific menu:

#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-01-07-2300# cd CMSSW_16_1_X_2026-01-07-2300/src# cmsenv# scram b
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# LS 174 
INPUTFILES="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 5000 events of given LS, with 1000 event limits per input file
convertToRaw -l 5000 -f 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- ${INPUTFILES}
tmpfile=$(mktemp)
hltConfigFromDB --configName /users/jprendi/ScoutingOnlineDQM/Test0/HLT/V7 > dump.py
cat <<@EOF >> dump.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.GlobalTag.globaltag = cms.string('150X_dataRun3_HLT_v1')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True@EOF
edmConfigDump dump.py > hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMOnlineScouting_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}

Recipe for streamDQMTestDataScouting

The streamer files for the streamDQMTestDataScouting were prepared using the following script:

#!/bin/bash -ex# cmsrel CMSSW_16_0_6_patch1# cd CMSSW_16_0_6_patch1/src# cmsenv
RUNNUMBER=402360
LUMISECTION=193
hltLabel=testAddTriggerEvents
hltLabel1="${hltLabel}_hlt1"
hltLabel2="${hltLabel}_hlt2"
INPUTFILE="root://eoscms.cern.ch//store/data/Run2026B/EphemeralHLTPhysics0/RAW/v1/000/402/360/00000/aa48b021-759e-4592-86e5-75eee46b88fc.root"
MENU=/online/collisions/2026/2e34/v1.2/HLT/V2
######################################### Helpers########################################run_cms() {
local cfg=$1local log=$2
bash -c "echo \$\$ > cmsrun.pid; exec cmsRun ${cfg} >& ${log}"
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"
}
prepare_output() {
local tag=$1# old_runXXX or new_runXXX
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*
mkdir -p prepared
local base="run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMTestDataScouting_pid${job_pid}"
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMTestDataScouting_pid${job_pid}.ini \
run${RUNNUMBER}/${base}.dat \
> prepared/${base}.dat
cp run${RUNNUMBER}/${base}.jsn \
prepared/${base}_prep.jsn
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"prepared/${base}_prep.jsn">"prepared/${base}.jsn"
rm -f prepared/${base}_prep.jsn
rm -rf run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared ${tag}
}
convert_input() {
convertToRaw -f 1000 -l 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
}
######################################### Setup########################################
rm -rf run${RUNNUMBER}*
hltConfigFromDB --configName ${MENU}>${hltLabel1}.py
cat <<@EOF >> ${hltLabel1}.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicestreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMTestDataScoutingOutput')for foo in streamPaths: process.__delattr__(foo)@EOF######################################### First run########################################
convert_input
run_cms "${hltLabel1}.py""${hltLabel1}.log"
prepare_output "${RUNNUMBER}"

Recipe for streamDQMGPUvsCPU

The streamer files for the streamDQMGPUVsCPU were prepared using the following script:

#!/bin/bash -ex
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# cmsrel CMSSW_16_1_X_2026-01-21-2300 # cd CMSSW_16_1_X_2026-01-21-2300/src/# cmsenv
INPUTFILE="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 500 events of LS, with 500 events per input file
convertToRaw -f 25 -l 25 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --configName /dev/CMSSW_16_0_0/GRun/V7 >"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = Trueprocess.GlobalTag.globaltag = cms.string( "150X_dataRun3_HLT_v1" )# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True# customization for the menufrom HLTrigger.Configuration.customizeHLTforCMSSW import *process = customizeHLTfor49799(process)process = customizeHLTfor49852(process)## just output the GPU vs CPU outputstreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMGPUvsCPUOutput')for foo in streamPaths: process.__delattr__(foo)@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMGPUvsCPU_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}

Possible extenstions

There are two more clients for which the unitTets could be activated in the future, namely:

 <!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-ecalcalib_dqm_sourceclient" command="runtest.sh ecalcalib_dqm_sourceclient-live_cfg.py" /> -->
<!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-hcalcalib_dqm_sourceclient" command="runtest.sh hcalcalib_dqm_sourceclient-live_cfg.py" /> -->

About

No description, website, or topics provided.

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

6 watching

Forks

5 forks
Report repository

Releases

Packages

Contributors

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Latest commit
History
50 Commits
Folders and files
NameName
Last commit message
Last commit date
run362321
run362321
run381594
run381594
run398183
run398183
run402360
run402360
README.md
README.md
Repository files navigation
DQM-Integration
Repository to store input files for the CMSSW DQM/Integration package.
Streamer Files
For the moment this repository is used to store streamer files that are needed as input for the unitTest.
Currently the unitTest that make use of input streamer files are:
  • the onlinebeammonitor_dqm_sourceclient_cfg.py client, it reads the streamDQM (prepared such that only the hltOnlineBeamSpot is present in the file) streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQM_pid1388480.dat
run381594/run381594_ls1000_streamDQM_pid1388480.jsn

  • the beamhlt_dqm_sourceclient-live_cfg.py client, it reads the streamDQMOnlineBeamspot streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.dat
run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.jsn

  • the ecalgpu_dqm_sourceclient-live_cfg.py, hcalgpu_dqm_sourceclient-live_cfg.py, pixelgpu_dqm_sourceclient-live_cfg.py and pfgpu_dqm_sourceclient-live_cfg.py read the streamDQMGPUvsCPU streamer files regenerated from run 398183 (from Run2025G pp run OMS link):

    run398183_ls0142_streamDQMGPUvsCPU_pid3119228.dat
run398183_ls0142_streamDQMGPUvsCPU_pid3119228.jsn

  • the sistrip_approx_dqm_sourceclient-live_cfg.py reads the streamDQM streamer files regenerated from run 362321 (from 2022 HI run, OMS link, though they have been re-HLT'ed, see for more details at CMSHLT-2884):

    run362321/run362321_ls0231_streamHIDQM_pid276864.dat
run362321/run362321_ls0231_streamHIDQM_pid276864.jsn

  • the scouting_dqm_sourceclient-live_cfg.py reads the streamDQMOnlineScouting streamer files generated from run 398183 (from Run2025G pp run, OMS link):

    run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.dat
run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.jsn

  • the ngt_dqm_sourceclient-live_cfg.py reads the streamDQMTestDataScouting streamer files generated from run 402360 (from Run2026B pp run, OMS link):

    run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.dat
run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.jsn 
Recipe to regenerate Streamer files (when streamer layout gets broken)
In repsonse to issue cms-sw/cmssw#45224, streamer files have been regenerated in a release that contains cms-sw/cmssw#44978 (in this case CMSSW_14_0_9_MULTIARCHS.
Recipe for streamDQM
This was done using the following script for the pp data:
#!/bin/bash -ex
RUNNUMBER=381594
LUMISECTION=1000
# cmsrel CMSSW_14_0_9_MULTIARCHS# cd CMSSW_14_0_9_MULTIARCHS/src# cmsenv# scram b
INPUTFILE=root://eoscms.cern.ch//store/express/Run2024E/ExpressPhysics/FEVT/Express-v1/000/381/594/00000/1e2c895f-a250-45be-a7ff-ee95e636a6e9.root
rm -rf run${RUNNUMBER}*# run on 300 events of LS 1000, with 300 events per input file
convertToRaw -f 300 -l 300 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --runNumber "${RUNNUMBER}">"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
Recipe for streamHIDQM
While the following script for the HIon data:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-03-03-2300# cd CMSSW_16_1_X_2026-03-03-2300/src# cmsenv# scram b# run 362321, LSs 231-232
RUNNUMBER=362321
LUMISECTION=231
INPUTFILE=root://eoscms.cern.ch//eos/cms/store/user/cmsbuild//store/hidata/HIRun2022A/HITestRaw0/RAW/v1/000/362/321/00000/f467ee64-fc64-47a6-9d8a-7ca73ebca2bd.root
HLTMENU=/dev/CMSSW_16_0_0/HIon/V31
rm -rf run${RUNNUMBER}*# run on 100 events of LS 231, with 100 events per input file
convertToRaw -f 100 -l 100 -r ${RUNNUMBER}:${LUMISECTION} -s rawDataRepacker -o . -- "${INPUTFILE}"
tmpfile=tmp.py
hltConfigFromDB --configName "${HLTMENU}">"${tmpfile}"
sed -i 's|process = cms.Process( "HLT" )|from Configuration.Eras.Era_Run3_cff import Run3\nprocess = cms.Process( "HLT", Run3 )|g'"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")#process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# override the GlobalTag, connection string and pfnPrefixfrom Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTagprocess.GlobalTag = customiseGlobalTag( process.GlobalTag, globaltag = "160X_dataRun3_HLT_v1", conditions = "L1Menu_CollisionsHeavyIons2025_v1_0_3_xml,L1TUtmTriggerMenuRcd,frontier://FrontierProd/CMS_CONDITIONS,,9999-12-31 23:59:59.000")# run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLTfrom HLTrigger.Configuration.CustomConfigs import L1REPACKprocess = L1REPACK(process, "uGT")# to run without any HLT prescalesdel process.PrescaleService# # to run using the same HLT prescales as used online in LS 231# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamHIDQM_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMOnlineScouting
The streamer files for the streamDQMOnlineScouting were prepared using the scouting specific menu:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-01-07-2300# cd CMSSW_16_1_X_2026-01-07-2300/src# cmsenv# scram b
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# LS 174 
INPUTFILES="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 5000 events of given LS, with 1000 event limits per input file
convertToRaw -l 5000 -f 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- ${INPUTFILES}
tmpfile=$(mktemp)
hltConfigFromDB --configName /users/jprendi/ScoutingOnlineDQM/Test0/HLT/V7 > dump.py
cat <<@EOF >> dump.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.GlobalTag.globaltag = cms.string('150X_dataRun3_HLT_v1')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True@EOF
edmConfigDump dump.py > hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMOnlineScouting_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMTestDataScouting
The streamer files for the streamDQMTestDataScouting were prepared using the following script:
#!/bin/bash -ex# cmsrel CMSSW_16_0_6_patch1# cd CMSSW_16_0_6_patch1/src# cmsenv
RUNNUMBER=402360
LUMISECTION=193
hltLabel=testAddTriggerEvents
hltLabel1="${hltLabel}_hlt1"
hltLabel2="${hltLabel}_hlt2"
INPUTFILE="root://eoscms.cern.ch//store/data/Run2026B/EphemeralHLTPhysics0/RAW/v1/000/402/360/00000/aa48b021-759e-4592-86e5-75eee46b88fc.root"
MENU=/online/collisions/2026/2e34/v1.2/HLT/V2
######################################### Helpers########################################run_cms() {
local cfg=$1local log=$2
bash -c "echo \$\$ > cmsrun.pid; exec cmsRun ${cfg} >& ${log}"
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"
}
prepare_output() {
local tag=$1# old_runXXX or new_runXXX
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*
mkdir -p prepared
local base="run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMTestDataScouting_pid${job_pid}"
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMTestDataScouting_pid${job_pid}.ini \
run${RUNNUMBER}/${base}.dat \
> prepared/${base}.dat
cp run${RUNNUMBER}/${base}.jsn \
prepared/${base}_prep.jsn
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"prepared/${base}_prep.jsn">"prepared/${base}.jsn"
rm -f prepared/${base}_prep.jsn
rm -rf run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared ${tag}
}
convert_input() {
convertToRaw -f 1000 -l 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
}
######################################### Setup########################################
rm -rf run${RUNNUMBER}*
hltConfigFromDB --configName ${MENU}>${hltLabel1}.py
cat <<@EOF >> ${hltLabel1}.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicestreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMTestDataScoutingOutput')for foo in streamPaths: process.__delattr__(foo)@EOF######################################### First run########################################
convert_input
run_cms "${hltLabel1}.py""${hltLabel1}.log"
prepare_output "${RUNNUMBER}"
Recipe for streamDQMGPUvsCPU
The streamer files for the streamDQMGPUVsCPU were prepared using the following script:
#!/bin/bash -ex
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# cmsrel CMSSW_16_1_X_2026-01-21-2300 # cd CMSSW_16_1_X_2026-01-21-2300/src/# cmsenv
INPUTFILE="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 500 events of LS, with 500 events per input file
convertToRaw -f 25 -l 25 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --configName /dev/CMSSW_16_0_0/GRun/V7 >"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = Trueprocess.GlobalTag.globaltag = cms.string( "150X_dataRun3_HLT_v1" )# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True# customization for the menufrom HLTrigger.Configuration.customizeHLTforCMSSW import *process = customizeHLTfor49799(process)process = customizeHLTfor49852(process)## just output the GPU vs CPU outputstreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMGPUvsCPUOutput')for foo in streamPaths: process.__delattr__(foo)@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMGPUvsCPU_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Possible extenstions
There are two more clients for which the unitTets could be activated in the future, namely:
 <!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-ecalcalib_dqm_sourceclient" command="runtest.sh ecalcalib_dqm_sourceclient-live_cfg.py" /> -->
<!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-hcalcalib_dqm_sourceclient" command="runtest.sh hcalcalib_dqm_sourceclient-live_cfg.py" /> -->

About
No description, website, or topics provided.
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
6 watching
Forks
5 forks
Report repository
Releases
Packages
Contributors

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Latest commit
History
50 Commits
Folders and files
NameName
Last commit message
Last commit date
run362321
run362321
run381594
run381594
run398183
run398183
run402360
run402360
README.md
README.md
Repository files navigation
DQM-Integration
Repository to store input files for the CMSSW DQM/Integration package.
Streamer Files
For the moment this repository is used to store streamer files that are needed as input for the unitTest.
Currently the unitTest that make use of input streamer files are:
  • the onlinebeammonitor_dqm_sourceclient_cfg.py client, it reads the streamDQM (prepared such that only the hltOnlineBeamSpot is present in the file) streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQM_pid1388480.dat
run381594/run381594_ls1000_streamDQM_pid1388480.jsn

  • the beamhlt_dqm_sourceclient-live_cfg.py client, it reads the streamDQMOnlineBeamspot streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.dat
run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.jsn

  • the ecalgpu_dqm_sourceclient-live_cfg.py, hcalgpu_dqm_sourceclient-live_cfg.py, pixelgpu_dqm_sourceclient-live_cfg.py and pfgpu_dqm_sourceclient-live_cfg.py read the streamDQMGPUvsCPU streamer files regenerated from run 398183 (from Run2025G pp run OMS link):

    run398183_ls0142_streamDQMGPUvsCPU_pid3119228.dat
run398183_ls0142_streamDQMGPUvsCPU_pid3119228.jsn

  • the sistrip_approx_dqm_sourceclient-live_cfg.py reads the streamDQM streamer files regenerated from run 362321 (from 2022 HI run, OMS link, though they have been re-HLT'ed, see for more details at CMSHLT-2884):

    run362321/run362321_ls0231_streamHIDQM_pid276864.dat
run362321/run362321_ls0231_streamHIDQM_pid276864.jsn

  • the scouting_dqm_sourceclient-live_cfg.py reads the streamDQMOnlineScouting streamer files generated from run 398183 (from Run2025G pp run, OMS link):

    run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.dat
run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.jsn

  • the ngt_dqm_sourceclient-live_cfg.py reads the streamDQMTestDataScouting streamer files generated from run 402360 (from Run2026B pp run, OMS link):

    run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.dat
run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.jsn 
Recipe to regenerate Streamer files (when streamer layout gets broken)
In repsonse to issue cms-sw/cmssw#45224, streamer files have been regenerated in a release that contains cms-sw/cmssw#44978 (in this case CMSSW_14_0_9_MULTIARCHS.
Recipe for streamDQM
This was done using the following script for the pp data:
#!/bin/bash -ex
RUNNUMBER=381594
LUMISECTION=1000
# cmsrel CMSSW_14_0_9_MULTIARCHS# cd CMSSW_14_0_9_MULTIARCHS/src# cmsenv# scram b
INPUTFILE=root://eoscms.cern.ch//store/express/Run2024E/ExpressPhysics/FEVT/Express-v1/000/381/594/00000/1e2c895f-a250-45be-a7ff-ee95e636a6e9.root
rm -rf run${RUNNUMBER}*# run on 300 events of LS 1000, with 300 events per input file
convertToRaw -f 300 -l 300 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --runNumber "${RUNNUMBER}">"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
Recipe for streamHIDQM
While the following script for the HIon data:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-03-03-2300# cd CMSSW_16_1_X_2026-03-03-2300/src# cmsenv# scram b# run 362321, LSs 231-232
RUNNUMBER=362321
LUMISECTION=231
INPUTFILE=root://eoscms.cern.ch//eos/cms/store/user/cmsbuild//store/hidata/HIRun2022A/HITestRaw0/RAW/v1/000/362/321/00000/f467ee64-fc64-47a6-9d8a-7ca73ebca2bd.root
HLTMENU=/dev/CMSSW_16_0_0/HIon/V31
rm -rf run${RUNNUMBER}*# run on 100 events of LS 231, with 100 events per input file
convertToRaw -f 100 -l 100 -r ${RUNNUMBER}:${LUMISECTION} -s rawDataRepacker -o . -- "${INPUTFILE}"
tmpfile=tmp.py
hltConfigFromDB --configName "${HLTMENU}">"${tmpfile}"
sed -i 's|process = cms.Process( "HLT" )|from Configuration.Eras.Era_Run3_cff import Run3\nprocess = cms.Process( "HLT", Run3 )|g'"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")#process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# override the GlobalTag, connection string and pfnPrefixfrom Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTagprocess.GlobalTag = customiseGlobalTag( process.GlobalTag, globaltag = "160X_dataRun3_HLT_v1", conditions = "L1Menu_CollisionsHeavyIons2025_v1_0_3_xml,L1TUtmTriggerMenuRcd,frontier://FrontierProd/CMS_CONDITIONS,,9999-12-31 23:59:59.000")# run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLTfrom HLTrigger.Configuration.CustomConfigs import L1REPACKprocess = L1REPACK(process, "uGT")# to run without any HLT prescalesdel process.PrescaleService# # to run using the same HLT prescales as used online in LS 231# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamHIDQM_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMOnlineScouting
The streamer files for the streamDQMOnlineScouting were prepared using the scouting specific menu:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-01-07-2300# cd CMSSW_16_1_X_2026-01-07-2300/src# cmsenv# scram b
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# LS 174 
INPUTFILES="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 5000 events of given LS, with 1000 event limits per input file
convertToRaw -l 5000 -f 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- ${INPUTFILES}
tmpfile=$(mktemp)
hltConfigFromDB --configName /users/jprendi/ScoutingOnlineDQM/Test0/HLT/V7 > dump.py
cat <<@EOF >> dump.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.GlobalTag.globaltag = cms.string('150X_dataRun3_HLT_v1')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True@EOF
edmConfigDump dump.py > hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMOnlineScouting_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMTestDataScouting
The streamer files for the streamDQMTestDataScouting were prepared using the following script:
#!/bin/bash -ex# cmsrel CMSSW_16_0_6_patch1# cd CMSSW_16_0_6_patch1/src# cmsenv
RUNNUMBER=402360
LUMISECTION=193
hltLabel=testAddTriggerEvents
hltLabel1="${hltLabel}_hlt1"
hltLabel2="${hltLabel}_hlt2"
INPUTFILE="root://eoscms.cern.ch//store/data/Run2026B/EphemeralHLTPhysics0/RAW/v1/000/402/360/00000/aa48b021-759e-4592-86e5-75eee46b88fc.root"
MENU=/online/collisions/2026/2e34/v1.2/HLT/V2
######################################### Helpers########################################run_cms() {
local cfg=$1local log=$2
bash -c "echo \$\$ > cmsrun.pid; exec cmsRun ${cfg} >& ${log}"
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"
}
prepare_output() {
local tag=$1# old_runXXX or new_runXXX
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*
mkdir -p prepared
local base="run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMTestDataScouting_pid${job_pid}"
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMTestDataScouting_pid${job_pid}.ini \
run${RUNNUMBER}/${base}.dat \
> prepared/${base}.dat
cp run${RUNNUMBER}/${base}.jsn \
prepared/${base}_prep.jsn
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"prepared/${base}_prep.jsn">"prepared/${base}.jsn"
rm -f prepared/${base}_prep.jsn
rm -rf run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared ${tag}
}
convert_input() {
convertToRaw -f 1000 -l 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
}
######################################### Setup########################################
rm -rf run${RUNNUMBER}*
hltConfigFromDB --configName ${MENU}>${hltLabel1}.py
cat <<@EOF >> ${hltLabel1}.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicestreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMTestDataScoutingOutput')for foo in streamPaths: process.__delattr__(foo)@EOF######################################### First run########################################
convert_input
run_cms "${hltLabel1}.py""${hltLabel1}.log"
prepare_output "${RUNNUMBER}"
Recipe for streamDQMGPUvsCPU
The streamer files for the streamDQMGPUVsCPU were prepared using the following script:
#!/bin/bash -ex
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# cmsrel CMSSW_16_1_X_2026-01-21-2300 # cd CMSSW_16_1_X_2026-01-21-2300/src/# cmsenv
INPUTFILE="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 500 events of LS, with 500 events per input file
convertToRaw -f 25 -l 25 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --configName /dev/CMSSW_16_0_0/GRun/V7 >"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = Trueprocess.GlobalTag.globaltag = cms.string( "150X_dataRun3_HLT_v1" )# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True# customization for the menufrom HLTrigger.Configuration.customizeHLTforCMSSW import *process = customizeHLTfor49799(process)process = customizeHLTfor49852(process)## just output the GPU vs CPU outputstreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMGPUvsCPUOutput')for foo in streamPaths: process.__delattr__(foo)@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMGPUvsCPU_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Possible extenstions
There are two more clients for which the unitTets could be activated in the future, namely:
 <!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-ecalcalib_dqm_sourceclient" command="runtest.sh ecalcalib_dqm_sourceclient-live_cfg.py" /> -->
<!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-hcalcalib_dqm_sourceclient" command="runtest.sh hcalcalib_dqm_sourceclient-live_cfg.py" /> -->

About
No description, website, or topics provided.
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
6 watching
Forks
5 forks
Report repository
Releases
Packages
Contributors

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Latest commit
History
50 Commits
Folders and files
NameName
Last commit message
Last commit date
run362321
run362321
run381594
run381594
run398183
run398183
run402360
run402360
README.md
README.md
Repository files navigation
DQM-Integration
Repository to store input files for the CMSSW DQM/Integration package.
Streamer Files
For the moment this repository is used to store streamer files that are needed as input for the unitTest.
Currently the unitTest that make use of input streamer files are:
  • the onlinebeammonitor_dqm_sourceclient_cfg.py client, it reads the streamDQM (prepared such that only the hltOnlineBeamSpot is present in the file) streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQM_pid1388480.dat
run381594/run381594_ls1000_streamDQM_pid1388480.jsn

  • the beamhlt_dqm_sourceclient-live_cfg.py client, it reads the streamDQMOnlineBeamspot streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.dat
run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.jsn

  • the ecalgpu_dqm_sourceclient-live_cfg.py, hcalgpu_dqm_sourceclient-live_cfg.py, pixelgpu_dqm_sourceclient-live_cfg.py and pfgpu_dqm_sourceclient-live_cfg.py read the streamDQMGPUvsCPU streamer files regenerated from run 398183 (from Run2025G pp run OMS link):

    run398183_ls0142_streamDQMGPUvsCPU_pid3119228.dat
run398183_ls0142_streamDQMGPUvsCPU_pid3119228.jsn

  • the sistrip_approx_dqm_sourceclient-live_cfg.py reads the streamDQM streamer files regenerated from run 362321 (from 2022 HI run, OMS link, though they have been re-HLT'ed, see for more details at CMSHLT-2884):

    run362321/run362321_ls0231_streamHIDQM_pid276864.dat
run362321/run362321_ls0231_streamHIDQM_pid276864.jsn

  • the scouting_dqm_sourceclient-live_cfg.py reads the streamDQMOnlineScouting streamer files generated from run 398183 (from Run2025G pp run, OMS link):

    run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.dat
run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.jsn

  • the ngt_dqm_sourceclient-live_cfg.py reads the streamDQMTestDataScouting streamer files generated from run 402360 (from Run2026B pp run, OMS link):

    run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.dat
run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.jsn 
Recipe to regenerate Streamer files (when streamer layout gets broken)
In repsonse to issue cms-sw/cmssw#45224, streamer files have been regenerated in a release that contains cms-sw/cmssw#44978 (in this case CMSSW_14_0_9_MULTIARCHS.
Recipe for streamDQM
This was done using the following script for the pp data:
#!/bin/bash -ex
RUNNUMBER=381594
LUMISECTION=1000
# cmsrel CMSSW_14_0_9_MULTIARCHS# cd CMSSW_14_0_9_MULTIARCHS/src# cmsenv# scram b
INPUTFILE=root://eoscms.cern.ch//store/express/Run2024E/ExpressPhysics/FEVT/Express-v1/000/381/594/00000/1e2c895f-a250-45be-a7ff-ee95e636a6e9.root
rm -rf run${RUNNUMBER}*# run on 300 events of LS 1000, with 300 events per input file
convertToRaw -f 300 -l 300 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --runNumber "${RUNNUMBER}">"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
Recipe for streamHIDQM
While the following script for the HIon data:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-03-03-2300# cd CMSSW_16_1_X_2026-03-03-2300/src# cmsenv# scram b# run 362321, LSs 231-232
RUNNUMBER=362321
LUMISECTION=231
INPUTFILE=root://eoscms.cern.ch//eos/cms/store/user/cmsbuild//store/hidata/HIRun2022A/HITestRaw0/RAW/v1/000/362/321/00000/f467ee64-fc64-47a6-9d8a-7ca73ebca2bd.root
HLTMENU=/dev/CMSSW_16_0_0/HIon/V31
rm -rf run${RUNNUMBER}*# run on 100 events of LS 231, with 100 events per input file
convertToRaw -f 100 -l 100 -r ${RUNNUMBER}:${LUMISECTION} -s rawDataRepacker -o . -- "${INPUTFILE}"
tmpfile=tmp.py
hltConfigFromDB --configName "${HLTMENU}">"${tmpfile}"
sed -i 's|process = cms.Process( "HLT" )|from Configuration.Eras.Era_Run3_cff import Run3\nprocess = cms.Process( "HLT", Run3 )|g'"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")#process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# override the GlobalTag, connection string and pfnPrefixfrom Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTagprocess.GlobalTag = customiseGlobalTag( process.GlobalTag, globaltag = "160X_dataRun3_HLT_v1", conditions = "L1Menu_CollisionsHeavyIons2025_v1_0_3_xml,L1TUtmTriggerMenuRcd,frontier://FrontierProd/CMS_CONDITIONS,,9999-12-31 23:59:59.000")# run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLTfrom HLTrigger.Configuration.CustomConfigs import L1REPACKprocess = L1REPACK(process, "uGT")# to run without any HLT prescalesdel process.PrescaleService# # to run using the same HLT prescales as used online in LS 231# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamHIDQM_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMOnlineScouting
The streamer files for the streamDQMOnlineScouting were prepared using the scouting specific menu:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-01-07-2300# cd CMSSW_16_1_X_2026-01-07-2300/src# cmsenv# scram b
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# LS 174 
INPUTFILES="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 5000 events of given LS, with 1000 event limits per input file
convertToRaw -l 5000 -f 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- ${INPUTFILES}
tmpfile=$(mktemp)
hltConfigFromDB --configName /users/jprendi/ScoutingOnlineDQM/Test0/HLT/V7 > dump.py
cat <<@EOF >> dump.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.GlobalTag.globaltag = cms.string('150X_dataRun3_HLT_v1')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True@EOF
edmConfigDump dump.py > hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMOnlineScouting_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMTestDataScouting
The streamer files for the streamDQMTestDataScouting were prepared using the following script:
#!/bin/bash -ex# cmsrel CMSSW_16_0_6_patch1# cd CMSSW_16_0_6_patch1/src# cmsenv
RUNNUMBER=402360
LUMISECTION=193
hltLabel=testAddTriggerEvents
hltLabel1="${hltLabel}_hlt1"
hltLabel2="${hltLabel}_hlt2"
INPUTFILE="root://eoscms.cern.ch//store/data/Run2026B/EphemeralHLTPhysics0/RAW/v1/000/402/360/00000/aa48b021-759e-4592-86e5-75eee46b88fc.root"
MENU=/online/collisions/2026/2e34/v1.2/HLT/V2
######################################### Helpers########################################run_cms() {
local cfg=$1local log=$2
bash -c "echo \$\$ > cmsrun.pid; exec cmsRun ${cfg} >& ${log}"
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"
}
prepare_output() {
local tag=$1# old_runXXX or new_runXXX
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*
mkdir -p prepared
local base="run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMTestDataScouting_pid${job_pid}"
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMTestDataScouting_pid${job_pid}.ini \
run${RUNNUMBER}/${base}.dat \
> prepared/${base}.dat
cp run${RUNNUMBER}/${base}.jsn \
prepared/${base}_prep.jsn
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"prepared/${base}_prep.jsn">"prepared/${base}.jsn"
rm -f prepared/${base}_prep.jsn
rm -rf run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared ${tag}
}
convert_input() {
convertToRaw -f 1000 -l 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
}
######################################### Setup########################################
rm -rf run${RUNNUMBER}*
hltConfigFromDB --configName ${MENU}>${hltLabel1}.py
cat <<@EOF >> ${hltLabel1}.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicestreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMTestDataScoutingOutput')for foo in streamPaths: process.__delattr__(foo)@EOF######################################### First run########################################
convert_input
run_cms "${hltLabel1}.py""${hltLabel1}.log"
prepare_output "${RUNNUMBER}"
Recipe for streamDQMGPUvsCPU
The streamer files for the streamDQMGPUVsCPU were prepared using the following script:
#!/bin/bash -ex
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# cmsrel CMSSW_16_1_X_2026-01-21-2300 # cd CMSSW_16_1_X_2026-01-21-2300/src/# cmsenv
INPUTFILE="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 500 events of LS, with 500 events per input file
convertToRaw -f 25 -l 25 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --configName /dev/CMSSW_16_0_0/GRun/V7 >"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = Trueprocess.GlobalTag.globaltag = cms.string( "150X_dataRun3_HLT_v1" )# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True# customization for the menufrom HLTrigger.Configuration.customizeHLTforCMSSW import *process = customizeHLTfor49799(process)process = customizeHLTfor49852(process)## just output the GPU vs CPU outputstreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMGPUvsCPUOutput')for foo in streamPaths: process.__delattr__(foo)@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMGPUvsCPU_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Possible extenstions
There are two more clients for which the unitTets could be activated in the future, namely:
 <!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-ecalcalib_dqm_sourceclient" command="runtest.sh ecalcalib_dqm_sourceclient-live_cfg.py" /> -->
<!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-hcalcalib_dqm_sourceclient" command="runtest.sh hcalcalib_dqm_sourceclient-live_cfg.py" /> -->

About
No description, website, or topics provided.
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
6 watching
Forks
5 forks
Report repository
Releases
Packages
Contributors

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Latest commit
History
50 Commits
Folders and files
NameName
Last commit message
Last commit date
run362321
run362321
run381594
run381594
run398183
run398183
run402360
run402360
README.md
README.md
Repository files navigation
DQM-Integration
Repository to store input files for the CMSSW DQM/Integration package.
Streamer Files
For the moment this repository is used to store streamer files that are needed as input for the unitTest.
Currently the unitTest that make use of input streamer files are:
  • the onlinebeammonitor_dqm_sourceclient_cfg.py client, it reads the streamDQM (prepared such that only the hltOnlineBeamSpot is present in the file) streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQM_pid1388480.dat
run381594/run381594_ls1000_streamDQM_pid1388480.jsn

  • the beamhlt_dqm_sourceclient-live_cfg.py client, it reads the streamDQMOnlineBeamspot streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.dat
run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.jsn

  • the ecalgpu_dqm_sourceclient-live_cfg.py, hcalgpu_dqm_sourceclient-live_cfg.py, pixelgpu_dqm_sourceclient-live_cfg.py and pfgpu_dqm_sourceclient-live_cfg.py read the streamDQMGPUvsCPU streamer files regenerated from run 398183 (from Run2025G pp run OMS link):

    run398183_ls0142_streamDQMGPUvsCPU_pid3119228.dat
run398183_ls0142_streamDQMGPUvsCPU_pid3119228.jsn

  • the sistrip_approx_dqm_sourceclient-live_cfg.py reads the streamDQM streamer files regenerated from run 362321 (from 2022 HI run, OMS link, though they have been re-HLT'ed, see for more details at CMSHLT-2884):

    run362321/run362321_ls0231_streamHIDQM_pid276864.dat
run362321/run362321_ls0231_streamHIDQM_pid276864.jsn

  • the scouting_dqm_sourceclient-live_cfg.py reads the streamDQMOnlineScouting streamer files generated from run 398183 (from Run2025G pp run, OMS link):

    run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.dat
run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.jsn

  • the ngt_dqm_sourceclient-live_cfg.py reads the streamDQMTestDataScouting streamer files generated from run 402360 (from Run2026B pp run, OMS link):

    run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.dat
run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.jsn 
Recipe to regenerate Streamer files (when streamer layout gets broken)
In repsonse to issue cms-sw/cmssw#45224, streamer files have been regenerated in a release that contains cms-sw/cmssw#44978 (in this case CMSSW_14_0_9_MULTIARCHS.
Recipe for streamDQM
This was done using the following script for the pp data:
#!/bin/bash -ex
RUNNUMBER=381594
LUMISECTION=1000
# cmsrel CMSSW_14_0_9_MULTIARCHS# cd CMSSW_14_0_9_MULTIARCHS/src# cmsenv# scram b
INPUTFILE=root://eoscms.cern.ch//store/express/Run2024E/ExpressPhysics/FEVT/Express-v1/000/381/594/00000/1e2c895f-a250-45be-a7ff-ee95e636a6e9.root
rm -rf run${RUNNUMBER}*# run on 300 events of LS 1000, with 300 events per input file
convertToRaw -f 300 -l 300 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --runNumber "${RUNNUMBER}">"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
Recipe for streamHIDQM
While the following script for the HIon data:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-03-03-2300# cd CMSSW_16_1_X_2026-03-03-2300/src# cmsenv# scram b# run 362321, LSs 231-232
RUNNUMBER=362321
LUMISECTION=231
INPUTFILE=root://eoscms.cern.ch//eos/cms/store/user/cmsbuild//store/hidata/HIRun2022A/HITestRaw0/RAW/v1/000/362/321/00000/f467ee64-fc64-47a6-9d8a-7ca73ebca2bd.root
HLTMENU=/dev/CMSSW_16_0_0/HIon/V31
rm -rf run${RUNNUMBER}*# run on 100 events of LS 231, with 100 events per input file
convertToRaw -f 100 -l 100 -r ${RUNNUMBER}:${LUMISECTION} -s rawDataRepacker -o . -- "${INPUTFILE}"
tmpfile=tmp.py
hltConfigFromDB --configName "${HLTMENU}">"${tmpfile}"
sed -i 's|process = cms.Process( "HLT" )|from Configuration.Eras.Era_Run3_cff import Run3\nprocess = cms.Process( "HLT", Run3 )|g'"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")#process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# override the GlobalTag, connection string and pfnPrefixfrom Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTagprocess.GlobalTag = customiseGlobalTag( process.GlobalTag, globaltag = "160X_dataRun3_HLT_v1", conditions = "L1Menu_CollisionsHeavyIons2025_v1_0_3_xml,L1TUtmTriggerMenuRcd,frontier://FrontierProd/CMS_CONDITIONS,,9999-12-31 23:59:59.000")# run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLTfrom HLTrigger.Configuration.CustomConfigs import L1REPACKprocess = L1REPACK(process, "uGT")# to run without any HLT prescalesdel process.PrescaleService# # to run using the same HLT prescales as used online in LS 231# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamHIDQM_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMOnlineScouting
The streamer files for the streamDQMOnlineScouting were prepared using the scouting specific menu:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-01-07-2300# cd CMSSW_16_1_X_2026-01-07-2300/src# cmsenv# scram b
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# LS 174 
INPUTFILES="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 5000 events of given LS, with 1000 event limits per input file
convertToRaw -l 5000 -f 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- ${INPUTFILES}
tmpfile=$(mktemp)
hltConfigFromDB --configName /users/jprendi/ScoutingOnlineDQM/Test0/HLT/V7 > dump.py
cat <<@EOF >> dump.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.GlobalTag.globaltag = cms.string('150X_dataRun3_HLT_v1')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True@EOF
edmConfigDump dump.py > hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMOnlineScouting_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMTestDataScouting
The streamer files for the streamDQMTestDataScouting were prepared using the following script:
#!/bin/bash -ex# cmsrel CMSSW_16_0_6_patch1# cd CMSSW_16_0_6_patch1/src# cmsenv
RUNNUMBER=402360
LUMISECTION=193
hltLabel=testAddTriggerEvents
hltLabel1="${hltLabel}_hlt1"
hltLabel2="${hltLabel}_hlt2"
INPUTFILE="root://eoscms.cern.ch//store/data/Run2026B/EphemeralHLTPhysics0/RAW/v1/000/402/360/00000/aa48b021-759e-4592-86e5-75eee46b88fc.root"
MENU=/online/collisions/2026/2e34/v1.2/HLT/V2
######################################### Helpers########################################run_cms() {
local cfg=$1local log=$2
bash -c "echo \$\$ > cmsrun.pid; exec cmsRun ${cfg} >& ${log}"
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"
}
prepare_output() {
local tag=$1# old_runXXX or new_runXXX
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*
mkdir -p prepared
local base="run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMTestDataScouting_pid${job_pid}"
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMTestDataScouting_pid${job_pid}.ini \
run${RUNNUMBER}/${base}.dat \
> prepared/${base}.dat
cp run${RUNNUMBER}/${base}.jsn \
prepared/${base}_prep.jsn
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"prepared/${base}_prep.jsn">"prepared/${base}.jsn"
rm -f prepared/${base}_prep.jsn
rm -rf run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared ${tag}
}
convert_input() {
convertToRaw -f 1000 -l 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
}
######################################### Setup########################################
rm -rf run${RUNNUMBER}*
hltConfigFromDB --configName ${MENU}>${hltLabel1}.py
cat <<@EOF >> ${hltLabel1}.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicestreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMTestDataScoutingOutput')for foo in streamPaths: process.__delattr__(foo)@EOF######################################### First run########################################
convert_input
run_cms "${hltLabel1}.py""${hltLabel1}.log"
prepare_output "${RUNNUMBER}"
Recipe for streamDQMGPUvsCPU
The streamer files for the streamDQMGPUVsCPU were prepared using the following script:
#!/bin/bash -ex
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# cmsrel CMSSW_16_1_X_2026-01-21-2300 # cd CMSSW_16_1_X_2026-01-21-2300/src/# cmsenv
INPUTFILE="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 500 events of LS, with 500 events per input file
convertToRaw -f 25 -l 25 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --configName /dev/CMSSW_16_0_0/GRun/V7 >"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = Trueprocess.GlobalTag.globaltag = cms.string( "150X_dataRun3_HLT_v1" )# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True# customization for the menufrom HLTrigger.Configuration.customizeHLTforCMSSW import *process = customizeHLTfor49799(process)process = customizeHLTfor49852(process)## just output the GPU vs CPU outputstreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMGPUvsCPUOutput')for foo in streamPaths: process.__delattr__(foo)@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMGPUvsCPU_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Possible extenstions
There are two more clients for which the unitTets could be activated in the future, namely:
 <!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-ecalcalib_dqm_sourceclient" command="runtest.sh ecalcalib_dqm_sourceclient-live_cfg.py" /> -->
<!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-hcalcalib_dqm_sourceclient" command="runtest.sh hcalcalib_dqm_sourceclient-live_cfg.py" /> -->

About
No description, website, or topics provided.
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
6 watching
Forks
5 forks
Report repository
Releases
Packages
Contributors

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Latest commit
History
50 Commits
Folders and files
NameName
Last commit message
Last commit date
run362321
run362321
run381594
run381594
run398183
run398183
run402360
run402360
README.md
README.md
Repository files navigation
DQM-Integration
Repository to store input files for the CMSSW DQM/Integration package.
Streamer Files
For the moment this repository is used to store streamer files that are needed as input for the unitTest.
Currently the unitTest that make use of input streamer files are:
  • the onlinebeammonitor_dqm_sourceclient_cfg.py client, it reads the streamDQM (prepared such that only the hltOnlineBeamSpot is present in the file) streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQM_pid1388480.dat
run381594/run381594_ls1000_streamDQM_pid1388480.jsn

  • the beamhlt_dqm_sourceclient-live_cfg.py client, it reads the streamDQMOnlineBeamspot streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.dat
run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.jsn

  • the ecalgpu_dqm_sourceclient-live_cfg.py, hcalgpu_dqm_sourceclient-live_cfg.py, pixelgpu_dqm_sourceclient-live_cfg.py and pfgpu_dqm_sourceclient-live_cfg.py read the streamDQMGPUvsCPU streamer files regenerated from run 398183 (from Run2025G pp run OMS link):

    run398183_ls0142_streamDQMGPUvsCPU_pid3119228.dat
run398183_ls0142_streamDQMGPUvsCPU_pid3119228.jsn

  • the sistrip_approx_dqm_sourceclient-live_cfg.py reads the streamDQM streamer files regenerated from run 362321 (from 2022 HI run, OMS link, though they have been re-HLT'ed, see for more details at CMSHLT-2884):

    run362321/run362321_ls0231_streamHIDQM_pid276864.dat
run362321/run362321_ls0231_streamHIDQM_pid276864.jsn

  • the scouting_dqm_sourceclient-live_cfg.py reads the streamDQMOnlineScouting streamer files generated from run 398183 (from Run2025G pp run, OMS link):

    run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.dat
run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.jsn

  • the ngt_dqm_sourceclient-live_cfg.py reads the streamDQMTestDataScouting streamer files generated from run 402360 (from Run2026B pp run, OMS link):

    run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.dat
run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.jsn 
Recipe to regenerate Streamer files (when streamer layout gets broken)
In repsonse to issue cms-sw/cmssw#45224, streamer files have been regenerated in a release that contains cms-sw/cmssw#44978 (in this case CMSSW_14_0_9_MULTIARCHS.
Recipe for streamDQM
This was done using the following script for the pp data:
#!/bin/bash -ex
RUNNUMBER=381594
LUMISECTION=1000
# cmsrel CMSSW_14_0_9_MULTIARCHS# cd CMSSW_14_0_9_MULTIARCHS/src# cmsenv# scram b
INPUTFILE=root://eoscms.cern.ch//store/express/Run2024E/ExpressPhysics/FEVT/Express-v1/000/381/594/00000/1e2c895f-a250-45be-a7ff-ee95e636a6e9.root
rm -rf run${RUNNUMBER}*# run on 300 events of LS 1000, with 300 events per input file
convertToRaw -f 300 -l 300 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --runNumber "${RUNNUMBER}">"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
Recipe for streamHIDQM
While the following script for the HIon data:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-03-03-2300# cd CMSSW_16_1_X_2026-03-03-2300/src# cmsenv# scram b# run 362321, LSs 231-232
RUNNUMBER=362321
LUMISECTION=231
INPUTFILE=root://eoscms.cern.ch//eos/cms/store/user/cmsbuild//store/hidata/HIRun2022A/HITestRaw0/RAW/v1/000/362/321/00000/f467ee64-fc64-47a6-9d8a-7ca73ebca2bd.root
HLTMENU=/dev/CMSSW_16_0_0/HIon/V31
rm -rf run${RUNNUMBER}*# run on 100 events of LS 231, with 100 events per input file
convertToRaw -f 100 -l 100 -r ${RUNNUMBER}:${LUMISECTION} -s rawDataRepacker -o . -- "${INPUTFILE}"
tmpfile=tmp.py
hltConfigFromDB --configName "${HLTMENU}">"${tmpfile}"
sed -i 's|process = cms.Process( "HLT" )|from Configuration.Eras.Era_Run3_cff import Run3\nprocess = cms.Process( "HLT", Run3 )|g'"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")#process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# override the GlobalTag, connection string and pfnPrefixfrom Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTagprocess.GlobalTag = customiseGlobalTag( process.GlobalTag, globaltag = "160X_dataRun3_HLT_v1", conditions = "L1Menu_CollisionsHeavyIons2025_v1_0_3_xml,L1TUtmTriggerMenuRcd,frontier://FrontierProd/CMS_CONDITIONS,,9999-12-31 23:59:59.000")# run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLTfrom HLTrigger.Configuration.CustomConfigs import L1REPACKprocess = L1REPACK(process, "uGT")# to run without any HLT prescalesdel process.PrescaleService# # to run using the same HLT prescales as used online in LS 231# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamHIDQM_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMOnlineScouting
The streamer files for the streamDQMOnlineScouting were prepared using the scouting specific menu:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-01-07-2300# cd CMSSW_16_1_X_2026-01-07-2300/src# cmsenv# scram b
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# LS 174 
INPUTFILES="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 5000 events of given LS, with 1000 event limits per input file
convertToRaw -l 5000 -f 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- ${INPUTFILES}
tmpfile=$(mktemp)
hltConfigFromDB --configName /users/jprendi/ScoutingOnlineDQM/Test0/HLT/V7 > dump.py
cat <<@EOF >> dump.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.GlobalTag.globaltag = cms.string('150X_dataRun3_HLT_v1')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True@EOF
edmConfigDump dump.py > hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMOnlineScouting_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMTestDataScouting
The streamer files for the streamDQMTestDataScouting were prepared using the following script:
#!/bin/bash -ex# cmsrel CMSSW_16_0_6_patch1# cd CMSSW_16_0_6_patch1/src# cmsenv
RUNNUMBER=402360
LUMISECTION=193
hltLabel=testAddTriggerEvents
hltLabel1="${hltLabel}_hlt1"
hltLabel2="${hltLabel}_hlt2"
INPUTFILE="root://eoscms.cern.ch//store/data/Run2026B/EphemeralHLTPhysics0/RAW/v1/000/402/360/00000/aa48b021-759e-4592-86e5-75eee46b88fc.root"
MENU=/online/collisions/2026/2e34/v1.2/HLT/V2
######################################### Helpers########################################run_cms() {
local cfg=$1local log=$2
bash -c "echo \$\$ > cmsrun.pid; exec cmsRun ${cfg} >& ${log}"
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"
}
prepare_output() {
local tag=$1# old_runXXX or new_runXXX
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*
mkdir -p prepared
local base="run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMTestDataScouting_pid${job_pid}"
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMTestDataScouting_pid${job_pid}.ini \
run${RUNNUMBER}/${base}.dat \
> prepared/${base}.dat
cp run${RUNNUMBER}/${base}.jsn \
prepared/${base}_prep.jsn
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"prepared/${base}_prep.jsn">"prepared/${base}.jsn"
rm -f prepared/${base}_prep.jsn
rm -rf run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared ${tag}
}
convert_input() {
convertToRaw -f 1000 -l 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
}
######################################### Setup########################################
rm -rf run${RUNNUMBER}*
hltConfigFromDB --configName ${MENU}>${hltLabel1}.py
cat <<@EOF >> ${hltLabel1}.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicestreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMTestDataScoutingOutput')for foo in streamPaths: process.__delattr__(foo)@EOF######################################### First run########################################
convert_input
run_cms "${hltLabel1}.py""${hltLabel1}.log"
prepare_output "${RUNNUMBER}"
Recipe for streamDQMGPUvsCPU
The streamer files for the streamDQMGPUVsCPU were prepared using the following script:
#!/bin/bash -ex
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# cmsrel CMSSW_16_1_X_2026-01-21-2300 # cd CMSSW_16_1_X_2026-01-21-2300/src/# cmsenv
INPUTFILE="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 500 events of LS, with 500 events per input file
convertToRaw -f 25 -l 25 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --configName /dev/CMSSW_16_0_0/GRun/V7 >"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = Trueprocess.GlobalTag.globaltag = cms.string( "150X_dataRun3_HLT_v1" )# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True# customization for the menufrom HLTrigger.Configuration.customizeHLTforCMSSW import *process = customizeHLTfor49799(process)process = customizeHLTfor49852(process)## just output the GPU vs CPU outputstreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMGPUvsCPUOutput')for foo in streamPaths: process.__delattr__(foo)@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMGPUvsCPU_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Possible extenstions
There are two more clients for which the unitTets could be activated in the future, namely:
 <!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-ecalcalib_dqm_sourceclient" command="runtest.sh ecalcalib_dqm_sourceclient-live_cfg.py" /> -->
<!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-hcalcalib_dqm_sourceclient" command="runtest.sh hcalcalib_dqm_sourceclient-live_cfg.py" /> -->

About
No description, website, or topics provided.
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
6 watching
Forks
5 forks
Report repository
Releases
Packages
Contributors

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Latest commit
History
50 Commits
Folders and files
NameName
Last commit message
Last commit date
run362321
run362321
run381594
run381594
run398183
run398183
run402360
run402360
README.md
README.md
Repository files navigation
DQM-Integration
Repository to store input files for the CMSSW DQM/Integration package.
Streamer Files
For the moment this repository is used to store streamer files that are needed as input for the unitTest.
Currently the unitTest that make use of input streamer files are:
  • the onlinebeammonitor_dqm_sourceclient_cfg.py client, it reads the streamDQM (prepared such that only the hltOnlineBeamSpot is present in the file) streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQM_pid1388480.dat
run381594/run381594_ls1000_streamDQM_pid1388480.jsn

  • the beamhlt_dqm_sourceclient-live_cfg.py client, it reads the streamDQMOnlineBeamspot streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.dat
run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.jsn

  • the ecalgpu_dqm_sourceclient-live_cfg.py, hcalgpu_dqm_sourceclient-live_cfg.py, pixelgpu_dqm_sourceclient-live_cfg.py and pfgpu_dqm_sourceclient-live_cfg.py read the streamDQMGPUvsCPU streamer files regenerated from run 398183 (from Run2025G pp run OMS link):

    run398183_ls0142_streamDQMGPUvsCPU_pid3119228.dat
run398183_ls0142_streamDQMGPUvsCPU_pid3119228.jsn

  • the sistrip_approx_dqm_sourceclient-live_cfg.py reads the streamDQM streamer files regenerated from run 362321 (from 2022 HI run, OMS link, though they have been re-HLT'ed, see for more details at CMSHLT-2884):

    run362321/run362321_ls0231_streamHIDQM_pid276864.dat
run362321/run362321_ls0231_streamHIDQM_pid276864.jsn

  • the scouting_dqm_sourceclient-live_cfg.py reads the streamDQMOnlineScouting streamer files generated from run 398183 (from Run2025G pp run, OMS link):

    run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.dat
run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.jsn

  • the ngt_dqm_sourceclient-live_cfg.py reads the streamDQMTestDataScouting streamer files generated from run 402360 (from Run2026B pp run, OMS link):

    run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.dat
run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.jsn 
Recipe to regenerate Streamer files (when streamer layout gets broken)
In repsonse to issue cms-sw/cmssw#45224, streamer files have been regenerated in a release that contains cms-sw/cmssw#44978 (in this case CMSSW_14_0_9_MULTIARCHS.
Recipe for streamDQM
This was done using the following script for the pp data:
#!/bin/bash -ex
RUNNUMBER=381594
LUMISECTION=1000
# cmsrel CMSSW_14_0_9_MULTIARCHS# cd CMSSW_14_0_9_MULTIARCHS/src# cmsenv# scram b
INPUTFILE=root://eoscms.cern.ch//store/express/Run2024E/ExpressPhysics/FEVT/Express-v1/000/381/594/00000/1e2c895f-a250-45be-a7ff-ee95e636a6e9.root
rm -rf run${RUNNUMBER}*# run on 300 events of LS 1000, with 300 events per input file
convertToRaw -f 300 -l 300 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --runNumber "${RUNNUMBER}">"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
Recipe for streamHIDQM
While the following script for the HIon data:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-03-03-2300# cd CMSSW_16_1_X_2026-03-03-2300/src# cmsenv# scram b# run 362321, LSs 231-232
RUNNUMBER=362321
LUMISECTION=231
INPUTFILE=root://eoscms.cern.ch//eos/cms/store/user/cmsbuild//store/hidata/HIRun2022A/HITestRaw0/RAW/v1/000/362/321/00000/f467ee64-fc64-47a6-9d8a-7ca73ebca2bd.root
HLTMENU=/dev/CMSSW_16_0_0/HIon/V31
rm -rf run${RUNNUMBER}*# run on 100 events of LS 231, with 100 events per input file
convertToRaw -f 100 -l 100 -r ${RUNNUMBER}:${LUMISECTION} -s rawDataRepacker -o . -- "${INPUTFILE}"
tmpfile=tmp.py
hltConfigFromDB --configName "${HLTMENU}">"${tmpfile}"
sed -i 's|process = cms.Process( "HLT" )|from Configuration.Eras.Era_Run3_cff import Run3\nprocess = cms.Process( "HLT", Run3 )|g'"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")#process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# override the GlobalTag, connection string and pfnPrefixfrom Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTagprocess.GlobalTag = customiseGlobalTag( process.GlobalTag, globaltag = "160X_dataRun3_HLT_v1", conditions = "L1Menu_CollisionsHeavyIons2025_v1_0_3_xml,L1TUtmTriggerMenuRcd,frontier://FrontierProd/CMS_CONDITIONS,,9999-12-31 23:59:59.000")# run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLTfrom HLTrigger.Configuration.CustomConfigs import L1REPACKprocess = L1REPACK(process, "uGT")# to run without any HLT prescalesdel process.PrescaleService# # to run using the same HLT prescales as used online in LS 231# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamHIDQM_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMOnlineScouting
The streamer files for the streamDQMOnlineScouting were prepared using the scouting specific menu:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-01-07-2300# cd CMSSW_16_1_X_2026-01-07-2300/src# cmsenv# scram b
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# LS 174 
INPUTFILES="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 5000 events of given LS, with 1000 event limits per input file
convertToRaw -l 5000 -f 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- ${INPUTFILES}
tmpfile=$(mktemp)
hltConfigFromDB --configName /users/jprendi/ScoutingOnlineDQM/Test0/HLT/V7 > dump.py
cat <<@EOF >> dump.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.GlobalTag.globaltag = cms.string('150X_dataRun3_HLT_v1')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True@EOF
edmConfigDump dump.py > hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMOnlineScouting_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMTestDataScouting
The streamer files for the streamDQMTestDataScouting were prepared using the following script:
#!/bin/bash -ex# cmsrel CMSSW_16_0_6_patch1# cd CMSSW_16_0_6_patch1/src# cmsenv
RUNNUMBER=402360
LUMISECTION=193
hltLabel=testAddTriggerEvents
hltLabel1="${hltLabel}_hlt1"
hltLabel2="${hltLabel}_hlt2"
INPUTFILE="root://eoscms.cern.ch//store/data/Run2026B/EphemeralHLTPhysics0/RAW/v1/000/402/360/00000/aa48b021-759e-4592-86e5-75eee46b88fc.root"
MENU=/online/collisions/2026/2e34/v1.2/HLT/V2
######################################### Helpers########################################run_cms() {
local cfg=$1local log=$2
bash -c "echo \$\$ > cmsrun.pid; exec cmsRun ${cfg} >& ${log}"
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"
}
prepare_output() {
local tag=$1# old_runXXX or new_runXXX
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*
mkdir -p prepared
local base="run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMTestDataScouting_pid${job_pid}"
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMTestDataScouting_pid${job_pid}.ini \
run${RUNNUMBER}/${base}.dat \
> prepared/${base}.dat
cp run${RUNNUMBER}/${base}.jsn \
prepared/${base}_prep.jsn
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"prepared/${base}_prep.jsn">"prepared/${base}.jsn"
rm -f prepared/${base}_prep.jsn
rm -rf run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared ${tag}
}
convert_input() {
convertToRaw -f 1000 -l 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
}
######################################### Setup########################################
rm -rf run${RUNNUMBER}*
hltConfigFromDB --configName ${MENU}>${hltLabel1}.py
cat <<@EOF >> ${hltLabel1}.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicestreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMTestDataScoutingOutput')for foo in streamPaths: process.__delattr__(foo)@EOF######################################### First run########################################
convert_input
run_cms "${hltLabel1}.py""${hltLabel1}.log"
prepare_output "${RUNNUMBER}"
Recipe for streamDQMGPUvsCPU
The streamer files for the streamDQMGPUVsCPU were prepared using the following script:
#!/bin/bash -ex
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# cmsrel CMSSW_16_1_X_2026-01-21-2300 # cd CMSSW_16_1_X_2026-01-21-2300/src/# cmsenv
INPUTFILE="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 500 events of LS, with 500 events per input file
convertToRaw -f 25 -l 25 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --configName /dev/CMSSW_16_0_0/GRun/V7 >"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = Trueprocess.GlobalTag.globaltag = cms.string( "150X_dataRun3_HLT_v1" )# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True# customization for the menufrom HLTrigger.Configuration.customizeHLTforCMSSW import *process = customizeHLTfor49799(process)process = customizeHLTfor49852(process)## just output the GPU vs CPU outputstreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMGPUvsCPUOutput')for foo in streamPaths: process.__delattr__(foo)@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMGPUvsCPU_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Possible extenstions
There are two more clients for which the unitTets could be activated in the future, namely:
 <!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-ecalcalib_dqm_sourceclient" command="runtest.sh ecalcalib_dqm_sourceclient-live_cfg.py" /> -->
<!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-hcalcalib_dqm_sourceclient" command="runtest.sh hcalcalib_dqm_sourceclient-live_cfg.py" /> -->

About
No description, website, or topics provided.
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
6 watching
Forks
5 forks
Report repository
Releases
Packages
Contributors

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Latest commit
History
50 Commits
Folders and files
NameName
Last commit message
Last commit date
run362321
run362321
run381594
run381594
run398183
run398183
run402360
run402360
README.md
README.md
Repository files navigation
DQM-Integration
Repository to store input files for the CMSSW DQM/Integration package.
Streamer Files
For the moment this repository is used to store streamer files that are needed as input for the unitTest.
Currently the unitTest that make use of input streamer files are:
  • the onlinebeammonitor_dqm_sourceclient_cfg.py client, it reads the streamDQM (prepared such that only the hltOnlineBeamSpot is present in the file) streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQM_pid1388480.dat
run381594/run381594_ls1000_streamDQM_pid1388480.jsn

  • the beamhlt_dqm_sourceclient-live_cfg.py client, it reads the streamDQMOnlineBeamspot streamer files regenerated from run 381594 (from 2024E pp run OMS link):

    run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.dat
run381594/run381594_ls1000_streamDQMOnlineBeamspot_pid1752643.jsn

  • the ecalgpu_dqm_sourceclient-live_cfg.py, hcalgpu_dqm_sourceclient-live_cfg.py, pixelgpu_dqm_sourceclient-live_cfg.py and pfgpu_dqm_sourceclient-live_cfg.py read the streamDQMGPUvsCPU streamer files regenerated from run 398183 (from Run2025G pp run OMS link):

    run398183_ls0142_streamDQMGPUvsCPU_pid3119228.dat
run398183_ls0142_streamDQMGPUvsCPU_pid3119228.jsn

  • the sistrip_approx_dqm_sourceclient-live_cfg.py reads the streamDQM streamer files regenerated from run 362321 (from 2022 HI run, OMS link, though they have been re-HLT'ed, see for more details at CMSHLT-2884):

    run362321/run362321_ls0231_streamHIDQM_pid276864.dat
run362321/run362321_ls0231_streamHIDQM_pid276864.jsn

  • the scouting_dqm_sourceclient-live_cfg.py reads the streamDQMOnlineScouting streamer files generated from run 398183 (from Run2025G pp run, OMS link):

    run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.dat
run398183/run398183_ls0142_streamDQMOnlineScouting_pid2737183.jsn

  • the ngt_dqm_sourceclient-live_cfg.py reads the streamDQMTestDataScouting streamer files generated from run 402360 (from Run2026B pp run, OMS link):

    run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.dat
run402360/run402360_ls0193_streamDQMTestDataScouting_pid285513.jsn 
Recipe to regenerate Streamer files (when streamer layout gets broken)
In repsonse to issue cms-sw/cmssw#45224, streamer files have been regenerated in a release that contains cms-sw/cmssw#44978 (in this case CMSSW_14_0_9_MULTIARCHS.
Recipe for streamDQM
This was done using the following script for the pp data:
#!/bin/bash -ex
RUNNUMBER=381594
LUMISECTION=1000
# cmsrel CMSSW_14_0_9_MULTIARCHS# cd CMSSW_14_0_9_MULTIARCHS/src# cmsenv# scram b
INPUTFILE=root://eoscms.cern.ch//store/express/Run2024E/ExpressPhysics/FEVT/Express-v1/000/381/594/00000/1e2c895f-a250-45be-a7ff-ee95e636a6e9.root
rm -rf run${RUNNUMBER}*# run on 300 events of LS 1000, with 300 events per input file
convertToRaw -f 300 -l 300 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --runNumber "${RUNNUMBER}">"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
Recipe for streamHIDQM
While the following script for the HIon data:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-03-03-2300# cd CMSSW_16_1_X_2026-03-03-2300/src# cmsenv# scram b# run 362321, LSs 231-232
RUNNUMBER=362321
LUMISECTION=231
INPUTFILE=root://eoscms.cern.ch//eos/cms/store/user/cmsbuild//store/hidata/HIRun2022A/HITestRaw0/RAW/v1/000/362/321/00000/f467ee64-fc64-47a6-9d8a-7ca73ebca2bd.root
HLTMENU=/dev/CMSSW_16_0_0/HIon/V31
rm -rf run${RUNNUMBER}*# run on 100 events of LS 231, with 100 events per input file
convertToRaw -f 100 -l 100 -r ${RUNNUMBER}:${LUMISECTION} -s rawDataRepacker -o . -- "${INPUTFILE}"
tmpfile=tmp.py
hltConfigFromDB --configName "${HLTMENU}">"${tmpfile}"
sed -i 's|process = cms.Process( "HLT" )|from Configuration.Eras.Era_Run3_cff import Run3\nprocess = cms.Process( "HLT", Run3 )|g'"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")#process.hltOnlineBeamSpotESProducer.timeThreshold = int(1e6)# override the GlobalTag, connection string and pfnPrefixfrom Configuration.AlCa.GlobalTag import GlobalTag as customiseGlobalTagprocess.GlobalTag = customiseGlobalTag( process.GlobalTag, globaltag = "160X_dataRun3_HLT_v1", conditions = "L1Menu_CollisionsHeavyIons2025_v1_0_3_xml,L1TUtmTriggerMenuRcd,frontier://FrontierProd/CMS_CONDITIONS,,9999-12-31 23:59:59.000")# run the Full L1T emulator, then repack the data into a new RAW collection, to be used by the HLTfrom HLTrigger.Configuration.CustomConfigs import L1REPACKprocess = L1REPACK(process, "uGT")# to run without any HLT prescalesdel process.PrescaleService# # to run using the same HLT prescales as used online in LS 231# process.PrescaleService.forceDefault = True@EOF
edmConfigDump "${tmpfile}"> hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamHIDQM_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamHIDQM_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMOnlineScouting
The streamer files for the streamDQMOnlineScouting were prepared using the scouting specific menu:
#!/bin/bash -ex# cmsrel CMSSW_16_1_X_2026-01-07-2300# cd CMSSW_16_1_X_2026-01-07-2300/src# cmsenv# scram b
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# LS 174 
INPUTFILES="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 5000 events of given LS, with 1000 event limits per input file
convertToRaw -l 5000 -f 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- ${INPUTFILES}
tmpfile=$(mktemp)
hltConfigFromDB --configName /users/jprendi/ScoutingOnlineDQM/Test0/HLT/V7 > dump.py
cat <<@EOF >> dump.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.GlobalTag.globaltag = cms.string('150X_dataRun3_HLT_v1')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = True@EOF
edmConfigDump dump.py > hlt.py
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMOnlineScouting_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMOnlineScouting_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Recipe for streamDQMTestDataScouting
The streamer files for the streamDQMTestDataScouting were prepared using the following script:
#!/bin/bash -ex# cmsrel CMSSW_16_0_6_patch1# cd CMSSW_16_0_6_patch1/src# cmsenv
RUNNUMBER=402360
LUMISECTION=193
hltLabel=testAddTriggerEvents
hltLabel1="${hltLabel}_hlt1"
hltLabel2="${hltLabel}_hlt2"
INPUTFILE="root://eoscms.cern.ch//store/data/Run2026B/EphemeralHLTPhysics0/RAW/v1/000/402/360/00000/aa48b021-759e-4592-86e5-75eee46b88fc.root"
MENU=/online/collisions/2026/2e34/v1.2/HLT/V2
######################################### Helpers########################################run_cms() {
local cfg=$1local log=$2
bash -c "echo \$\$ > cmsrun.pid; exec cmsRun ${cfg} >& ${log}"
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"
}
prepare_output() {
local tag=$1# old_runXXX or new_runXXX
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*
mkdir -p prepared
local base="run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMTestDataScouting_pid${job_pid}"
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMTestDataScouting_pid${job_pid}.ini \
run${RUNNUMBER}/${base}.dat \
> prepared/${base}.dat
cp run${RUNNUMBER}/${base}.jsn \
prepared/${base}_prep.jsn
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"prepared/${base}_prep.jsn">"prepared/${base}.jsn"
rm -f prepared/${base}_prep.jsn
rm -rf run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared ${tag}
}
convert_input() {
convertToRaw -f 1000 -l 1000 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
}
######################################### Setup########################################
rm -rf run${RUNNUMBER}*
hltConfigFromDB --configName ${MENU}>${hltLabel1}.py
cat <<@EOF >> ${hltLabel1}.pyprocess.load("run${RUNNUMBER}_cff")del process.PrescaleServicestreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMTestDataScoutingOutput')for foo in streamPaths: process.__delattr__(foo)@EOF######################################### First run########################################
convert_input
run_cms "${hltLabel1}.py""${hltLabel1}.log"
prepare_output "${RUNNUMBER}"
Recipe for streamDQMGPUvsCPU
The streamer files for the streamDQMGPUVsCPU were prepared using the following script:
#!/bin/bash -ex
RUNNUMBER=398183 # 2025 EphemeralHLTPhysics
LUMISECTION=142
# cmsrel CMSSW_16_1_X_2026-01-21-2300 # cd CMSSW_16_1_X_2026-01-21-2300/src/# cmsenv
INPUTFILE="root://eoscms.cern.ch//store/data/Run2025G/EphemeralHLTPhysics0/RAW/v1/000/398/183/00000/002bbd0c-b9ed-4758-b7a6-e2e13149ca34.root"
rm -rf run${RUNNUMBER}*# run on 500 events of LS, with 500 events per input file
convertToRaw -f 25 -l 25 -r ${RUNNUMBER}:${LUMISECTION} -o . -- "${INPUTFILE}"
tmpfile=$(mktemp)
hltConfigFromDB --configName /dev/CMSSW_16_0_0/GRun/V7 >"${tmpfile}"
cat <<@EOF >> "${tmpfile}"process.load("run${RUNNUMBER}_cff")# to run without any HLT prescalesdel process.PrescaleServicedel process.MessageLoggerprocess.load('FWCore.MessageLogger.MessageLogger_cfi')process.options.numberOfThreads = 32process.options.numberOfStreams = 32process.options.wantSummary = Trueprocess.GlobalTag.globaltag = cms.string( "150X_dataRun3_HLT_v1" )# # to run using the same HLT prescales as used online in LS 1000# process.PrescaleService.forceDefault = True# customization for the menufrom HLTrigger.Configuration.customizeHLTforCMSSW import *process = customizeHLTfor49799(process)process = customizeHLTfor49852(process)## just output the GPU vs CPU outputstreamPaths = [foo for foo in process.endpaths_() if foo.endswith('Output')]streamPaths.remove('DQMGPUvsCPUOutput')for foo in streamPaths: process.__delattr__(foo)@EOF
edmConfigDump "${tmpfile}"> hlt.py
cmsRun hlt.py &> hlt.log
bash -c 'echo $$ > cmsrun.pid; exec cmsRun hlt.py &> hlt.log'
job_pid=$(cat cmsrun.pid)echo"cmsRun is running with PID: $job_pid"# remove input files to save space
rm -f run${RUNNUMBER}/run${RUNNUMBER}_ls0*_index*.*# prepare the files by concatenating the .ini and .dat files
mkdir -p prepared
cat run${RUNNUMBER}/run${RUNNUMBER}_ls0000_streamDQMGPUvsCPU_pid${job_pid}.ini run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat > prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.dat
cp run${RUNNUMBER}/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn
# now remove the extra 0
input="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}_prep.jsn"
output="prepared/run${RUNNUMBER}_ls0${LUMISECTION}_streamDQMGPUvsCPU_pid${job_pid}.jsn"
jq ' .data as $d | .data = ( reduce range(0; $d|length) as $i ([]; if ($i > 0 and .[-1] == "0" and $d[$i] == "0") then . else . + [$d[$i]] end ) )'"$input">"$output"
rm -fr $input
rm -fr run${RUNNUMBER}* hlt.* cmsrun.pid dump.py __pycache__
mv prepared run${RUNNUMBER}
Possible extenstions
There are two more clients for which the unitTets could be activated in the future, namely:
 <!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-ecalcalib_dqm_sourceclient" command="runtest.sh ecalcalib_dqm_sourceclient-live_cfg.py" /> -->
<!-- streamDQMCalibration is required -->
<!-- <test name="TestDQMOnlineClient-hcalcalib_dqm_sourceclient" command="runtest.sh hcalcalib_dqm_sourceclient-live_cfg.py" /> -->

About
No description, website, or topics provided.
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
6 watching
Forks
5 forks
Report repository
Releases
Packages
Contributors