The Data Quality Monitoring software is a central tool in the CMS experiment. It is used in the following key environments: (i) Online, for real-time detector monitoring; (ii) Offline, for the prompt-offline-feedback and final fine-grained data quality analysis and certification; (iii) Validation of all the reconstruction software production releases; (iv) Validation in Monte Carlo productions. Though the basic structure of the Run1 DQM system remains the same for Run2, between the Run1 and Run2 periods, the DQM system underwent substantial upgrades in many areas, not only to adapt to the surrounding infrastructure changes, but also to provide improvements to meet the growing needs of the collaboration with an emphasis on more sophisticated methods for evaluating data quality. We need to cope with the higher-energy and -luminosity proton-proton collision data, as well as the data from various special runs, such as Heavy Ion runs. In this contribution, we will describe the current DQM software, structure and workflow in the different environments. We then discuss the performance and our experiences with the DQM system in Run2. The main technical challenges which we have encountered and the solutions adopted during Run2 will also be discussed, including efficient use of memory in multithreading environments. Finally, we present the prospect of a future DQM upgrade with emphasis on functionality and long-term robustness for LHC Run3.

/pdf/the-data-quality-monitoring-software-for-the-cms-experiment-1ekjr1y195.pdf

The data quality monitoring software for the CMS experiment at the LHC: past, present and future

The Data Quality Monitoring Software for the CMS experiment at the LHC: past, present and future

The Data Quality Monitoring Software proved to be a central tool in the Compact Muon Solenoid experiment. Its flexibility allowed its integration in several environments: online, for real-time detector monitoring; offline, for the final, fine-grained data certification. The usage of the Data Quality Monitoring software in the different environments and its integration in the Compact Muon Solenoid reconstruction software framework and in all production work-flows are presented. The main technical challenges and the adopted solutions to them will be also discussed with emphasis on functionality, longterm robustness and performance.

/pdf/the-cms-data-quality-monitoring-software-experience-and-ilwh5sm133.pdf

The CMS Data Quality Monitoring Software: Experience and future improvements

The Compact Muon Solenoid (CMS) is one of the experiments at the CERN Large Hadron Collider (LHC). CMS is a generalpurpose detector able to record particle collisions up to 40 million times each second. 40 MHz rate results in approx 40 Tb/s, however journey of recorded CMS data is long and only small fraction makes to the final step physics analysis. Chain of automated and semi-automated processes filter, reconstruct, calibrate and verify data before it can be used for physics analysis. Data certification process starts with data aggregation in histograms, plots and various statistical quantities, and finishes with manual data quality assessment and decision. During the last step multiple experts review and evaluate (certify) data as being “good” or “bad”. This results in high manpower demand and occasional involuntary human errors. Data certification process consists of online (to determine possible problems during the data taking, only the fraction of data is used) and offline (not constrained in time full data set analysis including physics data reconstruction). Main goal of this research is to investigate the applicability and provide the comparison of various supervised machine learning techniques for offline data certification process automation. Removal or significant reduction of manual labor and exclusion of human errors from the CMS data certification process are the main motivations of this research.

/pdf/comparison-of-supervised-machine-learning-techniques-for-14taai389s.pdf

Comparison of Supervised Machine Learning Techniques for CERN CMS Offline Data Certification.

CMS data preparation for Run II

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Atanas Batinkov

Author Tools

Chat about Author

Papers

The CMS Data Quality Monitoring Software: Experience and future improvements