Event Data Management Overview¶
Offline data processing algorithms for HERD typically perform data processing on a per-event basis, making the management of event data a crucial task within offline software. The management of event data includes defining how the data is organized and formatted during various stages of processing, as well as providing mechanisms for data input/output conversion. Additionally, features such as memory allocation/deallocation and input/output of data files, which are commonly shared across all data processing software, can significantly impact the performance of the entire offline software. Therefore, a software framework system is required to provide a powerful and high-performing event data management system that implements the following functions:
Provide a set of rules and tools for defining the data model, allowing application software developers to define the data models for various stages of processing (detector simulation, digitization, reconstruction, and analysis), including variable types, names, and access interfaces, and realizing various physical research needs by defining the relationships between data model objects (e.g., reconstructed tracks and simulated tracks, tracks and detector hits).
Provide an official data file format to achieve persistent data storage, and provide the ability to convert event data models in memory and data in files.
Provide efficient memory management services to manage the lifecycle of event data in memory, achieving efficient memory allocation/deallocation functions, and providing simple and easy-to-use data access interfaces for application software.
Ensure thread safety for the above functions and implement a shared memory pool for multiple worker threads to achieve high-concurrency data processing.
