HIPO  4.3.0
High Performance Output data format for experimental physics
hipo::chain Class Reference

#include <chain.h>

Public Types

using iterator = ChainIterator
 
using const_iterator = ChainIterator
 

Public Member Functions

 chain (int threads=0, bool progress=true, bool verbose=false)
 Construct a chain with specified thread count. More...
 
int add (std::string_view filename)
 Add a single file to the chain. More...
 
int add (std::vector< std::string > &filenames)
 Add multiple files to the chain. More...
 
int add_pattern (std::string_view pattern)
 Add files matching a glob pattern. More...
 
void clear ()
 Remove all files from the chain. More...
 
std::size_t size () const noexcept
 Number of files in the chain. More...
 
bool empty () const noexcept
 Check if chain is empty. More...
 
int get_nb_files () const noexcept
 Legacy alias for size() More...
 
const FileInfooperator[] (std::size_t index) const
 Get file info by index. More...
 
const FileInfofile_info (int index) const
 Legacy alias for operator[]. More...
 
std::vector< FileInfo > & files () noexcept
 Get all file infos. More...
 
void set_tags (const std::vector< long > &tags)
 Set event tags for filtering. More...
 
void set_threads (int n)
 Set number of processing threads. More...
 
void set_progress (bool show)
 Enable/disable progress display. More...
 
void set_verbose (bool verbose)
 Enable/disable verbose output. More...
 
void open (bool validate_all=true)
 Validate and optionally load metadata for all files. More...
 
void scan ()
 Scan and display detailed information about all files. More...
 
void list () const
 Print list of files in chain. More...
 
iterator begin ()
 Get iterator to first event. More...
 
iterator end ()
 Get end iterator. More...
 
template<typename ProcessFunc >
void process (ProcessFunc &&process_func, double percentage=100.0)
 Process events in parallel across all files (record-level parallelism) More...
 
template<typename ProcessFunc >
void process_filtered (ProcessFunc &&process_func, const std::vector< std::string > &required_banks, double percentage=100.0)
 Process events with bank filtering (record-level parallelism) More...
 
template<typename FileFunc >
void for_each_file (FileFunc &&func)
 Apply a function to each file (for file-level operations) More...
 
const ChainStatisticsstatistics () const noexcept
 Get processing statistics. More...
 
long total_events ()
 Get total event count (loads metadata if needed) More...
 
long total_events_count () const
 Legacy alias. More...
 
void print_statistics () const
 Print processing statistics. More...
 
void show_all_info ()
 Show detailed info for all files. More...
 
bool any_has_config (std::string_view name)
 Check if any file has a configuration key. More...
 
std::optional< std::string > get_config (std::string_view name)
 Get configuration from first file that has it. More...
 
ThreadPoolthreadpool () noexcept
 Get the thread pool for advanced use. More...
 

Friends

class ChainIterator
 

Detailed Description

@brief Chain multiple HIPO files for unified processing

The chain class provides:
- File management (add, remove, pattern matching)
- Sequential iteration via range-based for loops
- Parallel processing with configurable thread pools
- Lazy metadata loading for minimal memory overhead
- Progress tracking and statistics

@code
hipo::chain ch(4);  // 4 threads
ch.add_pattern("data/*.hipo");

  • // Sequential iteration
  • for (auto& [event, file_idx, event_idx] : ch) {
  • auto particles = event.getBank("REC::Particle");
  • }

  • // Parallel processing
  • ch.process([](auto& event, int file_idx, long event_idx) {
  • // Thread-safe processing...
  • });

Definition at line 262 of file chain.h.

Member Typedef Documentation

◆ const_iterator

Definition at line 265 of file chain.h.

◆ iterator

Definition at line 264 of file chain.h.

Constructor & Destructor Documentation

◆ chain()

hipo::chain::chain ( int  threads = 0,
bool  progress = true,
bool  verbose = false 
)
inlineexplicit

Construct a chain with specified thread count.

Parameters
threadsNumber of threads (0 = auto-detect hardware concurrency)
progressShow progress bar during processing
verboseEnable verbose output

Definition at line 277 of file chain.h.

Member Function Documentation

◆ add() [1/2]

int hipo::chain::add ( std::string_view  filename)
inline

Add a single file to the chain.

Parameters
filenamePath to the HIPO file
Returns
Number of files in chain after addition

Definition at line 296 of file chain.h.

◆ add() [2/2]

int hipo::chain::add ( std::vector< std::string > &  filenames)
inline

Add multiple files to the chain.

Parameters
filenamesVector of file paths
Returns
Number of files in chain after addition

Definition at line 315 of file chain.h.

◆ add_pattern()

int hipo::chain::add_pattern ( std::string_view  pattern)
inline

Add files matching a glob pattern.

Parameters
patternGlob pattern (e.g., "data/*.hipo", "run_???.hipo")
Returns
Number of files matched and added

Definition at line 325 of file chain.h.

◆ any_has_config()

bool hipo::chain::any_has_config ( std::string_view  name)
inline

Check if any file has a configuration key.

Definition at line 937 of file chain.h.

◆ begin()

iterator hipo::chain::begin ( )
inline

Get iterator to first event.

Enables range-based for loops:

for (auto& [event, file_idx, event_idx] : chain) {
// Process event...
}
chain(int threads=0, bool progress=true, bool verbose=false)
Construct a chain with specified thread count.
Definition: chain.h:277

Definition at line 598 of file chain.h.

◆ clear()

void hipo::chain::clear ( )
inline

Remove all files from the chain.

Definition at line 369 of file chain.h.

◆ empty()

bool hipo::chain::empty ( ) const
inlinenoexcept

Check if chain is empty.

Definition at line 385 of file chain.h.

◆ end()

iterator hipo::chain::end ( )
inline

Get end iterator.

Definition at line 601 of file chain.h.

◆ file_info()

const FileInfo& hipo::chain::file_info ( int  index) const
inline

Legacy alias for operator[].

Definition at line 401 of file chain.h.

◆ files()

std::vector<FileInfo>& hipo::chain::files ( )
inlinenoexcept

Get all file infos.

Definition at line 406 of file chain.h.

◆ for_each_file()

template<typename FileFunc >
void hipo::chain::for_each_file ( FileFunc &&  func)
inline

Apply a function to each file (for file-level operations)

Parameters
funcCallable with signature: void(reader&, const FileInfo&)

Definition at line 872 of file chain.h.

◆ get_config()

std::optional<std::string> hipo::chain::get_config ( std::string_view  name)
inline

Get configuration from first file that has it.

Definition at line 956 of file chain.h.

◆ get_nb_files()

int hipo::chain::get_nb_files ( ) const
inlinenoexcept

Legacy alias for size()

Definition at line 388 of file chain.h.

◆ list()

void hipo::chain::list ( ) const
inline

Print list of files in chain.

Definition at line 570 of file chain.h.

◆ open()

void hipo::chain::open ( bool  validate_all = true)
inline

Validate and optionally load metadata for all files.

Parameters
validate_allIf true, open each file to validate; if false, only check existence
Exceptions
std::runtime_errorif no valid files in chain

Definition at line 437 of file chain.h.

◆ operator[]()

const FileInfo& hipo::chain::operator[] ( std::size_t  index) const
inline

Get file info by index.

Definition at line 393 of file chain.h.

◆ print_statistics()

void hipo::chain::print_statistics ( ) const
inline

Print processing statistics.

Definition at line 904 of file chain.h.

◆ process()

template<typename ProcessFunc >
void hipo::chain::process ( ProcessFunc &&  process_func,
double  percentage = 100.0 
)
inline

Process events in parallel across all files (record-level parallelism)

Uses record-level parallelism for efficient I/O: each thread grabs entire records and processes all events within them sequentially. This minimizes random access and maximizes cache efficiency.

Parameters
process_funcCallable with signature: void(chain_event&, int file_idx, long event_idx)
percentagePercentage of events to process (0-100)
chain.process([](auto& event, int file_idx, long event_idx) {
auto particles = event.getBank("REC::Particle");
// Thread-safe processing...
});

Definition at line 625 of file chain.h.

◆ process_filtered()

template<typename ProcessFunc >
void hipo::chain::process_filtered ( ProcessFunc &&  process_func,
const std::vector< std::string > &  required_banks,
double  percentage = 100.0 
)
inline

Process events with bank filtering (record-level parallelism)

Only processes events containing all specified banks. Uses record-level parallelism for efficient I/O.

Parameters
process_funcCallable with signature: void(chain_event&, int file_idx, long event_idx)
required_banksList of bank names that must be present
percentagePercentage of matching events to process

Definition at line 751 of file chain.h.

◆ scan()

void hipo::chain::scan ( )
inline

Scan and display detailed information about all files.

Definition at line 496 of file chain.h.

◆ set_progress()

void hipo::chain::set_progress ( bool  show)
inline

Enable/disable progress display.

Definition at line 423 of file chain.h.

◆ set_tags()

void hipo::chain::set_tags ( const std::vector< long > &  tags)
inline

Set event tags for filtering.

Definition at line 415 of file chain.h.

◆ set_threads()

void hipo::chain::set_threads ( int  n)
inline

Set number of processing threads.

Definition at line 418 of file chain.h.

◆ set_verbose()

void hipo::chain::set_verbose ( bool  verbose)
inline

Enable/disable verbose output.

Definition at line 426 of file chain.h.

◆ show_all_info()

void hipo::chain::show_all_info ( )
inline

Show detailed info for all files.

Definition at line 916 of file chain.h.

◆ size()

std::size_t hipo::chain::size ( ) const
inlinenoexcept

Number of files in the chain.

Definition at line 382 of file chain.h.

◆ statistics()

const ChainStatistics& hipo::chain::statistics ( ) const
inlinenoexcept

Get processing statistics.

Definition at line 892 of file chain.h.

◆ threadpool()

ThreadPool& hipo::chain::threadpool ( )
inlinenoexcept

Get the thread pool for advanced use.

Definition at line 980 of file chain.h.

◆ total_events()

long hipo::chain::total_events ( )
inline

Get total event count (loads metadata if needed)

Definition at line 895 of file chain.h.

◆ total_events_count()

long hipo::chain::total_events_count ( ) const
inline

Legacy alias.

Definition at line 901 of file chain.h.

Friends And Related Function Documentation

◆ ChainIterator

friend class ChainIterator
friend

Definition at line 1031 of file chain.h.


The documentation for this class was generated from the following file: