Skip to main content

HIPO & CLAS12 data model

You don't need to know the byte layout of HIPO to use hipoq, but a mental model of events, banks, and the pindex link makes the --where language and the cross-bank commands click into place.

Files, records, events​

A HIPO file is a sequence of compressed records. Each record holds a batch of events. hipoq hides records from you — you address data by event index (0, 1, 2, …) and the reader handles records underneath.

  • info reports the event, record, and schema counts.
  • Random access (hipoq tui, hipoq doctor) reads one event at a time via an index; streaming commands (count, scan, skim, …) walk events in order.

A file also carries a trailer — an index of record positions written last. If a writer crashes before the trailer is written, the file is still recoverable; see repairing files.

Schemas and banks​

A schema (a.k.a. the dictionary) defines a named bank and its columns. In CLAS12 the names are group-qualified, e.g. REC::Particle, REC::Calorimeter, REC::Event.

  • A bank in a given event is a small table: some number of rows, each with the columns from its schema.
  • Columns are typed (byte, short, int, long, float, double) and may be scalar (one value per row) or array (a fixed-length vector per row, like a covariance cov).

Inspect schemas with dict; see which banks actually carry data with banks.

REC::Particle           (event 7)
 row │ pid   px     py     pz     charge
 ────┼────────────────────────────────────
  0  │  11   0.51  -0.02   2.31    -1
  1  │ 2212  0.30   0.11   0.98     1
  2  │  22   0.05   0.01   0.44     0

Particle vs detector banks​

CLAS12 reconstruction splits information across banks that refer to one another:

  • REC::Particle is the primary bank — one row per reconstructed particle (momentum, pid, charge, …). It has no pindex column.
  • Detector banks — REC::Calorimeter, REC::Scintillator, REC::Cherenkov, REC::Track, … — have a pindex column: each detector row names the REC::Particle row it belongs to.
  • Single-row event banks — REC::Event, RUN::config — carry one row that describes the whole event (helicity, run number, …).

pindex is a foreign key from a detector row to a particle row. hipoq resolves it automatically whenever an expression references two banks. The direction is decided by whether the driver bank has a pindex column — so the same reference REC::Calorimeter.energy sums on a particle scan but reads a single value on a calorimeter scan:

Sum (particle → detector)​

From REC::Particle (the driver), a detector column is summed over the rows whose pindex points at the current particle:

# each electron's TOTAL calorimeter energy (sum of its calorimeter rows)
hipoq scan rec.hipo --bank REC::Particle --cols pid --expr "ecal=REC::Calorimeter.energy"
hipoq count rec.hipo --where "REC::Particle.pid==11 && REC::Calorimeter.energy > 1.5"

So REC::Calorimeter.energy, evaluated on a particle row, is that particle's summed deposit — not one hit.

Gather (detector → particle)​

From a detector bank (the driver), a REC::Particle column is read for the particle that this detector row points at:

# each calorimeter hit annotated with its particle's pid
hipoq scan rec.hipo --bank REC::Calorimeter --cols energy --expr "pid=REC::Particle.pid"

Broadcast (single-row bank → every row)​

A single-row bank with no pindex (like REC::Event) applies to every driver row:

# electrons in positive-helicity events
hipoq count rec.hipo --where "REC::Particle.pid==11 && REC::Event.helicity > 0"

Driver bank​

Every expression runs over the rows of a driver bank, chosen in this order:

  1. the first referenced bank that has no pindex (so REC::Particle drives when it's mentioned);
  2. if every referenced bank has a pindex, the first referenced bank;
  3. if the expression is only bare columns (px, pid, …), REC::Particle when the file has it — otherwise hipoq asks you to qualify one column as BANK.column so the bank is unambiguous.

Same-bank references use the current row, so pid==11 && charge<0 means one particle that is both an electron and negatively charged — not "some electron exists and some negative particle exists". For the latter, independent-of-pindex question, use the any() / all() aggregates.

Why this matters

The driver bank and the pindex direction are chosen from which banks your expression names. You rarely set them explicitly — but knowing the rule explains why REC::Calorimeter.energy sums on a particle scan yet reads a single value on a calorimeter scan.

PDG particle ids​

Particle types use PDG id codes: 11 = electron, -11 = positron, 211 = π⁺, 2212 = proton, 22 = photon, and so on. hipoq bundles a PDG table used for two things:

  • --pdg decodes pid columns to symbols (e-, pi+, p, …) in human-facing output — see global flags.
  • pdg_mass(pid) returns the rest mass in GeV, used by the e()/energy() physics helper.