Inspecting a file

Seven read-only commands answer "what is in this file?" — from a one-line summary down to the rows of a specific event. info, dict and banks describe the whole file, records maps its physical layout, head and tail show events at either end, and status decodes a single REC::Particle.status value without opening a file at all.

info

File/chain summary: event count, record count, schema count, and per-file sizes.

hipoq info rec.hipo

input:   rec.hipo
files:   1
events:  100000
records: 42
schemas: 18

on disk: 24.1 MiB

Point it at a glob or directory to summarize a whole chain at once:

hipoq info "run/*.hipo"

dict

Dump the schema dictionary — every bank and its columns — or focus one bank.

hipoq dict rec.hipo                    # every schema
hipoq dict rec.hipo --bank REC::Particle   # one bank's columns (name, type, length)
hipoq dict rec.hipo --raw              # the raw HIPO schema text

Flag	Meaning
--bank B	show only bank B's columns
--raw	emit the raw schema text instead of a formatted table

banks

Which banks actually carry data, with per-bank event and row counts. Unlike dict (which lists what could appear), banks scans events and reports what does.

hipoq banks rec.hipo               # scan the whole file
hipoq banks rec.hipo --max 5000    # sample the first 5000 events (faster)

Flag	Meaning
--max N	stop after scanning N events (0 = whole file)

--max is no longer the way to make this fast: banks reads each bank's per-event extent out of the record's offset tables and inflates nothing, so a full pass over 20,000 events of a CLAS12 run takes 0.08 s rather than 0.38 s and uses half the memory. Use --max when you want the first N events specifically — it forces a sequential scan so that "first" has a defined meaning.

records

The per-record map: where each record sits, how many events it holds, and how much it inflates to. Nothing is decompressed — 0.02 s on an 8.5 GB file.

hipoq records rec.hipo                 # distribution plus the extremes
hipoq records rec.hipo --mode full     # one line per record
hipoq records rec.hipo --top 10        # list the 10 largest

Flag	Meaning
--mode summary|full	distribution + extremes (default), or every record
--top N	how many of the largest records to list in summary mode

HIPO inflates a whole record to reach any single event in it, so record shape is what explains a slow read, or a file whose size makes no sense against its event count — and none of it is visible in info's totals. On CLAS12 run 22083 the map shows a fixed ~8 MiB record target (p50 = p90 = p100), 15.2 GiB inflated against 8.5 GB on disk, and 21–35 KiB of inflation per event reached. That last number is why seeking to one event is not free.

The most expensive event to reach is reported separately from the largest record, because with size and event count varying independently they are not the same record.

status

Decode a REC::Particle.status value — for the number you have in a log and cannot interpret. Takes no input file.

$ hipoq status -4231
status -4231 → CD s2 c3 k1 !
  regions:
    central (CVT/BST/BMT/CTOF/CND/RTPC)
  responses: 2 scintillator, 3 calorimeter, 1 Cherenkov
  trigger particle (determined the event start time)

status packs a detector topology into one integer. With --pdg the column renders decoded in scan, head and the TUI (FD s1 k2) instead of as a raw number.

The encoding is coatjava's DetectorParticleStatus: region bits in |status| / 1000 (1 tagger, 2 forward, 4 central, 8 BAND), then scintillator, calorimeter and Cherenkov response counts in the hundreds, tens and units. Two things to know, because both are easy to get wrong when cutting on it by hand:

• The region is a bitmask, not an enum. A particle with forward and central hits carries 6000, and status / 1000 == 2 calls it neither.
• A negative status marks the trigger particle — the one that determined the event start time — and every other field comes from |status|.

head

Peek at the first N events: which banks are present and how big.

hipoq head rec.hipo -n 5

event 0: REC::Event(1), REC::Particle(3), REC::Calorimeter(7)
event 1: REC::Event(1), REC::Particle(2), REC::Calorimeter(4)
...

An event carrying a non-zero tag says so, and one that does not stays as it was:

event 0 [tag 3]: REC::Particle(1)
event 1: REC::Particle(1)

Unlike dump, which always emits the field because a machine format wants a stable schema, this line is read by a person — tag 0 on every event of a file that does not use tags is width taken from the bank list for no information. The TUI's status bar follows the same rule.

Add --bank to also render that bank's rows as a table for each event:

hipoq head rec.hipo -n 3 --bank REC::Particle
hipoq head rec.hipo -n 3 --bank REC::Particle --pdg   # decode pid to symbols

Flag	Meaning
-n, --count N	number of events to show (default 5)
--bank B	also print bank B's rows per event

head is the quickest way to eyeball real values; for a wider projection across an event range, reach for scan.

tail

The last N events, and the banks each carries.

hipoq tail rec.hipo -n 3
hipoq tail rec.hipo -n 1 --bank REC::Particle

Flag	Meaning
-n, --count N	how many events (default 5)
--bank BANK	also render that bank's rows as a table

It seeks from the trailer index rather than reading the file to get there, so it costs the same at either end of a file of any size: 0.01 s on an 8.5 GB DST, the same as head. Before this there was no way to look at the end of a file without first running count, doing the arithmetic, and passing --events.

Indices reported are global, matching dump --events.

Worth doing on any run file — the last events are usually not physics:

event 598735: HEL::flip(1)
event 598736: HEL::flip(1)
event 598737: RUN::config(1), RUN::unix(5955)