dump

Emit whole events as JSON — one object per line (NDJSON). Where scan projects a single bank into a flat table, dump gives you every present bank at once, so it's the bridge to any JSON-speaking tool: jq, pandas, DuckDB, awk, a notebook.

hipoq dump rec.hipo --events 0..1 | jq .

{
  "event": 0,
  "tag": 0,
  "REC::Calorimeter": [
    { "pindex": 0, "energy": 0.8, "sector": 1 },
    { "pindex": 0, "energy": 0.9, "sector": 4 },
    { "pindex": 1, "energy": 0.3, "sector": 2 }
  ],
  "REC::Event": [ { "helicity": 1 } ],
  "REC::Particle": [
    { "pid": 11, "px": 0.5, "py": 0.0, "pz": 2.0, "charge": -1 },
    { "pid": 2212, "px": 0.3, "py": 0.1, "pz": 1.0, "charge": 1 }
  ]
}

Each line is one event: an event index, then each present bank as an array of {column: value} row objects, the columns in schema order.

Banks come out sorted by name, not in the order the dictionary declares them — above, the dictionary's order is REC::Particle, REC::Event, REC::Calorimeter. Sorting keeps a line's shape stable across files whose dictionaries were written in different orders, which is what a jq filter or a pandas load needs. Within a bank the columns follow the schema.

Options

Flag	Meaning
--events A..B	event span, end-exclusive. Omit for the whole file
--bank B	include only these banks (repeatable, comma-ok). Empty = all present
--where E	keep only events passing a cut (repeatable, ANDed)
--format F	ndjson (default), json, or table; csv is refused — see shape vs encoding

In ndjson values stay numeric — pid is not decoded, even with global --pdg — so the stream is clean for downstream parsing. --format table is the human-facing form and does honour --pdg, exactly as scan's table does.

As tables, one per bank per event

--format table renders each bank of each event as its own table — that bank's columns, every row, nothing padded:

hipoq dump rec.hipo --events 0..1 --format table --bank REC::Particle,REC::Calorimeter

event 0 · REC::Calorimeter  (3 rows)
┌─────┬────────┬────────┬────────┐
│ row ┆ pindex ┆ energy ┆ sector │
╞═════╪════════╪════════╪════════╡
│ 0   ┆ 0      ┆ 0.8    ┆ 1      │
└─────┴────────┴────────┴────────┘

event 0 · REC::Particle  (2 rows)
┌─────┬──────┬─────┬─────┬─────┬────────┐
│ row ┆ pid  ┆ px  ┆ py  ┆ pz  ┆ charge │
╞═════╪══════╪═════╪═════╪═════╪════════╡
│ 0   ┆ 11   ┆ 0.5 ┆ 0.0 ┆ 2.0 ┆ -1     │
└─────┴──────┴─────┴─────┴─────┴────────┘

This is the shape for reading one event; scan --bank A,B is the shape for extracting a column across many. The difference is the union: a scan of two banks puts them in one table with blank cells where a bank lacks a column, while here each bank keeps its own columns and nothing is blank.

--bank selects which banks get a table (repeatable, comma-ok; empty = all present), and global --pdg decodes pid to symbols in the table the same way it does for scan.

csv is refused; json is not

--format json gives the same event objects as the NDJSON stream, wrapped in one array — for a reader that wants a single document rather than a stream.

csv has no such form, and for a reason specific to it: CSV is one header over uniform rows, and an event is several banks with different row counts and different columns. Padding to the longest bank, or emitting only the first, would be a guess about what you meant. scan --bank A,B --format csv is the flat projection.

The tag field

Every line carries the event's own tag, from the event header:

hipoq dump run.hipo | jq -r 'select(.tag == 3) | .event'

It is present even when zero, on purpose. A key that appeared only on tagged events would make the stream's schema depend on its contents — jq '.tag' would return null on some lines, and read_json(lines=True) would give you a column that exists for some files and not others. It costs ten bytes against ~155 KiB of event on a CLAS12 DST.

Filtering is cheaper done up front with --tag, which reads the header and inflates nothing, rather than by dumping everything and selecting in jq.

NDJSON, not a JSON array

Output is newline-delimited JSON: one self-contained object per line, not a single big array. This streams (you can start processing before the file is fully read) and plays nicely with line-oriented tools:

hipoq dump rec.hipo | head -1 | jq .          # first event, pretty
hipoq dump rec.hipo | wc -l                    # event count that reached stdout

Selecting banks and events

# only the particle and event banks, first 1000 events
hipoq dump rec.hipo --events 0..1000 --bank REC::Particle,REC::Event

# only events with an electron
hipoq dump rec.hipo --where "REC::Particle.pid == 11"

The --where cut uses the full expression language, so cross-bank count() / any() conditions work here too.

Into a dataframe

The common pattern — filter in hipoq, analyze in Python:

hipoq dump rec.hipo --where "REC::Particle.pid==11" > electrons.ndjson

import pandas as pd
df = pd.read_json("electrons.ndjson", lines=True)
# df["REC::Particle"] holds a list of row-dicts per event; explode/normalize as needed

See the programmatic access guide for fuller jq / pandas / DuckDB recipes.

dump vs scan

Use scan --format ndjson when you want one bank as flat rows (one JSON object per bank row). Use dump when you want the whole event (one JSON object per event, every bank nested inside).

Cost

dump writes every bank of every event, which on a CLAS12 DST is about 155 KiB of JSON per event. It streams — one event is serialised and dropped, so memory does not grow with the range — and runs at 247 MB/s, or 660 µs per event, on run 22083. A whole run file is roughly 6.6 minutes and about 95 GB of JSON.

With --bank it is far cheaper, because only the requested banks are read and the event's structure list is never walked: 25 µs per event for one bank. If you want one bank across many events, scan is usually the better shape anyway — it gives you rows rather than nested objects. See Output formats.