The --where expression language

--where takes an expression that is evaluated as an event-level existence cut: the event passes if any row of the driver bank makes the expression truthy. The same grammar powers scan --expr computed columns and the TUI's live / filter.

It is available on count, scan, dump, skim, sample, and tui.

hipoq count rec.hipo --where "REC::Particle.pid == 11 && REC::Particle.chi2pid < 3"

Leading minus

Because a value can start with - (e.g. -1 < px < 1), the parser accepts it — your shell won't mistake the expression for a flag. Always quote the whole expression.

How an expression is evaluated

An expression is a numeric function of one bank row. For each row of the driver bank, hipoq evaluates the expression to a f64 and asks whether it is truthy:

a value is truthy when it is non-zero and not NaN.

• Comparisons (==, <, …) and logic (&&, ||, !) evaluate to 1.0 (true) or 0.0 (false).
• Any other expression is truthy on its own terms — --where "charge" keeps rows whose charge is non-zero (positive or negative), and --where "px*px + py*py - 0.25" keeps rows where that quantity isn't exactly zero. You'll almost always write a comparison, but the language doesn't require one.

The event passes when any driver row is truthy. (An expression with no per-row column references — a pure count(...)/constant like count(REC::Particle) >= 2 — is evaluated once for the event.)

References

Form	Meaning
BANK.column	that column, e.g. REC::Particle.px
column (bare)	binds to the driver bank, e.g. px
column[N]	element N of a fixed-length array column, e.g. cov[0]
count(BANK)	number of rows in BANK (a multiplicity)
count(expr)	number of rows where expr holds
any(expr)	1 if any row of expr's bank satisfies it (unlinked)
all(expr)	1 if every row satisfies it (and there is ≥1 row)
number	11, -0.5, 1.5e-2
particle	e-, pi+, proton, … — the PID it stands for (see below)

Column references may name any scalar numeric column — every integer type (byte/short/int/long) and float type (float/double) is read as a number.

Array columns

A fixed-length array column (declared cov/F#6 — six floats per row) is cut on one element at a time, chosen with a subscript:

hipoq count rec.hipo --where "REC::Track.cov[0] > 1e-4"
hipoq count rec.hipo --where "REC::Track.cov[0] > REC::Track.cov[5]"

The index is 0-based and checked against the schema when the expression is compiled, so cov[9] on a six-element column fails before any file is read:

column REC::Track.cov has 6 elements, so the index must be 0..5; got [9]

Naming an array without a subscript is refused rather than guessed:

column REC::Track.cov is an array (length 6); pick an element, e.g. `cov[0]`

That is deliberate. "Any element", "every element" and "element 0" are three different cuts and nothing in cov > 1 says which was meant — so it asks. A subscript on a scalar is refused for the mirror reason.

What it costs

An element read is about 16 % slower than a scalar one — 10.2 vs 12.2 Mev/s on a 200 000-event fixture with three rows per event, single-threaded. The library exposes a whole scalar column in one call but an array column one row at a time, so an element costs a column lookup per row where a scalar costs one per event. --explain says which references pay it.

To see every element of a row at once, use dump (arrays come out as real JSON arrays), scan, or press v in the TUI.

Particle names

Anywhere a PID number is expected you can write the particle instead — the same names --topology takes and --pdg prints:

hipoq count rec.hipo --where "REC::Particle.pid == e-"
hipoq count rec.hipo --where "REC::Particle.pid in (e-, pi+, pi-, proton)"

e- e+ mu- mu+ gamma pi+ pi- pi0 K+ K- p pbar n d t he3 he4 alpha and the spelled-out forms (electron, positron, photon, proton, neutron, deuteron, triton) all resolve. Nuclei use CLAS12's Geant3 codes (45/46/47/49), matching what is in the file.

Two rules keep this from changing what any existing expression means:

• A real column always wins. The name is read as a particle only when the driver bank has no column of that name, so a bank that genuinely has a gamma or n column keeps reading the column.
• Qualified references are never particles. REC::Particle.n is always the column n — naming a bank says a column is meant.

One consequence worth knowing: a charge sign binds to the name when it follows immediately, so e- is the electron while e - 1 (spaced) is a subtraction. Ordinary columns are unaffected — px-1 still subtracts, because px- is not a particle name.

p and e are rejected on purpose

These two name a particle and a physics helper, so hipoq refuses to guess:

$ hipoq count rec.hipo --where "p > 1000"
  `p` is ambiguous: write `p()` for the physics quantity, or `2212` for the particle

The reason is the silent failure on the other side. p() > 1000 is false for a 2 GeV track; if a dropped () quietly meant the proton PID, p > 1000 would be 2212 > 1000 — true for every event, with no error. Write p() or proton (e() or e-) and both readings stay available.

Operators

Precedence, high to low: unary · * / % · + - · comparison · && · ||. Parenthesize with ( … ).

Group	Operators
arithmetic	+ - * / % and unary -
comparison	== != < <= > >=
logical not	!x — 1 when x is falsy, else 0
range (chained)	lo < x < hi is lo < x && x < hi (any comparison operators)
membership	x in (a, b, c) is x==a || x==b || x==c
logic	&& ||

% is the floating-point remainder, and / is floating-point division (there is no integer division — 7 / 2 is 3.5).

hipoq count rec.hipo --where "-5 < REC::Particle.chi2pid < 5"              # range
hipoq count rec.hipo --where "REC::Particle.pid in (11, 211, -211, 2212)" # membership

Functions

Unary: sqrt abs sin cos tan asin acos atan log exp floor ceil round sign deg rad pdg_mass.

Binary: atan2 pow min max hypot.

Notes:

• Trigonometric functions work in radians; deg / rad convert radians ↔ degrees (deg(atan2(py, px)) for an azimuth in degrees).
• log is the natural logarithm (base e); exp is its inverse.
• sign(x) is -1, 0, or +1.
• pdg_mass(pid) returns the PDG rest mass in GeV for a particle id. The bundled table covers the common CLAS12 species (leptons, photon, π/K/η/ω, p/n and hyperons, light nuclei); an unknown id yields NaN (which then makes any comparison false).
• Calling a function with the wrong number of arguments is an error.

hipoq count rec.hipo --where "sqrt(px*px + py*py + pz*pz) > 1.5"   # bare cols, driver = REC::Particle

Physics helpers

Zero-argument helpers that expand to bare-column arithmetic on the driver bank — so they read the driver's px / py / pz (and pid, for energy). They follow CLAS12 REC::Particle conventions and are meant for that bank (or a scan of it); used where those columns don't exist they raise the usual unknown-column error.

Helper	Expands to
p()	sqrt(px*px + py*py + pz*pz)
pt()	sqrt(px*px + py*py)
theta() / theta_deg()	polar angle acos(pz / p) (radians / degrees)
phi() / phi_deg()	azimuth atan2(py, px) (radians / degrees)
e() / energy()	energy sqrt(p² + pdg_mass(pid)²)

hipoq count rec.hipo --where "REC::Particle.pid==11 && p() > 1.5 && theta_deg() < 20"

Because e() uses pdg_mass(pid), a row whose pid is not in the PDG table gets a NaN energy. These are the same expressions behind the scan presets.

Inclusive kinematics: Q2 and W

Unlike the helpers above, these are per event, not per row: they read one specific particle and give the same value for every row of the event.

Helper	Meaning
Q2(beam_energy)	Q² = 2·E·E′·(1 − cos θ), the negative four-momentum transfer squared
W(beam_energy, target_mass)	W = √(M² + 2M(E − E′) − Q²), the invariant mass of the hadronic system

Both take the beam energy in GeV, because nothing in the file reliably carries it. W needs it as well as the target mass — the energy transfer ν = E − E′ and Q² both depend on the beam — so W(0.938272) is an error rather than a guess:

hipoq count rec.hipo --where "Q2(10.6) > 1.0 && W(10.6, 0.938272) > 2.0"
hipoq scan  rec.hipo --bank REC::Particle --expr "q2=Q2(10.6)" --expr "w=W(10.6, 0.938272)"

The trigger electron

Both read REC::Particle row 0, and only if it is an electron (pid == 11) with a negative status — the CLAS12 convention marking the particle that fired the trigger.

Anything else gives NaN: row 0 is a different particle, its status is positive, or the event has no REC::Particle at all. That is deliberate. A Q² computed off a pion looks entirely plausible and there is nothing downstream that could catch it, so the helpers decline rather than guess.

NaN compares as false, so --where "Q2(10.6) > 1" drops the events without a trigger electron instead of counting them. If you want to know how many those are, count them directly:

hipoq count rec.hipo --where "REC::Particle.pid == 11 && REC::Particle.status < 0"

The electron mass is neglected

E′ is taken as |p⃗|, the standard CLAS12 form. At 10.6 GeV the m_e² term is ~2.6 × 10⁻⁷ GeV², a part in 10⁷ of a typical Q².

Below pion threshold a badly reconstructed track can make W² negative; W is NaN there rather than an imaginary number rendered as a float.

The beam energy and target mass must be constants — they describe the run, not a row, and a column there would make the value per-row again, which is the one thing these are not.

Aggregates (multiplicity)

count(...) turns a per-row condition into an event-level number — the basis of multiplicity cuts:

hipoq count rec.hipo --where "count(REC::Particle.pid==11) >= 2"   # ≥2 electrons
hipoq count rec.hipo --where "count(REC::Particle)==2 && count(REC::Calorimeter)>=3"

• count(BANK) — total rows in the bank (0 if the bank is absent).
• count(expr) — rows where expr is true, evaluated over expr's own driver bank (see unlinked aggregates).

A bare column is not a valid aggregate argument

count(REC::Calorimeter) is the row count, but count(REC::Calorimeter.energy) would follow the count(expr) rule and count rows whose energy is non-zero — one dot apart, a silently different number. On a bank with three rows and pindex = 0, 0, 1, the two readings give 3 and 1.

So hipoq refuses it rather than picking one:

$ hipoq count rec.hipo --where "count(REC::Calorimeter.energy) > 2"
  count(REC::Calorimeter.energy) counts rows where `energy` is non-zero, which is rarely what is meant.
    Instead, for the number of rows write count(REC::Calorimeter); to count non-zero
    values write count(REC::Calorimeter.energy != 0)

The same applies to any(...) and all(...). Writing the comparison you mean (!= 0, > 0.1, …) always works.

CLAS12-native cross-bank evaluation

An expression runs over the rows of a driver bank, chosen like this:

1. the first referenced bank with no pindex column (so REC::Particle drives when present);
2. if every referenced bank has a pindex, the first referenced bank;
3. if the expression is only bare columns, REC::Particle (when the file has it) — otherwise hipoq asks you to qualify a column as BANK.column.

Same-bank references use the current row, so pid==11 && charge<0 means one particle that is both. A reference to another bank is resolved by the CLAS12 pindex link, and the direction is set by whether the driver has a pindex:

• sum — driver is the primary (REC::Particle, no pindex): a detector column is summed over the rows whose pindex points at the current particle;
• gather — driver is a detector bank (has pindex): a REC::Particle column is read for the particle this row points at;
• broadcast — a single-row bank with no pindex (e.g. REC::Event) applies to every driver row. (Two same-length banks align row-for-row.)

# electron whose OWN total calorimeter energy exceeds 1.5 GeV (same particle)
hipoq count rec.hipo --where "REC::Particle.pid==11 && REC::Calorimeter.energy > 1.5"

# electrons in positive-helicity events (broadcast)
hipoq count rec.hipo --where "REC::Particle.pid==11 && REC::Event.helicity > 0"

Checking which join you got: --explain

Sum, gather and broadcast give materially different answers from the same text, and the choice follows from which banks carry a pindex — not from anything visible in the expression. --explain prints the compiled plan and exits without reading any events, so a cut can be checked before a scan is spent on it:

hipoq count rec.hipo --where "REC::Particle.pid==11 && REC::Calorimeter.energy>0.5" --explain

--where "REC::Particle.pid==11 && REC::Calorimeter.energy>0.5"
  driver bank: REC::Particle
    REC::Particle.pid: read directly
    REC::Calorimeter.energy: summed over rows whose pindex points at the driver row

--explain is a global flag, so it applies anywhere: on the commands that take --where (count, scan, dump, skim, sample, tui), and on any command at all when it is --topology you want explained — stats, hist and quality included. It reports each aggregate's own driver bank. Given nothing to explain it says so and exits 2:

$ hipoq info run.hipo --explain
--explain: this command has no --where or --topology to explain

Unlinked aggregates: any() / all()

Sometimes you want to ask about banks independently — "the event has an electron and (any) calorimeter hit above 1.5 GeV", not tied to the same particle. Wrap each bank in any(...) (or count(...)): each aggregate is self-contained, with its own driver bank, so no pindex join happens between them.

# unlinked: an electron somewhere AND a high-energy calo hit somewhere
hipoq count rec.hipo --where "any(REC::Particle.pid==11) && any(REC::Calorimeter.energy>1.5)"

Contrast the two forms:

Form	Question
pid==11 && REC::Calorimeter.energy>1.5	is there an electron whose own (pindex-summed) calorimeter energy > 1.5?
any(pid==11) && any(REC::Calorimeter.energy>1.5)	is there an electron and, separately, any calorimeter hit > 1.5?

• any(expr) is exactly count(expr) > 0.
• all(expr) is true when every row satisfies expr and there is at least one row (an empty or absent bank makes all(...) false).

Multiple --where clauses

--where is repeatable, and the clauses are ANDed. But each clause is compiled independently and keeps its own driver bank — which is not the same as joining them with && inside one expression when they reference different banks:

# TWO clauses — unlinked: some electron exists AND some calo hit > 1.5 exists
hipoq count rec.hipo --where "REC::Particle.pid==11" --where "REC::Calorimeter.energy>1.5"

# ONE expression — linked: an electron whose OWN summed calo energy > 1.5
hipoq count rec.hipo --where "REC::Particle.pid==11 && REC::Calorimeter.energy>1.5"

So separate --where flags behave like wrapping each in any(...); && within a single expression links the banks by pindex on a shared driver row. Reach for separate clauses when the conditions target different banks and you don't want them tied together.

Numeric semantics

• Float tolerance. Comparisons are tolerant at ~f32 precision: two numbers count as equal when

  |a - b| <= 1e-6 * max(|a|, |b|, 1)

  This is why px == 0.1 matches a value stored as 0.1_f32. The tolerance applies to every comparison, not just ==: < and > are strict-but-tolerant (a value within tolerance of the bound is treated as equal, so it does not satisfy a strict <), while <= and >= are inclusive of it. Integer columns differ by at least 1, far above the tolerance, so integer comparisons like pid == 11 stay exact.

• Big integers. A long column compared against a whole-number literal is compared in 64-bit integers, not floats. Above 2⁵³ (9007199254740992) a f64 cannot represent every i64 — 10000000000000000 and 10000000000000001 share one — so a run number or timestamp up there would otherwise match two neighbouring values at once. Arithmetic on such a column (timestamp / 2) still goes through f64 and can lose the last digits; compare the column directly if that matters.

• NaN. A NaN operand makes a comparison false, and a bare NaN is not truthy. Domain errors produce NaN (acos of an out-of-range argument, log of a negative), and divide-by-zero produces ±Inf.

• Computed columns. In scan --expr, a row that evaluates to NaN or Inf is rendered as a null/empty cell rather than aborting the scan.

Errors

The language fails loudly rather than silently matching nothing — a broken cut is a hard error, reported before any event is read:

• an unknown bank or column (bank "REC::Partical" … typo);
• an array column used without an element, an out-of-range element, or a subscript on a scalar — see Array columns;
• an unknown function or the wrong argument count;
• malformed syntax, including friendly hints for = (use ==), & (use &&), and | (use ||).