Event tags

A HIPO event carries a tag in its header — a u32 the writer sets and every reader can see. It is the cheapest classification available: unlike anything in a bank, it can be tested without inflating a single compressed stream.

Records carry a tag of their own, and that one is cheaper still: a record whose tag does not match is skipped without being decompressed at all.

Finding out what a file uses

A tag is just a number unless the writer recorded names for its bits. info shows them when it does, straight from the file header:

hipoq info run.hipo

events:  60
records: 6
tags:    Dvcs (bit 0), Sidis (bit 1)
         select with `--tag-any NAME`, or `--tag N` for an exact tag

No tags: line means the file declares no names — the tags may still be there as bare numbers. To see what values actually occur, read a few events:

hipoq scan run.hipo --bank REC::Particle --cols pid --format csv | cut -d, -f2 | sort -u

Selecting by tag

Three global flags, so every command that reads events takes them — dump, scan, head, tail, count, skim, stats, the TUI:

flag	keeps	cost
--tag N	events whose tag equals one of these	reads the event header
--tag-any NAME|MASK	events with any of those bits set	reads the event header
--record-tag N	records whose tag matches	skips the record undecompressed

hipoq dump run.hipo --tag 1                    # only tag-1 events
hipoq scan run.hipo --bank REC::Particle --tag 1
hipoq count run.hipo --tag-any Dvcs            # by declared name
hipoq count run.hipo --tag-any 0x6             # or by mask: 12, 0b110 too
hipoq skim run.hipo dvcs.hipo --tag-any Dvcs   # write the selection out

--tag is for a tag used as one value; --tag-any for a tag used as a set of flags. Repeats of either accumulate — several --tag values are alternatives, several --tag-any masks are ORed.

All of them compose with --where and --topology, and are much cheaper than either, so put the tag first when it does most of the narrowing:

hipoq count run.hipo --tag-any Dvcs --where "REC::Particle.pid == 11"

An unknown name is an error, not an empty result

--tag-any Dcvs (a typo) fails and lists the names the file declares. A silently-zero mask would select nothing and be indistinguishable from a file that simply has no matching events.

Seeing the tag

command	shows the tag	when
dump	"tag" field	always, even when zero
scan	tag column, after event	always, in every format
head, tail	event 0 [tag 3]: …	only when non-zero
tui	status bar	only when non-zero

The split is deliberate. dump and scan are machine formats, and a field that appeared only on tagged events would make the output's schema depend on its contents — jq '.tag' would return null on some lines, and a CSV script reading column 3 would get row on one run and pid on another. head and the TUI are read by a person, where tag 0 on every line of an untagged file is width taken from the part they came for.

Getting back to the events

Under any filter, the event number that dump and scan print is the surviving position, not the file position — the same convention --require uses. For the real indices, ask count --list, which emits global ones:

hipoq count run.hipo --tag 1 --list > tag1.txt   # 3, 6, 9, 12, …
hipoq skim run.hipo tag1.hipo --events-from tag1.txt

--record-tag cannot be combined with --list

--list walks events by index and cannot see which record an event came from, so it refuses rather than emit a list that quietly ignores the clause — the list is meant to be fed to skim, which would act on it.

Tags and the commands that write files

Every write command preserves the event tag:

command	how
skim, sample, split, doctor, convert	the event's bytes are re-emitted verbatim
skim --banks	rebuilds the event, and copies the tag across
merge	takes the signal's tag — input 0, the same input its tag registry comes from

The two that rebuild an event rather than copy it used to lose the tag: a skim --banks projection came out with every tag zeroed while keeping every event, which reads as "this sample has no tags" rather than as data loss, and --tag on the output then matched nothing. Both are covered by a test now.

merge combines several events into one, so its inputs can disagree about the tag; it takes input 0's, which is the only choice consistent with input 0 being the signal that the others are overlaid onto.