Skip to content

FAQ & Troubleshooting

Quick answers to the gotchas that come up most often. Each entry links to the page where the topic is documented in full.

Reading

bank.getRows() returns 0 — what now?

A bank with zero rows means the bank wasn't present in the event you just read, not that an error occurred. Skip the event and continue:

event.read(particles);
if (particles.getRows() == 0) continue;

See recipes/reading.md — Handling Missing Banks.

getEntryOrder("foo") returns -1

The column doesn't exist on this schema. getEntryOrder returns -1 instead of throwing so you can guard cheaply:

int col = schema.getEntryOrder("nonexistent");
if (col < 0) {
    // not found — fall back, log, or skip
}

See reference/error-handling.md — getEntryOrder Returns -1.

How do I guard against out-of-bounds row/column access?

You don't get bounds checking from getInt/getFloat/… — they're hot-path accessors. You must check row < bank.getRows() and that the column index is valid before the call. Out-of-bounds access is undefined behavior.

See the warning at the bottom of reference/error-handling.md.

hipo::file_error on open

Almost always one of: file path wrong, file truncated, magic number invalid, or you're reading something that isn't a HIPO file. Wrap reader.open() in a try / catch (const hipo::file_error&) and inspect e.what().

See reference/error-handling.md — hipo::file_error.

Writing

My write loop is slow / produces tiny records

Records flush at 100,000 events or 8 MB uncompressed, whichever comes first. If you call writer.close() after only a handful of events, you get one tiny record and lose most of the LZ4 compression benefit. Batch your writes when possible.

See reference/performance.md — Record Sizing.

Can I write from multiple threads?

No. hipo::writer is single-threaded by design. The standard workaround is one writer per worker thread writing to a different output file, then merging (or just keeping the shards) at the end.

See reference/performance.md — No Parallel Writing.

Can I update a HIPO file in place?

No. Read from one file, write to another. There's no in-place modification API.

See reference/performance.md — No In-Place Modification.

My event hits the 128 KB buffer limit

The default event buffer is 128 KB and the library does not auto-grow it. Either split your data across more events, drop the largest banks, or work around it by writing to multiple records with smaller per-event payloads.

See reference/performance.md — Event Size Limit.

Why does addUserConfig need to be called before open()?

The user config keys/values are baked into the dictionary record, and the dictionary is written as the first record of the file. After open() it's already on disk.

See recipes/writing.md — User Configuration.

Performance

My read loop is much slower than the 1 M events/sec figure

Most likely you're paying for one of:

  • Map lookups in the hot loop — cache column indices once via schema.getEntryOrder(name) and use the integer overload.
  • Re-allocating banks per event — create the bank once outside the loop; it reuses its buffer.
  • Re-parsing filter strings — build a hipo::Parser once and reuse it.

See reference/performance.md — Optimization Tips.

I want parallelism without writing my own threads

Use hipo::chain — record-level parallelism across many files with a thread-safe callback. See recipes/parallel.md.

Integration

How do I call HIPO from ROOT?

Use the HipoDataFrame extension, which provides a ROOT RDataSource:

auto df = MakeHipoDataFrame("data.hipo");
auto h = df.Histo1D("REC::Particle_px");

Requires ROOT ≥ 6.28. See recipes/integration.md — ROOT RDataFrame.

How do I call HIPO from Python?

There's a wrapper called hipopy:

import hipopy
reader = hipopy.open("data.hipo")

See recipes/integration.md — Python (hipopy).

pkg-config is the supported path:

pkg-config --cflags --libs hipo4

See recipes/integration.md — pkg-config for both CMake and Makefile snippets.