Skip to main content
Use query output when you need to analyze runs outside hiloop, archive experiment evidence, or join events with another system. Self-hosted deployments can additionally turn on a continuous forward sink that streams every event to an object store you own, an OTLP forward exporter that pushes the same stream to an OTLP collector, or both. There is no separate bulk export endpoint in the public API today. For public surfaces, export by querying the rows you need and writing the response to a controlled destination.

​
Continuous forward sink

Self-hosted deployments can continuously forward telemetry to an object store you control — for audit retention, SIEM ingestion, or a data warehouse — without querying. When enabled, every batch of events the gateway flushes to its own cold storage is also written to your bucket as newline-delimited JSON (JSONL): one event per line.
  • Off by default, opt-in. Set the forward URL to enable it (see the Helm reference export.* values); leave it empty to disable.
  • What lands. One JSONL object per flushed batch, laid out by organization and date so you can retrieve a window without a full scan:
    Each line is one event carrying its event_id, org_id, project_id, run_id, root_run_id, lineage_path, and trace/span IDs — so you can correlate runs, sandboxes, and lineage on your side outside hiloop.
  • Redaction. The sink applies no transformation and runs no redaction of its own. Events forwarded here carry exactly whatever scrubbing happened upstream, which for runs captured by hiloop run is the narrow capture-side pattern set, and for telemetry sent straight to the ingest endpoints is none at all. See redaction before pointing this at a destination with its own handling requirements.
  • Delivery is best-effort. Forwarding runs off the flush path and never blocks ingest: if your bucket is briefly unreachable or slow, that batch is dropped (and logged) rather than retried, and ingest continues uninterrupted. Treat the sink as a near-real-time archive, not a guaranteed, gap-free ledger.
  • Tamper-evidence. Point the sink at a bucket with Object Lock (WORM) enabled to keep an append-only, immutable audit trail; grant the sink least-privilege, write-only credentials.
The sink writes to any S3-compatible object store (AWS S3 or MinIO). Credentials for the sink bucket are configured separately from the events store, so the audit sink can use its own write-only credentials. See the Helm reference export.* values.

​
OTLP collector forward

Self-hosted deployments can also push the same continuous stream to an OTLP collector — for SIEM and observability pipelines that speak OTLP (Splunk, Chronosphere, Grafana, or any OpenTelemetry Collector). When enabled, every batch of events the gateway flushes to its own cold storage is also exported over OTLP/gRPC as log records.
  • Off by default, opt-in. Set the collector endpoint to enable it (see the Helm reference otlpExport.* values); leave it empty to disable. It is independent of the object-store sink — enable either or both.
  • What lands. One LogsService.Export call per flushed batch. Each event becomes one log record whose timestamp is the event’s causal wall clock and whose event_name is the event name; the record body carries the event’s attributes as JSON, and the record attributes carry the correlation identity you join on: hiloop.event.id, hiloop.signal, hiloop.project.id, and — when the event belongs to a run — hiloop.run.id and hiloop.lineage.path. Events captured with trace context export their trace and span IDs natively, so trace-aware backends correlate without custom joins. The batch’s resource carries service.name and hiloop.org.id.
  • Authentication. Optional collector headers (for example authorization=Bearer <token>) are supplied from a secret and attached to every export call; they are never logged.
  • Redaction. The exporter applies no transformation and runs no redaction of its own. Exported events carry exactly whatever scrubbing happened upstream, which for runs captured by hiloop run is the narrow capture-side pattern set, and for telemetry sent straight to the ingest endpoints is none at all. See redaction before pointing this at a destination with its own handling requirements.
  • Delivery is best-effort. Exporting runs off the flush path and never blocks ingest: if the collector is unreachable, slow, or rejects records, that batch is dropped (and logged) rather than retried, and ingest continues uninterrupted. If you need buffering or retries, front the endpoint with an OpenTelemetry Collector you operate — the standard deployment shape for OTLP pipelines.

​
Query API export

For small exports, run a SELECT and save the rows. The CLI builds the SQL from scoping flags and --output json writes the raw {"rows": […]} body:
For an arbitrary projection, pass the SQL yourself with --sql, or POST /v1/telemetry/sql with a {"sql": "<SELECT …>"} body. For aggregate output, group in the SELECT — here over the otel_genai_calls data view, which derives one row per LLM exchange from the raw captured payloads:

​
Reproducibility bundle

A useful manual export bundle includes:
manifest.json should record:

​
Governance

Before export:
  • confirm the destination is approved;
  • apply your redaction policy;
  • prefer digests and metadata when full payloads are not needed;
  • record who requested the export;
  • set retention on the exported object;
  • encrypt exported files at rest and in transit.