> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hiloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Annotations

> Attach durable, structured judgments to a run's telemetry — and filter and aggregate on them later.

An annotation is a structured judgment about a run: an eval score, a pass/fail label, a reviewer's
note. Annotations are [events](/observability/event-model) like any other — they live on the same
[run identity](/concepts/run-scoped-observability) and are queryable alongside the LLM calls, tool calls, and logs they
describe. That means you can filter and aggregate on a human or model judgment exactly the way you
query the rest of a run.

An annotation's payload is **entirely yours** — there are no built-in fields. You decide what an
annotation carries (a score, a verdict, an annotator, a free-text note, a nested object) by
registering a schema for it. You write annotations from the CLI or SDK (shown below) and read them
back with a telemetry [query](/observability/query-telemetry).

You annotate two things:

* **A point** — a single event (`target_event_id`), or the run as a whole when you omit the target.
* **A range** — a `[start, end]` wall-clock window within a run.

Every annotation names a **schema** that its payload is validated against at ingest, so the data
stays well-shaped enough to query.

## Register a schema first

A schema is a named, versioned [JSON Schema](https://json-schema.org/) (draft 2020-12) that ingest
validates annotation payloads against. Register one before you annotate against it:

```sh theme={null}
hiloop annotation-schema register eval.quality \
  --json-schema @eval-quality.schema.json
```

Schemas are immutable and versioned per organization:

* **Names are organization-global, not project-local** — every project and workstream in the organization
  shares one namespace. Namespace your schema names per workstream (`kaggle.experiment.v1`, not
  `experiment.v1`): re-registering a name another workstream already uses doesn't error — it
  silently creates that schema's next version for everyone reading it.
* An unseen name starts at version 1.
* Re-registering an existing name creates the next version — **after a backward-compatibility
  check**. An incompatible change is rejected rather than silently breaking existing annotations.
* `hiloop annotation-schema archive eval.quality` retires a version without deleting it;
  annotations already stamped against it stay valid.
* `hiloop annotation-schema list` shows the latest live version per name (add `--include-archived`
  for every version); `get <name> [--version N]` fetches one.

### Promote the fields you query

Any field stays queryable from the JSON payload. To make a field **fast** to filter, sort, or join
on across many runs, promote it into a typed column at register time with `--promote
field:type[:identity][:bloom]` (`type` is `str` / `f64` / `i64` / `bool`):

```sh theme={null}
hiloop annotation-schema register eval.quality \
  --json-schema @eval-quality.schema.json \
  --promote score:f64 \
  --promote outcome:str \
  --promote annotator:str:identity
```

* A promoted field gets full columnar acceleration (statistics, page-index pruning, predicate
  pushdown) for the "compute once, filter forever" query — without baking any field into the
  platform. Unpromoted fields stay queryable from the payload, just without the column speedup.
* `:identity` marks a field part of the **latest-wins** key (see [Re-scoring](#re-scoring)). Mark an
  `annotator` field `identity` to keep the latest write *per annotator*. **Distinct records need an
  identity field**: without one, every write to the same target supersedes the last, so rows you
  meant to coexist (one per annotator, lane, or segment) collapse to the newest on the default read.
  Registration warns when a schema promotes fields but declares no identity field.
* `:bloom` adds a point-lookup index (string fields only) — useful for a high-cardinality promoted
  id you look up by exact match.
* Registration also auto-creates the schema-named views (here `ann_eval_quality`, plus its
  full-history sibling `ann_eval_quality_history`) that surface each promoted field under its
  declared name — see [Query annotations back](#query-annotations-back).

## Annotate a run or an event

Attach a judgment to a run — or to one event within it with `--target-event`. There are two ways
in, and they write the same annotation event:

* **The CLI** (`hiloop annotations add`) sends through the telemetry ingest path — the ergonomic option.
* **The `Annotate` API** is the typed call you make from your code.

Everything you want the annotation to carry goes in `--data`; there are no special value flags. The
payload is used verbatim — nested objects and arrays are preserved — and `--data` accepts inline
JSON, `@file`, or `-` for stdin:

```sh theme={null}
hiloop annotations add \
  --run 01K6Z… \
  --schema eval.quality \
  --target-event 01K71… \
  --data '{"score":0.2,"outcome":"fail","annotator":"human","note":"hallucinated the API name"}'
```

A structured payload — say a result with a nested `metric{}` block and an array of evidence event
ids — goes through the same flag:

```sh theme={null}
hiloop annotations add \
  --run 01K6Z… \
  --schema experiment.idea \
  --data '{
    "annotator": "llm",
    "score": 52.4,
    "outcome": "worked",
    "headline": "Add interaction + ratio features",
    "hypothesis": "Pairwise ratios expose nonlinearity a linear model can use directly.",
    "metric": { "name": "rmse", "value": 52.4, "direction": "lower_better", "valid": true },
    "evidence_event_ids": ["01HQZX1FEAT0CHANGE", "01HQZX1FEAT0METRIC"]
  }'
```

If the schema promotes `score`/`outcome`/`annotator`, those values lift into typed columns for fast
filter/sort while staying in the payload. Both examples anchor the annotation at the run you pass
as `--run` — an annotation always belongs to exactly one run and inherits that run's logical
lineage position. The sandbox runtime does not inject a sandbox credential or `HILOOP_RUN_ID`;
annotate from an authenticated external client. `--output json` prints the minted
`{"event_id": …}`.

By default the server mints a fresh `event_id` per invocation, so re-running the command writes a
new annotation. To make a retry safe after an ambiguous failure (a 5xx, a lost response), mint the
id yourself with `--event-id <ULID>`: re-running with the same id returns the stored annotation
instead of writing a duplicate. The id names this logical annotation — never reuse it for
different content.

The same annotation through the API, which anchors by `run_id`:

<CodeGroup>
  ```typescript TypeScript theme={null}
  import { annotationServiceAnnotate } from "@hiloopai/sdk";

  const { data } = await annotationServiceAnnotate({
    client,
    body: {
      run_id: "01K6Z…",
      schema_name: "eval.quality",
      target_event_id: "01K71…",
      payload_json: JSON.stringify({
        score: 0.2,
        outcome: "fail",
        annotator: "human",
        note: "hallucinated the API name",
      }),
    },
  });
  console.log(data?.event_id);
  ```

  ```python Python theme={null}
  import json
  from hiloop.api.annotation_service import annotation_service_annotate
  from hiloop.models import AnnotateRequest

  resp = annotation_service_annotate.sync(
      client=client,
      body=AnnotateRequest(
          run_id="01K6Z…",
          schema_name="eval.quality",
          target_event_id="01K71…",
          payload_json=json.dumps(
              {"score": 0.2, "outcome": "fail", "annotator": "human", "note": "hallucinated the API name"}
          ),
      ),
  )
  print(resp.event_id)
  ```
</CodeGroup>

The response carries the minted `event_id` — the annotation's stable dedup and lookup key. The
`Annotate` request body is:

| Field             | Meaning                                                                                                                                                                       |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `run_id`          | The run the annotation belongs to. Exactly one of `run_id` / `project_id`.                                                                                                    |
| `project_id`      | The project a run-less annotation belongs to (see [Annotate a project](#annotate-a-project)).                                                                                 |
| `schema_name`     | The registered schema the payload validates against; becomes the event `name`.                                                                                                |
| `target_event_id` | The event this annotation is about. Omit for a run/session-level annotation. Run targets only.                                                                                |
| `payload_json`    | The annotation body — a JSON object (scalars, nested objects, and arrays), stored verbatim as the event's `attributes_json` and validated against the schema. Entirely yours. |

## Annotate a range

A range annotation targets a window instead of a single event — useful for "this whole retry loop
was slow" or "the agent went off-task between these timestamps". Pass `--range <start>..<end>`,
where both endpoints are times — an RFC 3339 timestamp or a raw wall-clock nanosecond value
(exactly what a query's `ts_wall_ns` column returns) — or both are event ids, in which case the
window spans those two events' recorded timestamps:

```sh theme={null}
hiloop annotations add \
  --run 01K6Z… \
  --schema review.span \
  --range 2026-07-03T10:14:22Z..2026-07-03T10:15:40Z \
  --data '{"label":"off-task"}'

# The same window, bounded by the first and last event of the loop under review:
hiloop annotations add \
  --run 01K6Z… \
  --schema review.span \
  --range 01K71DXPZ3TQJ8FJQ4RG9BC2NM..01K71E2M4YV0S8BW3T0V0V5H8K \
  --data '{"label":"off-task"}'
```

An event-bounded range keeps the endpoint events linked on the annotation (so a reader can jump to
them) and stores the resolved timestamps alongside, so time-window queries work the same either
way. Both endpoint events must belong to the annotated run, in start-before-end order.

The `AnnotateRange` API is the same call, with `range_start_ns` / `range_end_ns` (or
`range_start_event_id` / `range_end_event_id`) in place of `target_event_id`.

## Annotate a project

A project-scoped annotation has **no run at all**: it belongs to the project itself. It is the home
for cross-run knowledge — promotion decisions, negative results, claims about the problem — that
must survive every sandbox and run. Pass `--project` (slug or id) instead of `--run`:

```sh theme={null}
hiloop annotations add \
  --project nano \
  --schema knowledge.v1 \
  --data '{"kind":"negative","claim":"GeLU hurts at this width","evidence_runs":["01K6Z…"]}'
```

Project annotations carry no lineage and no target event (`--target-event`/`--range` don't apply);
they are queryable like everything else, including through their schema-named view. Through the
API, set `project_id` instead of `run_id`.

## Re-scoring

Annotations are append-only and immutable: re-scoring writes a new event with a later timestamp. By
default a read returns the **latest** annotation per target and schema — the target is what you
annotated: the run, the event, or the range (its exact bounds) — so a re-score of the same target
supersedes the prior one, while annotations on different targets (say, two different ranges of the
same run) each stay current. Declaring an `identity` field (e.g. `annotator`) refines that key so the
latest write *per annotator* is kept — a human correction and an LLM judge on the same event both
survive, each at their latest. Identity is keyed by the field's **value in each record's payload**,
so it applies across schema versions: records written before a version bump that declared the field
fold into the same latest-wins chain as post-bump writes. Nothing is mutated or hidden: every
version stays readable.

<Note>
  **Distinct records need an identity field — or every write supersedes the last.** With no identity
  field declared, the supersession key is only the annotated target, so two same-schema writes to the
  same run (or event, or range) are treated as versions of *one* record and the default read serves
  just the newest. If your rows are separate records — per-worker results, append-only measurements —
  declare the field(s) that tell them apart as `identity` (a `lane`, a `segment`, an `annotator`) so
  they coexist. Superseded rows are never lost: the default listing reports how many it hid, and
  `--history` returns them all — in SQL, through the `ann_<schema>_history` view.
</Note>

## List annotations

`hiloop annotations list` (or `GET /v1/telemetry/annotations`) is the served read implementing
those semantics:

```sh theme={null}
# The run's own current annotations (a re-score shows once, at its latest):
hiloop annotations list --run 01K6Z…

# Include every stored version, newest first, with superseded ones marked:
hiloop annotations list --run 01K6Z… --history

# One schema only (declared identity fields refine the latest-wins key either way):
hiloop annotations list --run 01K6Z… --schema eval.quality

# A project's annotations — the rollup: run-less project annotations plus run-anchored
# annotations from the project's runs, with a SCOPE column naming each row's anchor:
hiloop annotations list --project nano
```

Each row carries the annotation's anchor (`run_id`, its lineage position; both absent on a
run-less project annotation), the schema `name`, the target (`target_event_id`, or the range
bounds plus the endpoint event ids for an event-bounded range), the writing `principal`,
`ts_wall_ns`, and your payload under `payload_json`. On the HTTP API the payload is a raw JSON
object **string** — the exact bytes you annotated, so payload values of every JSON type (including
64-bit integers, which plain JSON numbers cannot carry past 2^53) read back unchanged. The CLI's
`--output json` prints the full `{"annotations": […]}` envelope with each payload spliced back in
as a real nested JSON object (same bytes, same fidelity — no double-decoding needed):

```sh theme={null}
hiloop annotations list --run 01K6Z… --output json | jq '.annotations[].payload_json.score'
```

With `--history`, each JSON row also carries a `status` field — `"current"` or `"superseded"` —
the same marking the table's STATUS column shows, so scripts never re-derive latest-wins
themselves.

<Note>
  **`status` is per supersession key, not per payload value.** A row reads `superseded` when a newer
  write shares its whole key: the anchor scope, the schema, and the target, refined by the schema's
  declared `identity` fields. If the schema declares no identity field, every same-scope write
  belongs to one chain, so a standing row can read `superseded` even though no newer row carries its
  distinguishing payload value (a `fingerprint`, a `lane`): the write that superseded it is simply a
  different record in the same chain. Before treating `status` as a per-record fact (for example,
  verifying a standing registry row is still current before acting on it), make sure the field that
  names the record is declared `:identity` on the schema. The declaration applies retroactively:
  identity is keyed by each row's stored payload value, so existing rows re-read under the refined
  key without a rewrite.
</Note>

When the current view hides superseded versions, the listing says so rather than letting a write
silently disappear: the CLI prints `note: N superseded annotation row(s) hidden … use --history` on
stderr, and the API response carries the count as `superseded_count`.

### Change a few fields of a record: read-modify-resend

A re-annotation replaces the payload wholesale — there is no partial update. When a record moves
through states (a queue item going `queued → submitting → scored`, say) and a transition changes
only a couple of fields, read the record's current version, merge your changes into it, and resend
the whole payload, so every untouched field carries forward instead of silently dropping:

```sh theme={null}
# 1. Read the current version (filter on the field that identifies your record):
current=$(hiloop annotations list --project nano --schema submission.v1 --output json \
  | jq -c '.annotations[] | select(.payload_json.candidate_id == "cand-42") | .payload_json')

# 2. Merge only the fields that change (shallow merge; everything else carries forward):
next=$(jq -c '. + {"state": "scored", "lb": 0.712}' <<<"$current")

# 3. Resend — the merged payload becomes the record's current version:
hiloop annotations add --project nano --schema submission.v1 --data "$next"
```

The same pattern works run-anchored (`--run` in both commands). Because annotations are
append-only, the pre-transition version stays in `--history` — a bad merge is always recoverable
from the prior version.

## Query annotations back

Registering a schema auto-creates a **schema-named view**: `eval.quality` is queryable as the table
`ann_eval_quality` (the name is the schema name lowercased, with every non-alphanumeric run mapped
to `_`, behind an `ann_` prefix). The view serves the **current** annotations — the same
latest-wins read as `annotations list`: the newest version per annotated target, refined by the
schema's declared `identity` fields — so SQL and the served listing agree about what the registry
contains. Every stored version, superseded ones included, stays queryable through the full-history
sibling `ann_eval_quality_history` — the SQL counterpart of `--history`. Registration alone creates
both; a new schema version refreshes them, and archiving the schema retires them.

A view's payload-derived columns are **the promoted fields** — each under its declared name, backed
by its typed column, so filters and aggregates get the full columnar acceleration. Alongside them
every view carries the annotation's context columns (`event_id`, `run_id`, `root_run_id`,
`lineage_path`, `project_id`, `principal`, `ts_wall_ns`, and the point/range target columns) and a
`payload_json` column holding the whole payload as JSON text. A field you didn't promote is not a
column — selecting it is an unknown-column error — but it stays reachable through the payload
(promote it if you filter or aggregate on it often):

```sh theme={null}
hiloop query --sql "
  SELECT lineage_path, outcome, AVG(score) AS avg_score
  FROM ann_eval_quality
  WHERE run_id = '01K6Z…'
  GROUP BY lineage_path, outcome"

# A non-promoted field, through the payload column:
hiloop query --sql "
  SELECT hiloop_json_get(payload_json, 'note') AS note
  FROM ann_eval_quality
  WHERE run_id = '01K6Z…'"
```

A declared field name that collides with a SQL keyword works quoted (`GROUP BY \"group\"`), and one
that collides with a context column (or `payload_json`) is surfaced as `field_<name>`. The `ann_`
prefix is reserved: you can't store your own data view under it.

Annotations are still ordinary events, so the raw form works too — filter `events` by
`signal = 'annotation'` and the schema name. The raw `attributes_json` carries your payload plus
platform-stamped metadata; reach into it with `hiloop_json_get(attributes_json, '$.path.to.key')`:

```sh theme={null}
hiloop query --sql "
  SELECT hiloop_json_get(attributes_json, '\$.note') AS note
  FROM events
  WHERE run_id = '01K6Z…' AND signal = 'annotation' AND name = 'eval.quality'"
```

## Next

* [Query telemetry](/observability/query-telemetry) — the SQL query that reads annotations back.
* [Compare runs](/guides/querying-telemetry#compare-two-runs) — compare annotated outcomes with SQL.
* [Saved & data views](/observability/data-views) — save an annotation query as a reusable view.