> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hiloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Run an autonomous research loop

> Give a coding agent a fixed dataset and scorer, then watch it propose, test, annotate, ensemble, and rank ideas.

Run one interactive coding agent as the research orchestrator. You supply a task, a fixed dataset,
and a scorer. The agent proposes diverse ideas, executes bounded experiments, records every result
as a structured annotation, ensembles the winners, and leaves a queryable leaderboard.

The presenter or operator does not steer individual experiments. Watch the project through the
dashboard and read-only CLI commands while the agent works.

## Before you start

Install the released CLI and the public skills bundle for Claude Code:

```sh theme={null}
curl -fsSL https://hiloop.ai/install.sh | sh
hiloop skills install claude-code --ref v0.3.1
hiloop --version
```

This guide requires the autoresearch skill in bundle v0.3.1 or later. Restart Claude Code after
installing the skills. Authenticate the host and verify the organization:

```sh theme={null}
export HILOOP_API_URL="${HILOOP_API_URL:-https://api.hiloop.ai}"
hiloop login
hiloop whoami
```

For an unattended run, inject an organization-scoped `HILOOP_API_KEY` into the trusted host orchestrator
through your secret manager instead. Bundle v0.3.1 launches every local experiment under a clean,
allowlisted environment, so candidate scripts do not inherit that key, model-provider keys, cloud
credentials, or the agent's other environment variables. Preserve that isolation if you replace
the supplied runner. Do not copy a credential into the dataset, experiment scripts, annotations, or
sandboxes.

## Prepare the research directory

Start with a clean directory containing:

* `TASK.md`: the objective, optimization direction, time budget, allowed dependencies, and the
  documented dataset/scorer interfaces;
* the fixed dataset or a loader that exposes training and holdout features;
* a scorer that is the only code allowed to inspect holdout labels and emits one machine-readable
  `HILOOP_METRIC` line per scored experiment;
* a pinned environment, such as `pyproject.toml` and `uv.lock`;
* no previous predictions, experiment scripts, or result summaries.

The agent must train only on the training split. It copies metric values from the scorer verbatim.
A result marked invalid cannot win, and a failed idea stays recorded as failed.

## Choose or create the project

Use a quiet project so the live views contain only this loop:

```sh theme={null}
export HILOOP_PROJECT=autoresearch-$(date -u +%Y%m%d)
hiloop projects get "$HILOOP_PROJECT" >/dev/null 2>&1 || \
  hiloop projects create "$HILOOP_PROJECT" --description "Autonomous research loop"
```

The autoresearch skill checks the organization-wide annotation schemas before it writes anything. It
registers a missing schema once, but never re-registers an existing name. In particular,
`hiloop.idea` must promote `headline:str:identity`; changing that identity or promotion layout can
misread historical view columns. An incompatible existing schema is a hard stop, not a reason to
create another version.

## Start the interactive orchestrator

From the research directory, run Claude interactively inside one captured hiloop run:

```sh theme={null}
hiloop run \
  --project "$HILOOP_PROJECT" \
  --label autoresearch \
  --net-capture proxy \
  -- claude
```

If Claude asks whether you trust the new directory, confirm it before pasting the task. Then paste:

```text theme={null}
Read https://docs.hiloop.ai/guides/run-autonomous-research-loop and use the installed autoresearch
skill. The fixed task, dataset, scorer, and pinned environment are in this directory. Run the whole
research loop autonomously: reuse this hiloop project, check schemas before registering, propose
seven diverse ideas before testing, run the proven local experiment path, record every idea status
and scored experiment, ensemble the best decorrelated winners, and finish with the leaderboard and
a concise evidence-backed summary. Copy scorer metrics verbatim, preserve failed ideas, never read
holdout labels, run every local experiment under the skill's clean environment allowlist, never
expose credentials, do not use the sandboxed-arms preview, and do not ask me questions. Time budget:
12 minutes; stop new experiments with two minutes left.
```

The current verified workflow runs experiment scripts in this captured host working directory. The
public skill also contains a clearly gated sandboxed-arms preview. Use that preview only after your
operator confirms a full rehearsal is green; do not label a local result as sandboxed or switch
execution modes during a run.

## Watch without steering

Copy the run id from the wrapper's `hiloop: recorded run …` line. These commands are read-only.

See whether sandbox arms exist:

```sh theme={null}
hiloop sandbox list
```

An empty list is expected for the verified local path. A rehearsed sandboxed path shows its prepared
fleet here.

Follow events as they arrive:

```sh theme={null}
hiloop runs tail <run-id>
```

See the runs the loop has registered, in-flight ones first:

```sh theme={null}
hiloop runs list --project "$HILOOP_PROJECT"
```

Watch idea cards move from `proposed` to `testing` and then `worked` or `failed`:

```sh theme={null}
hiloop annotations list --project "$HILOOP_PROJECT" --schema hiloop.idea
```

The leaderboard examples below assume a lower-is-better metric. Use `DESC` instead of `ASC` for a
higher-is-better metric. Rank the current idea cards:

```sh theme={null}
hiloop query --project "$HILOOP_PROJECT" --sql \
  "SELECT headline, status, score, outcome FROM ann_hiloop_idea ORDER BY score ASC"
```

Inspect every scored execution, including regressions:

```sh theme={null}
hiloop query --project "$HILOOP_PROJECT" --sql \
  "SELECT experiment_id, hiloop_json_get(payload_json, 'headline') AS experiment, lane, score, outcome FROM ann_demo_experiment_v1 ORDER BY score ASC"
```

## Run the fleet dashboard

The public [hiloop skills repository](https://github.com/hiloopai/skills) ships a terminal dashboard
with its `rich` dependency declared inline:

```sh theme={null}
git clone https://github.com/hiloopai/skills.git hiloop-skills
cd hiloop-skills
uv run tools/fleet-dashboard/dashboard.py \
  --project "$HILOOP_PROJECT" \
  --schema demo.experiment.v1 \
  --direction lower
```

Use `--direction higher` for a higher-is-better metric. Run once with `--once` before the session to
verify project access and the schema view. `Ctrl-C` stops only the dashboard; the research run keeps
going.

## Verify the result

A complete loop has:

* seven idea cards, with every tested card terminal and untested cards honestly still proposed;
* one immutable experiment annotation for every scored execution;
* the baseline, valid improvements, regressions, invalid results, and failures preserved;
* an ensemble scored through the same fixed scorer;
* a final summary whose numbers exactly match annotations;
* a succeeded run you can reopen with `hiloop runs show <run-id>`.

Do not treat a single successful arm or a populated dashboard as proof of the whole loop. Verify the
annotations, experiment rows, ensemble, and terminal run together.