Skip to main content
Run one interactive coding agent as the research orchestrator. You supply a task, a fixed dataset, and a scorer. The agent proposes diverse ideas, executes bounded experiments, records every result as a structured annotation, ensembles the winners, and leaves a queryable leaderboard. The presenter or operator does not steer individual experiments. Watch the project through the dashboard and read-only CLI commands while the agent works.

​
Before you start

Install the released CLI and the public skills bundle for Claude Code:
This guide requires the autoresearch skill in bundle v0.3.1 or later. Restart Claude Code after installing the skills. Authenticate the host and verify the organization:
For an unattended run, inject an organization-scoped HILOOP_API_KEY into the trusted host orchestrator through your secret manager instead. Bundle v0.3.1 launches every local experiment under a clean, allowlisted environment, so candidate scripts do not inherit that key, model-provider keys, cloud credentials, or the agent’s other environment variables. Preserve that isolation if you replace the supplied runner. Do not copy a credential into the dataset, experiment scripts, annotations, or sandboxes.

​
Prepare the research directory

Start with a clean directory containing:
  • TASK.md: the objective, optimization direction, time budget, allowed dependencies, and the documented dataset/scorer interfaces;
  • the fixed dataset or a loader that exposes training and holdout features;
  • a scorer that is the only code allowed to inspect holdout labels and emits one machine-readable HILOOP_METRIC line per scored experiment;
  • a pinned environment, such as pyproject.toml and uv.lock;
  • no previous predictions, experiment scripts, or result summaries.
The agent must train only on the training split. It copies metric values from the scorer verbatim. A result marked invalid cannot win, and a failed idea stays recorded as failed.

​
Choose or create the project

Use a quiet project so the live views contain only this loop:
The autoresearch skill checks the organization-wide annotation schemas before it writes anything. It registers a missing schema once, but never re-registers an existing name. In particular, hiloop.idea must promote headline:str:identity; changing that identity or promotion layout can misread historical view columns. An incompatible existing schema is a hard stop, not a reason to create another version.

​
Start the interactive orchestrator

From the research directory, run Claude interactively inside one captured hiloop run:
If Claude asks whether you trust the new directory, confirm it before pasting the task. Then paste:
The current verified workflow runs experiment scripts in this captured host working directory. The public skill also contains a clearly gated sandboxed-arms preview. Use that preview only after your operator confirms a full rehearsal is green; do not label a local result as sandboxed or switch execution modes during a run.

​
Watch without steering

Copy the run id from the wrapper’s hiloop: recorded run … line. These commands are read-only. See whether sandbox arms exist:
An empty list is expected for the verified local path. A rehearsed sandboxed path shows its prepared fleet here. Follow events as they arrive:
See the runs the loop has registered, in-flight ones first:
Watch idea cards move from proposed to testing and then worked or failed:
The leaderboard examples below assume a lower-is-better metric. Use DESC instead of ASC for a higher-is-better metric. Rank the current idea cards:
Inspect every scored execution, including regressions:

​
Run the fleet dashboard

The public hiloop skills repository ships a terminal dashboard with its rich dependency declared inline:
Use --direction higher for a higher-is-better metric. Run once with --once before the session to verify project access and the schema view. Ctrl-C stops only the dashboard; the research run keeps going.

​
Verify the result

A complete loop has:
  • seven idea cards, with every tested card terminal and untested cards honestly still proposed;
  • one immutable experiment annotation for every scored execution;
  • the baseline, valid improvements, regressions, invalid results, and failures preserved;
  • an ensemble scored through the same fixed scorer;
  • a final summary whose numbers exactly match annotations;
  • a succeeded run you can reopen with hiloop runs show <run-id>.
Do not treat a single successful arm or a populated dashboard as proof of the whole loop. Verify the annotations, experiment rows, ensemble, and terminal run together.