> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hiloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Sandbox troubleshooting

> Diagnose sandbox creation, command execution, workspace, and cleanup issues.

Start with the sandbox itself, then its lifecycle telemetry.

## Inspect the sandbox

```sh theme={null}
hiloop sandbox get "$SANDBOX_NAME" --output json
```

The `state` field is the observed lifecycle state: `requested`, `reserved`, `materializing`,
`ready`, `running`, `stopped`, `terminating`, `failed`, `terminated`, `quarantined`, or
`attention`. A sandbox that reached `failed` or `quarantined` carries a `state_reason` explaining
why; states that need no explanation omit it. The record also reports the resolved `image`,
`storage_class`, `resources`, and `endpoints`.

## Inspect a command's outcome

There is no separate execution resource to fetch. `hiloop sandbox exec` returns the command's real
exit code with stdout and stderr separated, so read them from the invocation itself:

```sh theme={null}
hiloop sandbox exec "$SANDBOX_NAME" --output json -- ./run-tests.sh
```

The JSON carries `stdout`, `stderr`, `exit`, and `truncated`. A `truncated` of `true` means output
hit the server's combined output cap, not that the command failed. A command that exceeded its
timeout returns partial output with no exit code.

## Common causes

| Symptom                    | Check                                                                                                |
| -------------------------- | ---------------------------------------------------------------------------------------------------- |
| Create fails               | Image reference, resource request, requested capabilities, project id.                               |
| Execute fails              | Program path, arguments, working directory, timeout, environment.                                    |
| Workspace continuity fails | The storage class the sandbox was created with, and whether the work was written under `/workspace`. |
| Delete appears slow        | Poll the delete operation; cleanup is asynchronous.                                                  |

## Correlate with telemetry

Every project-backed sandbox records
[`runtime` lifecycle events](/observability/event-model#runtime-platform-lifecycle) under its run
(the `run_id` returned by sandbox create). That platform timeline is independent of workload
capture and answers the questions above without tailing anything:

```sh theme={null}
hiloop query --run-id "$RUN_ID" --signal runtime --output json
```

* **Sandbox slow to start?** Read `queue_wait_ms` on `operation.started` — it separates time spent
  queued from time spent provisioning.
* **What failed, exactly?** `operation.failed` carries a stable `error.code` alongside the
  operation id you're polling.
* **Who stopped my sandbox?** `sandbox.state_changed` records each committed transition
  (`state.from` → `state.to`); `sandbox.preempted` marks an eviction the platform observed but did
  not request.

Query the same ambient run id across all signals for captured entrypoint, exec, SSH, HTTP, and OTLP
activity:

```sh theme={null}
hiloop query --run-id "$RUN_ID" --limit 50 --output json
```

Use the execution response as the source of truth for a buffered command's output and exit status;
captured telemetry is the correlated, queryable record and is best effort by design.