# CI Integration (/docs/ci)



Whether you use the [GitHub Action](/docs/github-action) or run the
[CLI](/docs/cli) directly, `agentiqa run` exposes a **governed, versioned
contract** for CI: a fixed exit-code model and a machine-readable JSON envelope.

To choose *which* plans a pipeline runs and *how* they execute — a labeled subset,
sequential or parallel — see [Labels](/docs/guides/labels) and
[Parallel runs](/docs/cli#parallel-vs-sequential). This page covers how to **gate**
on and **parse** whatever you run.

## Exit codes [#exit-codes]

The CLI exits with a status that gates your job automatically. It's a fixed,
governed contract, so you can rely on it:

| Code | Meaning                                                                                                                                                                                 |
| ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `0`  | Success — all selected plans passed (or there was nothing to run).                                                                                                                      |
| `1`  | **Plan failure** — plans executed, at least one failed. A real product-quality signal.                                                                                                  |
| `2`  | **Usage / configuration error** — bad flags, not authenticated, a selector that matched no plans, **or a quota / plan-limit block** (account state — retrying won't help). Nothing ran. |
| `3`  | **Infra / runtime error** — engine unreachable, an auth failure, or an unexpected error. Nothing reached a verdict, so it's safe for CI to retry.                                       |

The `1` vs `3` split is deliberate and stable: `1` is a genuine test failure you
should investigate; `3` is a transient/infra problem that is safe to retry. A gate
can retry on `3` without masking regressions:

```yaml
- name: Run test plans (retry only on infra/runtime errors)
  run: |
    for attempt in 1 2 3; do
      npx -y agentiqa@latest run --engine https://engine.agentiqa.com
      code=$?
      # 0 = pass, 1 = plan failure (do NOT retry), 2 = usage error (do NOT retry)
      [ "$code" -ne 3 ] && exit "$code"
      echo "Infra/runtime error (exit 3) on attempt $attempt — retrying…"
      sleep 15
    done
    exit 3
```

## JSON envelope [#json-envelope]

Add `--json` (or set `AG_OUTPUT=json`) to emit exactly **one JSON document on
stdout**. Every document carries `"schemaVersion": 1` at the top level; a breaking
change to the shape bumps that number.

* **All logs go to stderr.** stdout is only the JSON document — no banners, no
  progress, no ANSI color. `stdout | jq` is always safe.
* **The exit code is independent of the envelope** — always branch on the exit
  code for pass/fail; use the JSON for detail.

**Success** (`ok: true`):

```json
{
  "ok": true,
  "schemaVersion": 1,
  "outcome": "passed",
  "plans": [
    {
      "title": "Checkout flow",
      "outcome": "passed",
      "durationSec": 42,
      "exitCode": 0,
      "runUrl": "https://web.agentiqa.com/projects/proj_…/test-plans-v2/tp_…/history/run_…",
      "videoUrl": "https://assets.agentiqa.com/e2e-videos/run-…/checkout-flow.mp4"
    },
    { "title": "Login", "outcome": "failed", "durationSec": 18, "exitCode": 1, "summary": "…" }
  ]
}
```

`outcome` is `"passed"` only when every plan passed, otherwise `"failed"`. Each
`plans[]` entry carries a per-plan `outcome`, `durationSec`, `exitCode`, and an
optional `summary`. When artifacts are captured, an entry may also carry `runUrl`,
`videoUrl`, `videoPath`, `artifactDir`, and (with `--share`) a public `shareUrl` —
each present **only** when available (omitted, never `null`, so `schemaVersion`
stays `1`).

**Failure** (`ok: false`) — emitted for usage errors and thrown infra/runtime
errors:

```json
{ "ok": false, "schemaVersion": 1, "error": { "code": "run_error", "message": "…" } }
```

## Gate on the exit code, extract detail with jq [#gate-on-the-exit-code-extract-detail-with-jq]

```yaml
- name: Run test plans (JSON)
  run: |
    npx -y agentiqa@latest run --engine https://engine.agentiqa.com --json > result.json
    code=$?
    jq -r '.plans[] | "\(.outcome)\t\(.title)"' result.json || cat result.json
    exit "$code"
```

Because logs are on stderr, `> result.json` captures only the envelope; the
human-readable run log still streams to the console.

## Action outcome buckets [#action-outcome-buckets]

The [GitHub Action](/docs/github-action) maps these exits to `outcome` buckets for
gating: `0` (with a valid envelope and ≥1 plan) → `passed`; `0` with zero plans or
no valid envelope → `config-error`; `1` → `plan-failure`; `2`/unknown →
`config-error`; `3` → `infra-error`. `fail-on: plan-failure` (default) fails on
plan failures and config errors and swallows retryable infra errors; `fail-on: any`
gates on anything nonzero.