# Agent skill (/docs/agent-skill)
Agentiqa ships a **user-facing agent skill** — a compact instruction set that
teaches an AI coding agent (Claude Code and compatible agents) how to use Agentiqa
across all four surfaces: the CLI, the web app, the desktop app, and the GitHub
Action. It uses progressive disclosure: a short `SKILL.md` with the essentials,
bundled reference files for the CLI, service keys, the JSON envelope, exit codes,
and quickstarts, and a delegation to this site's [`/llms-full.txt`](/llms-full.txt)
for anything not bundled.
## What it is (and what it isn't) [#what-it-is-and-what-it-isnt]
This is a **Claude Code skill** — a `SKILL.md` plus reference files. It teaches an
agent **how to drive Agentiqa**: which surface to pick, the `agentiqa explore` /
`agentiqa run` commands, the GitHub Action, the JSON envelope, and the exit-code
contract.
It ships as part of the **Agentiqa plugin**
([`Agentiqa/agentiqa-plugin`](https://github.com/Agentiqa/agentiqa-plugin)), which
contains two skills:
* **`agentiqa-test`** — runs Agentiqa against your app from inside the coding
agent (explore, report bugs with reproduction steps and media);
* **`agentiqa`** (this skill) — the usage reference: CLI commands and selectors,
the GitHub Action, service keys, the JSON envelope, exit codes.
It is **not**:
* **an MCP server** — it exposes no tools and runs no process;
* **the Agentiqa product** — the skill contains only instructions. The actual
testing is done by the tools it points at: the `agentiqa` **CLI** (published on
npm) and the `agentiqa/qa-action` **GitHub Action**.
## Install [#install]
Claude Code:
```
/plugin marketplace add Agentiqa/agentiqa-plugin
/plugin install agentiqa@agentiqa
```
Cursor: `/add-plugin https://github.com/Agentiqa/agentiqa-plugin` · Codex CLI:
`codex plugin marketplace add Agentiqa/agentiqa-plugin`
The agent loads the skill on demand when you ask it to test an app with Agentiqa,
run `agentiqa` in CI, or parse an Agentiqa result. No configuration is required.
## What it covers [#what-it-covers]
* **Which surface to use** — CLI/Action for automation, the apps for interactive QA.
* **The two CLI commands** — `agentiqa explore` (agent-led discovery) and `agentiqa
run` (deterministic saved-plan replay), plus engine modes and plan selection.
* **The CI contract** — the 4-code exit model and the `schemaVersion: 1` JSON
envelope, so an agent can gate and parse results correctly.
* **The GitHub Action** — `agentiqa/qa-action@v1` setup.
## Consuming the docs directly [#consuming-the-docs-directly]
Any agent (with or without the skill) can read these docs as plain text:
* [`/llms.txt`](/llms.txt) — a machine-readable index of every page.
* [`/llms-full.txt`](/llms-full.txt) — the entire docs corpus as one document.
* Append `.md` to any docs URL for that page as markdown (e.g.
[`/docs/ci.md`](/docs/ci.md)).
Because the CLI and Action references are generated from the code and the prose is
audited nightly, the skill delegates to `/llms-full.txt` for freshness — when the
bundled references and the live docs disagree, the live docs win.
# CI Integration (/docs/ci)
Whether you use the [GitHub Action](/docs/github-action) or run the
[CLI](/docs/cli) directly, `agentiqa run` exposes a **governed, versioned
contract** for CI: a fixed exit-code model and a machine-readable JSON envelope.
To choose *which* plans a pipeline runs and *how* they execute — a labeled subset,
sequential or parallel — see [Labels](/docs/guides/labels) and
[Parallel runs](/docs/cli#parallel-vs-sequential). This page covers how to **gate**
on and **parse** whatever you run.
## Exit codes [#exit-codes]
The CLI exits with a status that gates your job automatically. It's a fixed,
governed contract, so you can rely on it:
| Code | Meaning |
| ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `0` | Success — all selected plans passed (or there was nothing to run). |
| `1` | **Plan failure** — plans executed, at least one failed. A real product-quality signal. |
| `2` | **Usage / configuration error** — bad flags, not authenticated, a selector that matched no plans, **or a quota / plan-limit block** (account state — retrying won't help). Nothing ran. |
| `3` | **Infra / runtime error** — engine unreachable, an auth failure, or an unexpected error. Nothing reached a verdict, so it's safe for CI to retry. |
The `1` vs `3` split is deliberate and stable: `1` is a genuine test failure you
should investigate; `3` is a transient/infra problem that is safe to retry. A gate
can retry on `3` without masking regressions:
```yaml
- name: Run test plans (retry only on infra/runtime errors)
run: |
for attempt in 1 2 3; do
npx -y agentiqa@latest run --engine https://engine.agentiqa.com
code=$?
# 0 = pass, 1 = plan failure (do NOT retry), 2 = usage error (do NOT retry)
[ "$code" -ne 3 ] && exit "$code"
echo "Infra/runtime error (exit 3) on attempt $attempt — retrying…"
sleep 15
done
exit 3
```
## JSON envelope [#json-envelope]
Add `--json` (or set `AG_OUTPUT=json`) to emit exactly **one JSON document on
stdout**. Every document carries `"schemaVersion": 1` at the top level; a breaking
change to the shape bumps that number.
* **All logs go to stderr.** stdout is only the JSON document — no banners, no
progress, no ANSI color. `stdout | jq` is always safe.
* **The exit code is independent of the envelope** — always branch on the exit
code for pass/fail; use the JSON for detail.
**Success** (`ok: true`):
```json
{
"ok": true,
"schemaVersion": 1,
"outcome": "passed",
"plans": [
{
"title": "Checkout flow",
"outcome": "passed",
"durationSec": 42,
"exitCode": 0,
"runUrl": "https://web.agentiqa.com/projects/proj_…/test-plans-v2/tp_…/history/run_…",
"videoUrl": "https://assets.agentiqa.com/e2e-videos/run-…/checkout-flow.mp4"
},
{ "title": "Login", "outcome": "failed", "durationSec": 18, "exitCode": 1, "summary": "…" }
]
}
```
`outcome` is `"passed"` only when every plan passed, otherwise `"failed"`. Each
`plans[]` entry carries a per-plan `outcome`, `durationSec`, `exitCode`, and an
optional `summary`. When artifacts are captured, an entry may also carry `runUrl`,
`videoUrl`, `videoPath`, `artifactDir`, and (with `--share`) a public `shareUrl` —
each present **only** when available (omitted, never `null`, so `schemaVersion`
stays `1`).
**Failure** (`ok: false`) — emitted for usage errors and thrown infra/runtime
errors:
```json
{ "ok": false, "schemaVersion": 1, "error": { "code": "run_error", "message": "…" } }
```
## Gate on the exit code, extract detail with jq [#gate-on-the-exit-code-extract-detail-with-jq]
```yaml
- name: Run test plans (JSON)
run: |
npx -y agentiqa@latest run --engine https://engine.agentiqa.com --json > result.json
code=$?
jq -r '.plans[] | "\(.outcome)\t\(.title)"' result.json || cat result.json
exit "$code"
```
Because logs are on stderr, `> result.json` captures only the envelope; the
human-readable run log still streams to the console.
## Action outcome buckets [#action-outcome-buckets]
The [GitHub Action](/docs/github-action) maps these exits to `outcome` buckets for
gating: `0` (with a valid envelope and ≥1 plan) → `passed`; `0` with zero plans or
no valid envelope → `config-error`; `1` → `plan-failure`; `2`/unknown →
`config-error`; `3` → `infra-error`. `fail-on: plan-failure` (default) fails on
plan failures and config errors and swallows retryable infra errors; `fail-on: any`
gates on anything nonzero.
# Desktop App (/docs/desktop-app)
The Agentiqa desktop app is an Electron application with a **built-in headed
browser**. It has the complete capability set and lets you watch the agent drive
the browser in real time. It can test both public URLs and apps running on
`localhost`.
## Install [#install]
The desktop app is distributed as a signed build for macOS (and other desktop
platforms). Download the latest release from
[agentiqa.com](https://agentiqa.com), install it, and sign in with your account.
{/* verify: confirm the user-facing desktop download page/link and the supported OS list (build+publish flow is release-desktop-*.yml → releases.agentiqa.com). */}
Releases are published to a manifest at `releases.agentiqa.com`; the app checks it
and shows an in-app update prompt when a newer build is available.
## Your first test [#your-first-test]
1. **Create a project** with your app's URL (a `http://localhost:…` URL works
here).
2. **Ask the Assistant** to test it. The built-in browser opens and the agent
begins exploring.
3. **Curate** the scope, plan, and findings checkpoints.
The feature set is shared with the web app — the [feature guides](/docs/guides/projects)
apply to both surfaces.
## Why desktop [#why-desktop]
The desktop app adds capabilities the browser SPA cannot provide because it runs
locally: browser-extension / wallet (web3) testing, local file-system access, and
native OS notifications. See [Desktop vs Web](/docs/desktop-vs-web) for the full
matrix.
# Desktop vs Web (/docs/desktop-vs-web)
The web app and the desktop app give you the **same features** — projects, the
Assistant, test plans, runs, issues, the dashboard, and settings all work the same way
in both. The difference comes down to a few capabilities that need to run on your
own machine, so they're available in the **desktop app** but not in the browser.
| What you can do | Desktop app | Web app |
| ----------------------------------------------------------------- | ----------- | ------- |
| Test browser-extension / wallet flows (e.g. Metamask, web3 dApps) | Yes | No |
| Save run video and artifacts to a folder on your computer | Yes | No |
| Get native desktop notifications (e.g. when a long run finishes) | Yes | No |
| Native window controls | Yes | No |
| Test an app running on `http://localhost` | Yes | No |
## Which one should I use? [#which-one-should-i-use]
* **Use the web app** to try Agentiqa with nothing to install and to test a public
URL — the browser runs in Agentiqa's cloud.
* **Use the desktop app** when you need to test a site on `http://localhost`, test
a browser extension or crypto wallet, or save run artifacts to your computer.
* **Use the [CLI](/docs/cli)** to run tests locally from a terminal or in CI.
Whichever you pick, everything in the [feature guides](/docs/guides/projects) works
the same.
> **A note on mobile:** Agentiqa tests **web apps**. Mobile-app testing was retired
> and is no longer supported.
{/* verify: mobileAutomation flag is still true for desktop in PlatformCapabilities but the product dropped mobile 2026-07-01 — kept out of the user table intentionally. */}
# Introduction (/docs)
Agentiqa is an AI-powered testing platform for **web applications**. An LLM agent
navigates a real browser, discovers what matters, executes test plans, and
reports issues with screenshots and reproduction steps.
## The model: the agent proposes, you curate [#the-model-the-agent-proposes-you-curate]
You provide a URL. The agent provides everything else — what to test, why it
matters, how to test it, and what looks broken. Your role is to approve, reject,
or redirect at three checkpoints:
* **Scope** — the areas the agent discovered, ranked by risk. Default: approve all.
* **Plan** — how each area will be tested. Default: approve all.
* **Findings** — bugs with evidence and repro steps. The only step that always
needs your review.
In headless contexts (CLI, CI, scheduled runs) every checkpoint auto-approves and
the agent uses its own judgment — same engine, same quality, no human in the loop.
## Where you run it [#where-you-run-it]
| Surface | Best for | Guide |
| ----------------- | -------------------------------------------------------------- | ------------------------------------ |
| **Web app** | Trying Agentiqa in the browser, cloud execution | [Web App](/docs/web-app) |
| **Desktop app** | The full experience with a built-in headed browser | [Desktop App](/docs/desktop-app) |
| **CLI** | Local runs and scripting (`agentiqa explore` / `agentiqa run`) | [CLI](/docs/cli) |
| **GitHub Action** | Running saved plans in CI and gating on the result | [GitHub Action](/docs/github-action) |
The web and desktop apps share the same feature set — the difference is a small
set of platform capabilities. See [Desktop vs Web](/docs/desktop-vs-web).
## Next steps [#next-steps]
* New here? Start with the [Quickstart](/docs/quickstart).
* Automating in CI? Read [CI Integration](/docs/ci) for the exit-code contract and
JSON envelope.
* For agents: this site serves [`/llms.txt`](/llms.txt) and
[`/llms-full.txt`](/llms-full.txt), and every page has a `.md` variant (append
`.md` to any docs URL).
# Quickstart (/docs/quickstart)
Pick the surface that fits how you want to work. All three drive the same agent.
## Web app [#web-app]
1. Open the [Agentiqa web app](https://web.agentiqa.com) and sign in.
2. Create a project by entering the **URL** of the app you want to test (and a
name).
3. Open the **Assistant** and ask for what you want — for example
*"Test the signup flow"* or *"What can you test here?"*
4. Review the agent's scope and plan checkpoints, then confirm the findings it
reports.
Cloud execution runs the browser on Agentiqa's infrastructure — nothing to install.
## Desktop app [#desktop-app]
1. Download the desktop app (see [Desktop App](/docs/desktop-app)) and sign in.
2. Create a project with your app's URL.
3. Ask the Assistant to test it. The desktop app drives a **built-in headed
browser**, so you can watch the agent work.
The desktop app has the full capability set — see [Desktop vs Web](/docs/desktop-vs-web).
## CLI [#cli]
Requires Node.js 18+ (Node 20 recommended).
```bash
# Explore a URL with the agent (in-process engine)
npx -y agentiqa@latest explore "Find bugs on this page" --url https://example.com
# Run saved test plans against the hosted cloud engine (CI-friendly)
AGENTIQA_SERVICE_KEY=sk_... npx -y agentiqa@latest run --engine https://engine.agentiqa.com
```
See the [CLI guide](/docs/cli) and the generated [CLI reference](/docs/cli/reference).
To run a labeled subset or run plans in parallel, see [Labels](/docs/guides/labels)
and [Parallel runs](/docs/cli#parallel-vs-sequential).
## GitHub Action [#github-action]
Run your saved plans in CI and gate the job on the outcome:
```yaml
- uses: agentiqa/qa-action@v1
with:
service-key: ${{ secrets.AGENTIQA_SERVICE_KEY }}
```
See the [GitHub Action guide](/docs/github-action) and
[CI Integration](/docs/ci).
# Troubleshooting (/docs/troubleshooting)
## CLI and CI [#cli-and-ci]
**`Node.js version too old` / install fails.** The CLI needs Node.js 18+ (Node 20
recommended). On Ubuntu CI, the distro default can be too old — use
`actions/setup-node@v4` or NodeSource. Always use `npx -y` in CI: bare `npx`
prompts "Ok to proceed?" on first install and hangs a non-interactive shell.
**LLM access denied / quota errors (cloud).** The account behind the service key
isn't entitled to the managed LLM, or has hit its limit. Upgrade to a plan that
includes managed LLM. A quota block
exits `2` (permanent — do not retry), not `3`.
**Exit `2` vs `3`.** `2` is a usage/configuration error (bad flags, not
authenticated, a selector that matched nothing) — nothing ran, and retrying won't
help. `3` is a transient infra/runtime error (engine unreachable, auth exchange
failure) — safe to retry. See [CI Integration](/docs/ci).
**A run "passed" but executed zero plans.** A run that executes zero plans is a
configuration error, not a pass — check that the service key's project has plans
and that your `--plan-id` / `--label-ids` selectors match. The
[GitHub Action](/docs/github-action) buckets zero-plan exit-0 runs as
`config-error` and fails them under the default policy.
## Web and desktop apps [#web-and-desktop-apps]
**Testing `localhost` doesn't work in the web app.** The web app uses the cloud
engine, which cannot reach your machine. Use the [desktop app](/docs/desktop-app)
or the [CLI](/docs/cli) in-process engine for local targets.
**A fix shipped but the web app still misbehaves.** The SPA shows a "New version
available — Reload" prompt when it detects a newer build; hard-reload the tab to
pick up a shipped fix.
{/* verify: confirm the exact copy/behavior of the web-app update banner. */}
**The desktop app isn't updating.** The app checks a release manifest and prompts
when a newer build is available — accept the prompt to update.
# Web App (/docs/web-app)
The Agentiqa web app runs entirely in your browser. When you start a test, the
browser being tested runs in **Agentiqa's cloud**, so there's nothing to install
and no setup — just open the app and go.
Open [web.agentiqa.com](https://web.agentiqa.com) and sign in.
{/* verify: confirm the canonical production web-app URL and sign-in entry point. */}
## Run your first test [#run-your-first-test]
1. **Create a project.** Enter the URL of the app you want to test and give the
project a name.
2. **Open the Assistant** and say what you want — from *"What can you test here?"*
to *"Test the checkout flow."* The agent figures out the rest (see
[the Assistant guide](/docs/guides/assistant)).
3. **Review the checkpoints.** Approve or redirect the agent's scope and plan, then
confirm the bugs it reports.
4. **Keep the results.** Your findings, saved test plans, and everything the agent
learns about your app carry over to your next visit.
The web app and the [desktop app](/docs/desktop-app) have the same features — the
[feature guides](/docs/guides/projects) apply to both.
## What the web app can't do [#what-the-web-app-cant-do]
A few things need to run on your own machine, so they're only in the desktop app:
testing browser extensions or crypto wallets, saving run files to your computer,
and testing a site running on `http://localhost`. See
[Desktop vs Web](/docs/desktop-vs-web).
To test a `localhost` app, use the [desktop app](/docs/desktop-app) or the
[CLI](/docs/cli).
# CLI overview (/docs/cli)
The `agentiqa` CLI runs the same agent as the apps, headless. It has two testing
commands — `explore` (agent-led discovery) and `run` (deterministic saved-plan
execution) — plus `login`, `logout`, and `whoami`. See the generated
[CLI reference](/docs/cli/reference) for every flag.
## Install [#install]
The CLI ships on npm and needs **Node.js 18+** (Node 20 recommended). There is
nothing to install ahead of time — `npx -y` fetches it on first use:
```bash
npx -y agentiqa@latest whoami
```
On Ubuntu CI, the distro's default `apt-get install nodejs` can be too old — use
`actions/setup-node@v4` or NodeSource to get Node 20+.
## explore vs run [#explore-vs-run]
* **`agentiqa explore ""`** — the agent explores a URL, proposes what to
test, and reports findings. Great for ad-hoc QA.
```bash
npx -y agentiqa@latest explore "Find bugs on the signup page" --url https://example.com
```
* **`agentiqa run`** — replays your saved test plans and returns deterministic
pass/fail. This is the command you wire into CI.
```bash
# Run all plans in the service key's project on the cloud engine
AGENTIQA_SERVICE_KEY=sk_... npx -y agentiqa@latest run --engine https://engine.agentiqa.com
```
## Where the engine runs [#where-the-engine-runs]
* **In-process (default).** With no `--engine`, the CLI starts an execution engine
locally: it downloads Chromium on first use and drives the browser on your
machine. Good for local targets (including `localhost`).
* **Hosted (`--engine `).** Sends the run to Agentiqa's cloud engine — no
Chromium on your machine. `AGENTIQA_SERVICE_KEY` alone authenticates; a
short-lived engine credential is minted automatically.
## Selecting which plans run [#selecting-which-plans-run]
With a service key, `agentiqa run` operates on that key's project. With **no
selector it runs every** non-deleted plan; narrow it with:
* `--plan-id tplan_…` — run a single saved plan.
* `--label-ids a,b,c` — run every plan tagged with **any** of those
[labels](/docs/guides/labels) (comma-separated `lbl_…` ids, OR match).
`--plan-id` overrides `--label-ids` when both are given, and a selector that
matches no plans is a configuration error (nonzero exit) — a run that tests
nothing is never a pass.
## Parallel vs sequential [#parallel-vs-sequential]
`--mode` controls how the selected plans execute. Each plan always runs as an
**independent engine session with its own browser**, so plans never share cookies
or state — `--mode` only changes ordering and concurrency.
* **`--mode sequential`** (default) — plans run one at a time. Run memory is
threaded from each plan into the next, and a transient per-plan engine
disconnect is retried on a fresh session, up to three attempts total.
* **`--mode parallel`** — plans run through a bounded worker pool to shrink
wall-clock time. By default, up to four plans run concurrently and remaining
plans start as workers become available. Each plan gets the same
engine-disconnect recovery as sequential mode: it is retried on a fresh
session, up to three attempts total.
Parallel has one deliberate caveat:
1. **No run-memory threading** — a plan that consumes an earlier plan's saved run
memory has no upstream memory in parallel and is reported as
dependency-blocked. Run dependent chains **sequentially**.
Prefer sequential when plans depend on each other; prefer parallel for an
independent regression suite where bounded concurrency shortens wall-clock time.
Both modes apply the same per-plan engine-disconnect retry.
## Authentication [#authentication]
Interactive use: `agentiqa login` (opens a browser). Unattended use (CI): set
`AGENTIQA_SERVICE_KEY` to a project-scoped service key — see
[CLI Service Keys](/docs/guides/cli-service-keys).
## In CI [#in-ci]
For running the CLI in GitHub Actions, prefer the [GitHub Action](/docs/github-action);
for the exit-code contract and JSON envelope you parse, see
[CI Integration](/docs/ci).
# CLI reference (/docs/cli/reference)
{/* GENERATED FILE — DO NOT EDIT BY HAND.
Source of truth: apps/cli/src/argSchema.ts
Regenerate: node_modules/.bin/tsx apps/docs/scripts/render-cli-reference.ts
A CI drift gate fails if this file is stale vs the schema. */}
Agentiqa CLI — AI-powered testing for web apps.
## Commands [#commands]
| Command | Summary |
| ------------------ | ------------------------------------------ |
| `agentiqa explore` | Test a web app with an AI agent |
| `agentiqa run` | Execute saved test plans |
| `agentiqa login` | Authenticate with Agentiqa (opens browser) |
| `agentiqa logout` | Remove stored credentials |
| `agentiqa whoami` | Show the current authenticated user |
## explore [#explore]
Test a web app with an AI agent.
`agentiqa explore "" [flags]`
| Flag | Description |
| ---------------------------- | --------------------------------------------------------------------------------------------------- |
| `--url ` | Web URL to test. Optional when logged in with a single project — the CLI reuses that project's URL. |
| `--feature ` | What was built, from the user's perspective. |
| `--hint ` | A specific thing to test. *(repeatable)* |
| `--known-issue ` | Something the agent should NOT report. *(repeatable)* |
| `--credential ` | A login credential to hand the agent. *(repeatable)* |
| `--dry-run` | Check the engine and exit without running the agent. |
| `--no-artifacts` | Don't save screenshots/video to the temp directory. |
| `--verbose` | Show raw observations and actions. |
| `--auto-approve` | Auto-approve scope and plan checkpoints (required for non-interactive runs). |
| `--json` | Emit machine-readable JSON on stdout (schemaVersion 1). |
| `--format ` | Explicit output format; overrides the AG\_OUTPUT env var. |
## run [#run]
Execute saved test plans.
`agentiqa run --url --plan [flags]`
| Flag | Description |
| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--url ` | Target URL (required with --plan; deprecated with a service key). |
| `--plan ` | Path to a test plan JSON (required without a service key). |
| `--plan-id ` | Run a single saved plan by id (service-key mode). |
| `--label-ids ` | Run every plan tagged with any of these labels (csv, service-key mode). |
| `--label-id ` | Alias for --label-ids. *(deprecated)* |
| `--mode ` | Execution order for the selected plans (default: sequential). |
| `--artifacts-dir ` | Directory for run artifacts (default: a temp directory). |
| `--no-artifacts` | Don't save video/frame artifacts. |
| `--share` | Mint an opt-in PUBLIC, revocable share link for each cloud run (added to the JSON envelope as shareUrl per plan) so a reviewer can open the run without an Agentiqa login. Cloud runs only; a no-op with a warning otherwise. Equivalent to AG\_SHARE=1. |
## login [#login]
Authenticate with Agentiqa (opens browser).
`agentiqa login [--no-browser]`
| Flag | Description |
| -------------- | ----------------------------------------------------------- |
| `--no-browser` | Don't open a browser — print the auth URL to visit instead. |
## logout [#logout]
Remove stored credentials.
`agentiqa logout`
*No command-specific flags.*
## whoami [#whoami]
Show the current authenticated user.
`agentiqa whoami`
*No command-specific flags.*
## Common flags [#common-flags]
Accepted by every command.
| Flag | Description |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--engine ` | Engine URL (default: auto-start in-process). When set, Playwright/Chromium and ffmpeg are not required locally — the remote engine drives the browser and CLI run artifacts download the engine-rendered video when available (a local ffmpeg render is only used as a fallback). Customer-facing usage authenticates via AGENTIQA\_SERVICE\_KEY alone (a short-lived engine credential is minted automatically). |
| `--help` | Show this help and exit. |
| `--version` | Print the CLI version and exit. |
## Environment variables [#environment-variables]
| Variable | Description |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `AGENTIQA_API_URL` | Override the Agentiqa control-plane API (default: [https://agentiqa.com](https://agentiqa.com)). |
| `AGENTIQA_SERVICE_KEY` | Service key for unattended runs (CI). Replaces interactive login and unlocks hosted engine access — no separate engine token required. |
| `AGENTIQA_ENGINE_TOKEN` | Optional internal bearer override for hosted engine HTTP + WebSocket calls. Only needed for internal infra; customer-facing CI should rely on AGENTIQA\_SERVICE\_KEY instead. |
| `AG_OUTPUT` | Set to "json" to enable JSON output mode globally (equivalent to --json). Use --format text to override. |
| `AG_SHARE` | Set to a truthy value (anything other than 0/false/empty) to enable public share links for cloud runs (equivalent to --share). |
| `AGENTIQA_UPDATE_CHECK` | Set to "0" to disable the "Update available" check (a cached, fire-and-forget npm dist-tags lookup; sends no user data). Also disabled by \{ "updateCheck": false } in \~/.agentiqa/config.json. |
## Exit codes [#exit-codes]
| Code | Meaning |
| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `0` | Success — all selected plans passed (or there was nothing to run). |
| `1` | Plan failure — plans executed, at least one failed. |
| `2` | Usage / configuration error (bad flags, not authenticated, a selector that matched no plans, or a project with no plans to run), OR a persistent account-state block (quota / org run-cap exhausted). Nothing ran — retrying cannot help. |
| `3` | Infra / runtime error (engine unreachable, auth/exchange failure, a no-verdict batch, or an unexpected internal error) — a correct, entitled invocation could not reach a verdict, so it is safe for CI to retry. |
## JSON output [#json-output]
With `--json` or `AG_OUTPUT=json`:
* Every JSON document includes "schemaVersion": 1 at the top level.
* Success: \{ "ok": true, "schemaVersion": 1, ...commandFields }
* Failure: \{ "ok": false, "schemaVersion": 1, "error": \{ "code": "...", "message": "..." } }
* All log lines go to stderr; stdout contains exactly one JSON document.
* ANSI color is disabled when --json is active or stdout is not a TTY.
# GitHub Action (/docs/github-action)
The Agentiqa composite GitHub Action runs your saved test plans in CI — nightly,
on a release branch, or on demand — and fails the job per a configurable
exit-code policy. It is a thin, versioned wrapper around the `agentiqa run` CLI:
it pins the argv, captures the machine-readable JSON envelope, always uploads
artifacts, and gates on the outcome so those details don't drift per-repo.
The public action is **`agentiqa/qa-action@v1`**.
## Quickstart [#quickstart]
1. **Mint a service key** — Project Settings → CLI Service Keys → Create Key (see
[CLI Service Keys](/docs/guides/cli-service-keys)). Copy the `sk_...` value; it
is shown once.
2. **Add it as a repository secret** named `AGENTIQA_SERVICE_KEY`.
3. **Add a workflow:**
```yaml
name: Agentiqa QA
on:
workflow_dispatch:
jobs:
qa:
runs-on: ubuntu-latest
steps:
- uses: agentiqa/qa-action@v1
with:
service-key: ${{ secrets.AGENTIQA_SERVICE_KEY }}
```
That runs **all** plans in the key's project on Agentiqa Cloud and fails the job on
a plan failure or a config error. The full input/output list is on the generated
[Action reference](/docs/github-action/reference).
## Selecting which plans run [#selecting-which-plans-run]
By default the action runs every plan in the key's project. Two inputs narrow the
selection, mirroring the CLI:
* **`plan-id`** — run a single plan by id (`tplan_…`).
* **`label-ids`** — comma-separated [label](/docs/guides/labels) ids (`lbl_…`);
runs every plan tagged with **any** of them. `plan-id` overrides `label-ids`.
**`mode`** picks the execution order for the selected plans — `sequential`
(default) or `parallel`. Parallel mode uses the CLI's bounded worker pool, with
up to four plans at a time by default and each plan in its own browser. A
transient engine disconnect is retried per plan on a fresh session, up to three
attempts total. Run dependent plan chains sequentially because parallel plans
do not share run memory (see
[parallel vs. sequential](/docs/cli#parallel-vs-sequential)).
```yaml
# Run just the plans labeled `smoke`, in parallel — e.g. a fast PR gate.
- uses: agentiqa/qa-action@v1
with:
service-key: ${{ secrets.AGENTIQA_SERVICE_KEY }}
label-ids: lbl_2b7f0e
mode: parallel
```
The quickest way to get the exact `label-ids` value is the app's **Run from CLI**
dialog — tick the labels and copy the ids. See [Labels](/docs/guides/labels).
## Gating and outputs [#gating-and-outputs]
`fail-on` decides which outcomes fail the job — `never`, `plan-failure` (default),
or `any` (release gate). The action exposes `outcome`, `exit-code`,
`plans-total/passed/failed`, and the path to the JSON envelope as step outputs.
The exit-code contract and the envelope schema are documented in
[CI Integration](/docs/ci); every input and output is on the
[Action reference](/docs/github-action/reference).
## Prefer the action over hand-rolled steps [#prefer-the-action-over-hand-rolled-steps]
The raw `npx agentiqa` command still works anywhere (see [CI Integration](/docs/ci)),
but the action pins the argv, the JSON capture, the artifact upload, and the
exit-code gating so they don't drift.
# Action reference (/docs/github-action/reference)
{/* GENERATED FILE — DO NOT EDIT BY HAND.
Source of truth: integrations/github-action/action.yml
Regenerate: node_modules/.bin/tsx apps/docs/scripts/render-action-reference.ts
A CI drift gate fails if this file is stale vs the manifest. */}
E2E tests written in plain English, run against your live app. Results, videos and run links right in your workflow.
Public reference: `agentiqa/qa-action@v1`. See the [GitHub Action guide](/docs/github-action) for setup and examples, and [CI Integration](/docs/ci) for the exit-code contract and JSON envelope.
## Inputs [#inputs]
| Input | Required | Default | Description |
| ------------------ | -------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `service-key` | **yes** | — | Project-scoped Agentiqa CLI service key (sk\_...). Required. Mint one from the Agentiqa web app or desktop app (Project Settings -> CLI Service Keys -> Create Key) or via /api/cli-quickstart-key. Pass it from a repository secret; never inline. |
| `api-url` | no | — | Override the Agentiqa control-plane API URL (AGENTIQA\_API\_URL). Leave empty to use production ([https://agentiqa.com](https://agentiqa.com)). Set to [https://s.agentiqa.com](https://s.agentiqa.com) to target staging. |
| `engine-url` | no | — | Optional hosted-engine URL override; only meaningful with `runtime: cloud`. Leave empty for the default Agentiqa cloud engine ([https://engine.agentiqa.com](https://engine.agentiqa.com)). Set [https://s-engine.agentiqa.com](https://s-engine.agentiqa.com) for staging, or the URL of your own hosted engine. |
| `plan-id` | no | — | Run a single test plan by id (tplan\_...). Overrides label-ids when set. |
| `label-ids` | no | — | Comma-separated label ids; runs all plans tagged with any of them. Ignored when plan-id is set. |
| `mode` | no | `sequential` | Execution order for the selected plans: sequential \| parallel. |
| `cli-version` | no | `latest` | Version of the Agentiqa CLI to run: an npm dist-tag (latest, staging) or an exact semver (e.g. 1.4.2). Default latest. |
| `artifacts-dir` | no | `agentiqa-artifacts` | Directory (relative to the workspace) for run artifacts and the JSON result envelope. |
| `fail-on` | no | `plan-failure` | Exit-code gating policy. One of: never - never fail the job on the run outcome (report-only; read outputs downstream). plan-failure - (default) fail when a plan failed (exit 1) OR on a config/usage error (exit 2 — permanent, must fail loudly). Only retryable infra errors (exit 3) are swallowed, surfacing as outcome=infra-error for a digest/reporter to handle. Exit 3 also covers a NO-VERDICT batch — one whose only non-passing plans could not reach a verdict (a persistent engine disconnect, or the blocked cascade of one) — so a transient WS drop no longer reads as a red plan-failure. Read the `outcome` step output to gate more strictly (fail-on: any). any - fail on any nonzero exit. Use for release gates. |
| `node-version` | no | `22` | Node.js version to set up on the runner (18+). |
| `upload-artifacts` | no | `true` | Upload the artifacts directory (screenshots/video/result JSON) as a build artifact. |
| `artifact-name` | no | — | Name for the uploaded artifact. Defaults to agentiqa-artifacts-\-\ when empty. |
| `share-links` | no | `false` | Mint an opt-in PUBLIC, revocable share link for each cloud run (passes --share to the CLI) so reviewers can open a run from the step summary without an Agentiqa login. The Run column links the share URL when present. Default false (privacy-conservative). Cloud runtime only. |
| `retention-days` | no | `14` | Retention period (in days) for the uploaded artifact. Only applies when upload-artifacts is true. Defaults to 14; the repo/org artifact-retention cap still bounds it. |
## Outputs [#outputs]
| Output | Description |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `outcome` | Run outcome bucket: passed \| plan-failure \| config-error \| infra-error. Note: exit 0 with zero executed plans buckets as config-error (a run that tests nothing is not a pass). |
| `json-path` | Path to the captured JSON result envelope (schemaVersion:1). Empty if the run was not attempted. |
| `exit-code` | Raw CLI process exit code (0 pass / 1 plan failure / 2 usage-config / 3 infra). |
| `plans-total` | Total plans reported in the JSON envelope (empty if no envelope). |
| `plans-passed` | Number of plans with outcome=passed. |
| `plans-failed` | Number of plans with outcome != passed. |
| `artifact-name` | Resolved artifact name used for the upload. |
| `cli-resolved-version` | The Agentiqa CLI version actually executed, resolved at run time from `agentiqa --version` (e.g. 1.1.32). Makes verdict drift across publishes attributable when cli-version is a floating dist-tag (latest/staging). Empty if capture failed or the run was not attempted. |
# Assistant (Explore) (/docs/guides/assistant)
The Assistant is where you drive the agent. You describe what you want in plain
language; the agent reads your intent, picks the right depth, explores the app in
a real browser, and reports back. You never describe clicks or selectors — the
agent figures out navigation, forms, and flows on its own.
## Say what you want, at any level [#say-what-you-want-at-any-level]
The same surface handles any level of specificity:
* *"What can you test here?"* — a conversational answer, no testing.
* *"Does the login page load?"* — a single check with an immediate answer.
* *"Test registration"* — focused exploration of one flow.
* *"Test the app"* — full discovery, scoped testing, a comprehensive report.
## The three checkpoints [#the-three-checkpoints]
The agent proposes; you curate. Each level is an approval gate that defaults to
"yes":
* **Scope** — the areas it discovered, ranked by risk. Approve all, uncheck
low-value areas, or redirect.
* **Plan** — the testing strategy for each area (what to focus on and why, not
hallucinated click steps). Approve, edit, or skip.
* **Findings** — the bugs it found, each with a screenshot and repro steps. This
is the one step that always needs your review: confirm, dismiss, or re-test.
You choose your depth — hands-off (approve at scope and review only the final
report), guided (curate scope and plans), or detailed (edit plans and redirect
mid-session). All three are first-class.
## Proactive questions [#proactive-questions]
The agent asks for what it needs when it needs it — credentials for a page behind a
login, or an API endpoint to reach an entity state the UI can't. It batches these
requests at checkpoints rather than interrupting mid-action, and remembers what you
provide for future sessions.
{/* verify: confirm the exact checkpoint UI (Scope/Plan/Findings panels, and the findings fix/triage action buttons) in ProjectAssistantV2Page. */}
The Assistant is also available headless via the [CLI](/docs/cli) `explore`
command, where every checkpoint auto-approves.
# CLI Service Keys (/docs/guides/cli-service-keys)
A **CLI service key** (`sk_...`) authenticates the [CLI](/docs/cli) and the
[GitHub Action](/docs/github-action) without an interactive login. It is
**project-scoped** — `agentiqa run` operates on that project's plans — and can be
revoked at any time.
## Create a key [#create-a-key]
1. Open **Project Settings → CLI Service Keys**.
2. **Create Key**, give it a name (e.g. `GitHub Actions`), and **copy the raw
`sk_...` value immediately** — it is shown only once.
{/* verify: confirm the CLI Service Keys settings screen and the exact create/copy flow. */}
The settings screen also shows the exact `agentiqa run` command to copy for that
project.
## Use the key [#use-the-key]
Pass it via the `AGENTIQA_SERVICE_KEY` environment variable — never inline it:
```bash
AGENTIQA_SERVICE_KEY=sk_... npx -y agentiqa@latest run --engine https://engine.agentiqa.com
```
In CI, store it as a repository secret named `AGENTIQA_SERVICE_KEY` and reference
it from the workflow. The service key alone authenticates the hosted engine (a
short-lived engine credential is minted automatically) — no separate engine token
is needed.
## Run a labeled subset per pipeline [#run-a-labeled-subset-per-pipeline]
A service key authenticates the run; a [label](/docs/guides/labels) selector
scopes it. Together they let one project serve several pipelines — each running
just its subset. A key runs **all** the project's plans unless you add a selector:
```bash
# Fast PR gate: only the plans tagged `smoke`
AGENTIQA_SERVICE_KEY=sk_... npx -y agentiqa@latest run \
--engine https://engine.agentiqa.com \
--label-ids lbl_2b7f0e
# Nightly: no selector → every plan in the key's project
AGENTIQA_SERVICE_KEY=sk_... npx -y agentiqa@latest run \
--engine https://engine.agentiqa.com --mode parallel
```
The labels must belong to the key's project. The app's **Run from CLI** dialog
composes the whole command (label ids and all) for you — see
[Labels](/docs/guides/labels).
## Security [#security]
* Treat the key like a password. Pass it from a secret, never commit it.
* The key is scoped to a single project. Revoke and re-create it from the same
settings screen if it leaks.
# Dashboard (/docs/guides/dashboard)
The project **Dashboard** is your at-a-glance view of how testing is going. It pulls
together every run of every saved test plan so you can see, in one place, what's
passing, what's failing, and which plans have started to regress.
## Runs over time [#runs-over-time]
The top chart, **Test plans runs daily**, shows one stacked bar per day, colored by
how each run finished:
* **Passed** — the run met its checks.
* **Failed** — a check failed.
* **Inconclusive** — the run finished but couldn't reach a clear verdict.
* **Infra error** — the run couldn't complete for an environmental reason, not a
defect in your app.
Use the **3d / 7d / 14d** buttons to change the time window. Click any day's bar to
expand the runs behind it — each row shows the test plan, its outcome, and the time
it ran, and links straight to that run's full results.
## By test plan [#by-test-plan]
Below the daily chart, each saved test plan that ran in the selected window gets its
own row with a small history of its recent runs. For each plan you'll see its pass
rate, when it last ran, and the outcome of its latest run.
Two badges call out plans that need attention:
* **Regressed** — a plan that was passing and has now started failing. The row lists
the steps that newly failed.
* **Flaky** — a plan whose runs have been switching between pass and fail.
Sort the list by **Recent** (most recently run first) or **Sidebar** (to match the
order of your test plans in the sidebar), and filter by name when you have a lot of
plans. Click any run in a plan's history to open its results. Plans that exist but
didn't run in the selected window are noted at the bottom of the list.
## Before your first run [#before-your-first-run]
Until a saved test plan has actually run, the Dashboard has nothing to chart, so it
shows a short getting-started prompt instead. Test plans are saved from your chats
with the Assistant and then run — once the first one does, its results show up here
automatically. See [Test Plans](/docs/guides/test-plans) and [Runs](/docs/guides/runs).
# Issues (/docs/guides/issues)
An **issue** is a bug the agent found. Every reported finding includes what's
wrong, reproduction steps, and screenshot evidence. Issues live in the project and
have a lifecycle from draft finding to confirmed to resolved.
## The findings checkpoint [#the-findings-checkpoint]
Issues first appear at the findings checkpoint in the [Assistant](/docs/guides/assistant),
where each has two groups of actions:
* **Fix actions** — copy a fix request (a prompt for a coding agent like Claude
Code or Cursor), or re-test to verify.
* **Triage actions** — screenshot, dismiss, or confirm.
The verdict adjusts reactively: if you dismiss all reported issues, it upgrades to
"ship."
{/* verify: confirm the findings-card action set and labels in the current UI. */}
## Fix and re-test [#fix-and-re-test]
1. **Review** the issue (repro steps + screenshot).
2. **Copy request** to your coding agent and fix the code.
3. **Re-test** — the agent spawns a focused explorer that replays the repro steps.
4. **Auto-resolve** — if the issue no longer reproduces, it is marked resolved
automatically. (An unconfirmed draft that doesn't reproduce is auto-dismissed
instead.)
## Browsing issues [#browsing-issues]
The project's issues view lists open issues; resolved issues are kept separately,
and each issue has a detail view with its evidence and history.
{/* verify: confirm the Issues / Resolved / issue-detail views (IssuesPage, ResolvedIssuesPage, IssueDetailPage). */}
# Labels (/docs/guides/labels)
A **label** is a project-scoped tag — a name and a color — that you attach to test
plans. A plan can carry any number of labels, and labels can be shared across
plans, so a label names a **subset** of your suite: `smoke`, `checkout`,
`nightly`, `billing`. That subset is what you then run in automation — instead of
running every plan, run "the smoke plans" from the CLI or the GitHub Action.
Labels belong to a project (their ids look like `lbl_…`) and are stored in the
cloud alongside the rest of your project data, so the same labels are visible in
the web app, the desktop app, and to the CLI.
## Create and assign labels in the app [#create-and-assign-labels-in-the-app]
Labels are created and assigned **in the product** (the web or desktop app), on a
test plan — there is no CLI command that creates a label.
* **Assign / create from a plan.** Open a test plan (or use the label control on
its card) and open the label picker. Type a name: pick an existing label, or
choose **Create "\"** to make a new one. A color is auto-assigned from
the palette (you can pick a different one). The label attaches to that plan
immediately and syncs to the cloud.
* **Manage the project's labels** — rename, recolor, or delete — from **Project
Settings**. Deleting a label removes it from every plan that carried it (the
plans themselves are untouched).
## Select labeled plans in automation [#select-labeled-plans-in-automation]
`agentiqa run` (with a [service key](/docs/guides/cli-service-keys)) operates on
the key's project. Two selectors narrow which plans execute:
| Selector | Runs |
| ------------------- | --------------------------------------------------------------- |
| *(none)* | **All** non-deleted plans in the project. |
| `--plan-id tplan_…` | One specific plan. |
| `--label-ids a,b,c` | Every plan tagged with **any** of the listed labels (OR match). |
`--label-ids` takes label **ids** (the `lbl_…` values), comma-separated. If you
pass both, `--plan-id` wins over `--label-ids`. A selector that matches **no**
plans is a configuration error (a nonzero exit) — a run that tests nothing is
never a pass. See the [CLI reference](/docs/cli/reference) and
[CI Integration](/docs/ci).
The [GitHub Action](/docs/github-action) exposes the same two selectors as inputs:
`plan-id` and `label-ids` (again, `plan-id` overrides `label-ids`).
## Copy the command without hunting for ids [#copy-the-command-without-hunting-for-ids]
You don't have to look up `lbl_…` ids by hand. In the app, open **Run from CLI**
for a project, tick the labels you want (and the **Run in parallel** toggle if you
want it), and copy the generated command. It already contains the right
`--label-ids ` (and `--mode parallel`) plus the `AGENTIQA_SERVICE_KEY`
environment prefix — paste it straight into your shell or a CI step.
## Run a labeled subset in CI [#run-a-labeled-subset-in-ci]
Combining a **service key** (auth) with a **label selector** (scope) is the core
CI pattern: give each pipeline the subset it should run. A common split is a fast
`smoke` label on pull requests and the full suite nightly.
CLI step — run only the plans tagged `smoke`:
```bash
AGENTIQA_SERVICE_KEY=sk_... npx -y agentiqa@latest run \
--engine https://engine.agentiqa.com \
--label-ids lbl_2b7f0e # the id of your "smoke" label
```
GitHub Action — the same subset, run in parallel:
```yaml
- uses: agentiqa/qa-action@v1
with:
service-key: ${{ secrets.AGENTIQA_SERVICE_KEY }}
label-ids: lbl_2b7f0e
mode: parallel
```
The service key is project-scoped, so the labels you reference must belong to that
same project. For minting and storing keys, see
[CLI Service Keys](/docs/guides/cli-service-keys); for sequential-vs-parallel
behavior, see [Parallel runs](/docs/cli#parallel-vs-sequential).
## Related [#related]
* [Test Plans](/docs/guides/test-plans) — where plans (and their labels) come from.
* [CLI overview](/docs/cli) — plan selection and parallel execution.
* [GitHub Action](/docs/github-action) — the `plan-id` / `label-ids` / `mode`
inputs.
# Projects (/docs/guides/projects)
A **project** is the unit of work in Agentiqa. It pairs the URL of the app you're
testing with everything the agent accumulates about it over time: its model of the
app, findings, saved test plans, and project memory. These guides apply equally to
the [web app](/docs/web-app) and the [desktop app](/docs/desktop-app) — both render
the same features.
## Create a project [#create-a-project]
Create a project by entering the app's **URL** and a name. That URL becomes the
project's default target, so later runs don't need you to re-enter it.
{/* verify: confirm the exact new-project form fields (URL, name, and any target-type selector). */}
## What a project holds [#what-a-project-holds]
* **The Assistant** — the agent's chat surface for this project. See
[Assistant](/docs/guides/assistant).
* **Test plans** — saved, replayable plans. See [Test Plans](/docs/guides/test-plans).
* **Runs** — the history of executions and their per-step results. See
[Runs](/docs/guides/runs).
* **Issues** — confirmed and draft findings. See [Issues](/docs/guides/issues).
* **Dashboard** — run outcomes and pass/fail trends across all this project's test
plans. See [Dashboard](/docs/guides/dashboard).
* **Settings** — project configuration, credentials, and CLI service keys. See
[Settings](/docs/guides/settings).
## The agent's evolving model [#the-agents-evolving-model]
Agentiqa builds a structured QA model of each project that persists across
sessions — a map of the app's surfaces, entities, and flows, plus a journal of
what was tested and how deeply. On a return visit the agent already knows the app
and skips re-discovery, so you don't re-explain. Human context the agent can't
discover on its own (test credentials, environment quirks) lives in **project
memory**, which is only written when you explicitly ask it to remember something.
# Runs & Run Detail (/docs/guides/runs)
A **run** is one execution of a test plan (or a batch of them). Every run is
recorded so you can review what happened, compare against history, and share the
result.
## Run history [#run-history]
Each plan keeps a history of its runs. From the history you can open any past run
to see how it went and how the result changed over time.
{/* verify: confirm the run-history list and navigation in TestPlansV2HistoryPage. */}
## Run detail [#run-detail]
The run detail view shows the execution end to end:
* **Per-step results** — pass/fail for each step, with the criteria that were
checked.
* **A verdict** — the agent's professional opinion: ship, ship with known risks,
or don't ship, with rationale.
* **Evidence** — screenshots and, when recorded, a video of the session.
{/* verify: confirm the exact panels in RunDetailPage (steps, verdict, video/screenshots). */}
## Runs in automation [#runs-in-automation]
CLI and Action runs surface the same result as a JSON envelope, including a
per-plan `outcome`, `durationSec`, and (when available) a `runUrl` deep link back
to this view, a `videoUrl`, and — with `--share` / the Action's `share-links` — a
public `shareUrl` a reviewer can open without signing in. See
[CI Integration](/docs/ci).
# Settings (/docs/guides/settings)
Settings cover both the current project and your account.
## Project settings [#project-settings]
Per-project configuration lives in the project's settings — the default target URL,
saved credentials the agent can use to reach pages behind a login, project memory,
and **CLI Service Keys** for running the project's plans in automation.
{/* verify: confirm the exact sections in ProjectSettingsPage (target URL, credentials, memory, service keys). */}
Credentials you save here let the agent authenticate into your app during a run;
project memory holds human context the agent can't discover on its own and is only
written when you explicitly ask.
## Account settings [#account-settings]
Account-level settings apply across projects.
{/* verify: confirm the contents of the global SettingsPage. */}
## CLI service keys [#cli-service-keys]
To run a project's plans from the CLI or CI you need a service key — see
[CLI Service Keys](/docs/guides/cli-service-keys).
# Test Plans (/docs/guides/test-plans)
A **test plan** is a saved, replayable definition of something to verify. Plans are
the deterministic counterpart to the exploratory Assistant: running one produces
per-step pass/fail against defined criteria, so the same check can be repeated over
time.
## Where plans come from [#where-plans-come-from]
Plans are created from real interaction — typically saved out of an exploratory
session's findings, so they reflect UI the agent actually saw (not hallucinated
steps). You can also edit a plan's steps directly.
{/* verify: confirm how a plan is created/saved and edited in TestPlanPage (create from findings, manual step editing). */}
## Running plans [#running-plans]
* **In the app** — run a plan from the project and watch it execute; results land
in [Runs](/docs/guides/runs).
* **Regression** — run all saved plans in parallel (each gets its own browser) to
check nothing broke. For each failure the agent classifies the cause: a **real
bug** (behavior changed), a **stale plan** (the UI changed but the functionality
is intact — the plan needs updating), or a **flake** (timing/intermittent).
* **In CI** — run plans headless with the [CLI](/docs/cli) `run` command or the
[GitHub Action](/docs/github-action). Select plans by id (`--plan-id`) or by
label (`--label-ids`), sequentially or in parallel.
## Selecting plans in automation [#selecting-plans-in-automation]
`agentiqa run` with no selector runs **all** non-deleted plans in the project. Use
`--plan-id` for one plan, or `--label-ids a,b` to run every plan tagged with any of
those labels. Group plans with [Labels](/docs/guides/labels) to run a named subset
(e.g. `smoke`) per pipeline; see the [CLI reference](/docs/cli/reference) for every
flag.