Execution & Agents

How AXIS executes scenarios, manages agent processes, and isolates workspaces.

Execution Model

When you run axis run, AXIS loads your config, discovers scenarios, and executes each scenario/agent combination as an independent job. Jobs run in parallel up to the configured concurrency limit.

Each job follows the same lifecycle:

1. Setup -Run setup actions defined in the scenario (if any).
2. Spawn -Start the agent process in an isolated workspace.
3. Capture -Stream and record the full interaction transcript.
4. Score -Evaluate the transcript against the rubric and interaction signals (unless --no-score is set).
5. Teardown -Run teardown actions (if any).
6. Save -Write the result to the report.

The agent has a 10-minute timeout by default. If the agent does not finish within the timeout, AXIS sends SIGTERM, waits briefly, then SIGKILL. Timed-out runs are marked as failed with a timeout error.

Supported Agents

AXIS ships with built-in support for popular AI coding agents. Each agent integration handles CLI resolution, process spawning, transcript capture, and output normalization.

CLI binaries are resolved automatically. If not found locally, AXIS falls back to npx --yes <package> silently. Default flags are applied unless you override them in your agent configuration.

Custom Agents

You can test any agent by creating a custom agent module. Use the createAgentAdapter() factory and register the module in your config.

// adapters/my-agent.ts
import { createAgentAdapter } from "@netlify/axis";

export default createAgentAdapter<{ stdout: string }>({
  name: "my-agent",
  resolveCommand: () => ({ command: "my-cli", prefixArgs: [] }),
  buildArgs: (input) => [input.prompt],
  initialState: () => ({ stdout: "" }),
  streamConfig: {
    mode: "aggregate",
    onChunk: (chunk, ctx) => {
      ctx.state.stdout += chunk;
    },
  },
  getResult: (ctx) => ({
    result: ctx.state.stdout.trim() || null,
  }),
});

Register it in axis.config.json:

{
  "adapters": {
    "my-agent": "./adapters/my-agent.ts"
  },
  "agents": ["my-agent"]
}

Stream Modes

Custom agents support two modes for processing output:

• Lines mode -For agents that emit NDJSON (one JSON object per line). AXIS parses each line and passes the parsed object to your onLine handler. The built-in Claude Code, Codex, and Gemini agents all use this mode.
• Aggregate mode -For agents that emit plain text or non-JSON output. Raw chunks are passed to onChunk and accumulated in state. Use this for agents with custom output formats or simple stdout capture.

The module must export an AgentAdapter as the default export or as a named adapter export.

Workspace Isolation

Each agent run gets a fresh temporary directory as its workspace. AXIS isolates the following to prevent configuration leakage and cross-run interference:

• HOME directory: Set to the workspace to prevent global config leakage.
• Agent-specific dirs: CLAUDE_CONFIG_DIR, CODEX_HOME, GEMINI_CLI_HOME are all set to isolated paths.
• Environment variables: Only explicitly listed vars and system essentials (PATH, USER, SHELL, LANG, TERM, TMPDIR) are passed through.

MCP server configuration files are written into each workspace before the agent spawns, in the format native to each agent CLI. See MCP Servers in the configuration reference.

Multi-variant Scenarios

A single scenario file can produce multiple jobs by defining variants. Each variant runs as an independent job with its own key, inheriting the base scenario's fields and applying any overrides. This is useful for testing the same task under different tool configurations, prompts, or agent restrictions without duplicating scenario files.

For example, a scenario with two variants and two agents produces four jobs (2 variants × 2 agents). Each variant appears as a separate row in the CLI output and a separate entry in reports, identified by its @-suffixed key (e.g., create-post@with-mcp).

See Writing Scenarios → Variants for the full field reference and examples.