> ## Documentation Index
> Fetch the complete documentation index at: https://docs.simmer.markets/llms.txt
> Use this file to discover all available pages before exploring further.

# Autoresearch

> Autonomous skill optimization — your agent mutates skill config, measures P&L, and keeps what works.

<Note>
  **Pro & Elite feature.** Autoresearch is available on [Simmer Pro](https://simmer.markets?ref=docs\&utm_campaign=docs) and Elite plans (Elite includes everything in Pro). Free users get a 403 when calling autoresearch API endpoints.
</Note>

<Note>
  **Package renamed.** `simmer-autoresearch` (npm) has been renamed to `simmer-mcp`. Update your install command and MCP config — see [Install](#install) below. The v1 OpenClaw plugin (`openclaw plugins install simmer-autoresearch`) is unaffected and documented in the [Legacy](#legacy-v1-plugin) section.
</Note>

Autoresearch lets your agent optimize its own trading skills. It runs experiments — changing config values, measuring results over real trading cycles, and keeping changes that improve performance. Think of it as automated A/B testing for your trading strategy.

## Prerequisites

* **Simmer Pro plan** with a valid `SIMMER_API_KEY`
* **simmer-sdk** installed with at least one trading skill running on sim venue
* **Node.js** 18+ (for the MCP server)
* **Git** initialized in your skill workspace (autoresearch uses git for commit/revert)

<Warning>
  **Start with sim venue.** Always run autoresearch against the simulated venue first. Autoresearch mutates your skill's code and config — running against a real-money venue risks unexpected losses from untested changes.
</Warning>

## How it works

```
init_experiment → run_experiment (N cycles) → log_experiment → repeat
```

1. **Init** — Pick a skill and a metric (e.g., P\&L, edge %, trade count)
2. **Run** — Execute the skill with the new config for several trading cycles
3. **Log** — Record results and decide: keep or revert. Keeps auto-commit to git.
4. **Backtest** — Replay historical trades against new config thresholds (fast config tuning)
5. **Repeat** — Try the next hypothesis

Your agent drives the loop — autoresearch provides the tools, your agent provides the reasoning.

## Install

```bash theme={null}
npm install -g simmer-mcp
```

Then add the MCP server to your agent's config:

<CodeGroup>
  ```json OpenClaw theme={null}
  {
    "mcpServers": {
      "simmer": {
        "command": "npx",
        "args": ["-y", "simmer-mcp"],
        "env": {
          "SIMMER_API_KEY": "your-api-key"
        }
      }
    }
  }
  ```

  ```json Hermes theme={null}
  {
    "mcpServers": {
      "simmer": {
        "command": "npx",
        "args": ["-y", "simmer-mcp"],
        "env": {
          "SIMMER_API_KEY": "your-api-key"
        }
      }
    }
  }
  ```

  ```json Claude Code theme={null}
  {
    "mcpServers": {
      "simmer": {
        "command": "npx",
        "args": ["-y", "simmer-mcp"],
        "env": {
          "SIMMER_API_KEY": "your-api-key"
        }
      }
    }
  }
  ```
</CodeGroup>

Then install the behavioral skill (tells your agent how to run the experiment loop):

```bash theme={null}
npx simmer-mcp install-skill
```

This auto-detects your runtime (OpenClaw, Hermes) and copies the skill instructions to the right directory. For Claude Code, add the skill content to your project's `CLAUDE.md`.

## Config

Configure autoresearch via environment variables:

| Variable                       | Default                      | Description                                                           |
| ------------------------------ | ---------------------------- | --------------------------------------------------------------------- |
| `SIMMER_API_KEY`               | —                            | **Required.** Your Simmer API key.                                    |
| `SIMMER_API_URL`               | `https://api.simmer.markets` | API base URL. Override for self-hosted.                               |
| `AUTORESEARCH_MAX_EXPERIMENTS` | `50`                         | Max experiments per session. Prevents runaway loops. `0` = unlimited. |

## Running Autoresearch

### Setup (once per optimization target)

1. **Pick a skill** to optimize and a primary metric (usually P\&L)
2. Create a git branch: `git checkout -b autoresearch/<skill>-<date>`
3. Read the skill source code thoroughly — understand what it does before mutating
4. Write `autoresearch.md` — a session spec describing the goal, metrics, how to run, and constraints
5. Write `autoresearch.sh` — a single command that runs the skill for one cycle
6. Commit both files
7. Call `init_experiment` → run the baseline with `run_experiment` → `log_experiment` → start looping

### The experiment loop

Each iteration:

1. **Hypothesize** — what change might improve the metric?
2. **Mutate** — change the skill's code or config
3. **Run** — call `run_experiment` to execute the skill
4. **Log** — call `log_experiment` to record the result (`keep`, `discard`, or `crash`)

`keep` auto-commits to git. `discard` and `crash` auto-revert the working directory.

Use `backtest_experiment` for fast config exploration (seconds) before committing to live runs (minutes).

### Key rules

* **Never skip the baseline run.** The first experiment establishes the reference point for all comparisons.
* **Always log — even crashes.** Crash data matters for confidence scoring and crash detection.
* **Check confidence scores.** ≥2× noise floor = improvement is likely real. under 1× = within noise. 1-2× = marginal, re-run to confirm.
* **Code mutations beat config tuning.** Structural changes (new data sources, different models, alternative strategies) find bigger wins than parameter sweaks.
* **Keep ideas in `autoresearch.ideas.md`.** Promising but deferred optimizations go here.

### When you're done

Review the autoresearch git branch. Experiments that were `keep`-ed are committed with result metadata in the commit message. Merge the branch (or cherry-pick specific experiments) into your main skill branch to lock in the improvements.

## Tools

The MCP server registers four tools your agent can call:

### `init_experiment`

Configure an experiment session. Call again to start a new segment with a fresh baseline.

| Parameter     | Required | Description                                                          |
| ------------- | -------- | -------------------------------------------------------------------- |
| `name`        | Yes      | Human-readable session name                                          |
| `skill_slug`  | Yes      | ClawHub slug of the skill to optimize (e.g., `polymarket-fast-loop`) |
| `metric_name` | Yes      | Primary metric to track (e.g., `pnl`, `avg_edge`)                    |
| `metric_unit` | No       | Unit label (e.g., `$SIM`, `%`)                                       |
| `direction`   | No       | `higher` or `lower` — which direction is better (default: `higher`)  |

### `run_experiment`

Execute a command (usually the skill), capture output and timing.

| Parameter | Required | Description                                                                          |
| --------- | -------- | ------------------------------------------------------------------------------------ |
| `command` | Yes      | Shell command to run (e.g., `python skills/polymarket-fast-loop/fastloop_trader.py`) |
| `timeout` | No       | Timeout in seconds (default: 300)                                                    |

### `log_experiment`

Record experiment results. `keep` auto-commits to git. `discard`/`crash` reverts working directory.

| Parameter           | Required | Description                          |
| ------------------- | -------- | ------------------------------------ |
| `status`            | Yes      | `keep`, `discard`, or `crash`        |
| `metric`            | Yes      | Primary metric value (number)        |
| `description`       | Yes      | What was tried and what happened     |
| `secondary_metrics` | No       | Additional metrics as key-value dict |

### `backtest_experiment`

Replay historical trades against new config thresholds without live execution. Returns simulated P\&L in seconds — use this for fast config tuning before committing to live experiments.

<Note>
  Backtest requires trades with `signal_data`. Skills must pass structured signal data on `client.trade()` calls (SDK 0.9.17+). All official Simmer skills include signal\_data as of March 2026.
</Note>

| Parameter    | Required | Description                                           |
| ------------ | -------- | ----------------------------------------------------- |
| `skill_slug` | Yes      | Skill to backtest                                     |
| `config`     | Yes      | Config overrides to test (e.g., `{"min_edge": 0.05}`) |
| `days`       | No       | Days of history to replay (default: 7, max: 30)       |
| `venue`      | No       | `sim` or `polymarket` (default: `sim`)                |

**Config threshold convention:**

* `min_edge: 0.05` → only include trades where `signal_data.edge >= 0.05`
* `max_probability: 0.85` → only include trades where `signal_data.probability <= 0.85`
* Bare keys (e.g., `edge: 0.10`) → treated as min threshold

## Signal Data

Skills can include structured signal data on each trade to enable backtest replay. This is optional — trades work fine without it — but required for the `backtest_experiment` tool.

```python theme={null}
result = client.trade(
    market_id, "yes", 10.0,
    reasoning="NOAA forecasts 35°F, bucket underpriced at 12%",
    signal_data={
        "edge": 0.15,
        "confidence": 0.8,
        "signal_source": "noaa_forecast",
        "forecast_temp": 35,
        "bucket_range": "30-39",
    },
    skill_slug="polymarket-weather-trader",
)
```

**Common fields** (recommended for all skills):

| Field           | Type      | Description                      |
| --------------- | --------- | -------------------------------- |
| `edge`          | float     | Perceived edge over market price |
| `confidence`    | float 0-1 | Agent confidence in the trade    |
| `signal_source` | string    | What triggered the signal        |

Additional skill-specific fields are freeform. Values must be strings or numbers (flat dict, no nesting).

Signal data is **private** — only visible to the trade owner via authenticated API calls. Never exposed publicly.

## Session management

Your agent manages its own session state using the SKILL.md behavioral instructions (installed via `npx simmer-mcp install-skill`). There is no `/autoresearch` command interface — the agent drives the loop autonomously.

* **Resume:** The agent reads `autoresearch.jsonl` on startup and resumes where it left off.
* **New session:** Call `init_experiment` with a new name to start a fresh segment (previous results are archived, not deleted).
* **Context compaction:** If the agent's context resets, it should re-read `autoresearch.md` and `autoresearch.jsonl` to restore state.

## Safety features

### Crash protection

* **Baseline crash** — If the very first experiment in a session crashes, autoresearch pauses automatically. This usually means the skill is misconfigured.
* **Consecutive crashes** — 3 crashes in a row triggers auto-pause. Your agent can't run more experiments until the issue is investigated.
* **Recovery** — Call `init_experiment` with a new session name to clear the pause and start fresh.

### Budget caps

Experiments are capped at `AUTORESEARCH_MAX_EXPERIMENTS` (default 50) per session. At 80% of the cap, your agent gets a warning. At the limit, `run_experiment` is blocked.

Set `AUTORESEARCH_MAX_EXPERIMENTS=0` to disable the cap (not recommended for unattended agents).

### Metric verification

The server cross-checks self-reported P\&L metrics against the Simmer API. If the agent-reported metric diverges significantly from actual trade data, a warning is logged. This prevents metric gaming — the agent can't inflate results by changing how metrics are calculated.

## Experiment persistence

Results are saved in two places:

* **Local JSONL** — `autoresearch.jsonl` in your working directory for offline access
* **Dashboard API** — Synced to your Simmer dashboard (Pro users see an Autoresearch tab)

Git auto-commits on `keep` decisions so you can track what changed and roll back if needed.

## API endpoints

These endpoints power the server's sync. You don't call them directly — the MCP server handles it.

| Endpoint                                 | Description                                                          |
| ---------------------------------------- | -------------------------------------------------------------------- |
| `POST /api/sdk/autoresearch/experiments` | Sync experiment results                                              |
| `GET /api/sdk/autoresearch/experiments`  | List experiment history                                              |
| `GET /api/sdk/autoresearch/state`        | Resume state for server startup                                      |
| `POST /api/sdk/autoresearch/backtest`    | Replay trades against new config                                     |
| `GET /api/sdk/outcomes`                  | Trade outcome summary — v1 cash-flow + v2 settlement-accurate fields |

### `/api/sdk/outcomes` v2 fields

`GET /api/sdk/outcomes` returns both backward-compatible cash-flow fields (v1) and new settlement-accurate fields (v2). Use v2 fields for autoresearch metric verification and skill-health signals — they correctly attribute buys held to resolution.

**SDK:** `client.get_outcomes(skill_slug=..., since=...)`

| Field                  | Type   | Description                                                                                              |
| ---------------------- | ------ | -------------------------------------------------------------------------------------------------------- |
| `trades`               | int    | Total executed trade count (v1 — cash-flow)                                                              |
| `pnl`                  | float  | Net cash-flow P\&L: sell proceeds minus buy costs (v1 — cash-flow)                                       |
| `wins`                 | int    | Sell rows with positive cash-flow (v1 — cash-flow)                                                       |
| `losses`               | int    | Sell rows with zero or negative cash-flow (v1 — cash-flow)                                               |
| `settled_pnl`          | float  | Sum of realized P\&L across all resolved markets (v2)                                                    |
| `resolved_markets`     | int    | Number of resolved markets where this skill held a position (v2)                                         |
| `settled_wins`         | int    | Resolved markets with positive realized P\&L (v2)                                                        |
| `settled_losses`       | int    | Resolved markets with zero or negative realized P\&L (v2)                                                |
| `confidence_breakdown` | object | Count by confidence tag: `settlement` (PolyNode v3), `native` (sim venue), `mirror` (market mirror) (v2) |

**Why v2 exists:** v1 `wins` only count sell rows — a buy held to resolution and settled as a winner never appears. v2 `settled_wins` counts every resolved market correctly.

## Legacy (v1 Plugin)

<Accordion title="Legacy (v1 Plugin) — OpenClaw only">
  v1 was an OpenClaw plugin, not an MCP server. If you're still running v1:

  ```bash theme={null}
  openclaw plugins install simmer-autoresearch
  ```

  Configure via `plugins.json`:

  ```json theme={null}
  {
    "simmer-autoresearch": {
      "apiKey": "your-api-key",
      "maxExperiments": 30
    }
  }
  ```

  v1 supports the `/autoresearch` command interface:

  | Command                 | Description                                                               |
  | ----------------------- | ------------------------------------------------------------------------- |
  | `/autoresearch <skill>` | Start or resume autoresearch mode for a skill                             |
  | `/autoresearch off`     | Stop autoresearch mode                                                    |
  | `/autoresearch status`  | Current skill, experiment count, keep rate, budget remaining, pause state |
  | `/autoresearch reset`   | Clear state and start fresh (clears pause if paused)                      |

  **Upgrade to v2** — Install `simmer-mcp` via npm (`npm install -g simmer-mcp`) and switch to the MCP config above. v2 works with OpenClaw, Hermes, and Claude Code.
</Accordion>