Run Cursor in your evals

Use the Cursor SDK to run Cursor's agent loop inside your own eval harness. The same agent that powers the Cursor IDE, CLI, and web app is scriptable from TypeScript, so you can score Grok 4.6 (and other models we support) on your benchmarks.

Benchmark authors like Artificial Analysis and SWE-rebench already use it to score Cursor on their leaderboards.

Why the SDK

Eval harnesses need a stable, programmatic interface to an agent: pass a task, get a transcript and final state, score the result. The SDK gives you that without shelling out to the CLI:

• Real agent loop. Tool calls, file edits, terminal commands, and reasoning run through the same code path as the product.
• Grok-first, multi-model. Grok 4.6 is the default to evaluate, but any model in Cursor's catalog works through the same API.
• Local or sandboxed cloud runtime. Run against a working tree on disk for fast iteration, or use Cursor's hosted VMs for isolated, parallel runs.
• Structured streams and results. Typed SDKMessage events, per-step deltas, and a final RunResult with model, duration, and git info.

For the full API surface, see the Cursor SDK reference.

Setup

npm install @cursor/sdk

export CURSOR_API_KEY="your-key"

Evaluating Grok 4.6

A single-task eval against Grok 4.6 on a local working tree:

import { Agent } from "@cursor/sdk";const result = await Agent.prompt(  "Implement the failing tests in tests/string_utils.test.ts. Do not modify the tests.",  {    apiKey: process.env.CURSOR_API_KEY!,    model: { id: "grok-4.6" },    local: { cwd: "/path/to/task/checkout" },  },);console.log(result.status);     // "finished" | "error" | "cancelled"console.log(result.result);     // final assistant textconsole.log(result.durationMs); // wall-clock duration

Agent.prompt() creates an agent, sends one prompt, waits for the run to finish, and disposes. It's the right primitive for stateless eval tasks.

After the run completes, score the working tree with whatever your harness already uses (test runner, judge model, exact-match checker, etc.).

Streaming events for transcripts

Most harnesses want the full transcript, not the final text. Open a long-lived agent and stream SDKMessage events:

import { Agent, type SDKMessage } from "@cursor/sdk";await using agent = await Agent.create({  apiKey: process.env.CURSOR_API_KEY!,  model: { id: "grok-4.6" },  local: { cwd: taskCwd },});const run = await agent.send(taskPrompt);const transcript: SDKMessage[] = [];for await (const event of run.stream()) {  transcript.push(event);  if (event.type === "assistant") {    for (const block of event.message.content) {      if (block.type === "text") process.stdout.write(block.text);    }  }  if (event.type === "tool_call" && event.status === "completed") {    console.log(`[tool] ${event.name}`);  }}const final = await run.wait();saveTranscript(transcript, final);

run.stream() yields typed events for assistant text, thinking, tool calls (start and completion), and lifecycle status. Final metadata (model, duration, git info) reads off the Run after the stream ends. See Stream events for the full event schema.

Evaluating other models

The same harness code evaluates any model in Cursor's catalog. Swap the id:

const models = ["grok-4.6", "composer-2.5", "gpt-5.6-sol", "claude-opus-5", "gemini-3.1-pro"];for (const id of models) {  const result = await Agent.prompt(taskPrompt, {    apiKey: process.env.CURSOR_API_KEY!,    model: { id },    local: { cwd: taskCwd },  });  recordScore(id, scoreTask(result));}

The agent loop, tool schema, prompts, and stream shape stay constant across models, so you measure model-level differences instead of harness drift. List supported ids with Cursor.models.list().

Running tasks in parallel (cloud)

For large eval sets, run each task in an isolated cloud VM. The VM clones the repo, runs the agent, and surfaces git results back to your harness.

import { Agent } from "@cursor/sdk";async function runTask(task: EvalTask) {  await using agent = await Agent.create({    apiKey: process.env.CURSOR_API_KEY!,    model: { id: "grok-4.6" },    cloud: {      repos: [{ url: task.repoUrl, startingRef: task.baseRef }],    },  });  const run = await agent.send(task.prompt);  const result = await run.wait();  return {    taskId: task.id,    status: result.status,    branch: result.git?.branches[0]?.branch,    durationMs: result.durationMs,  };}const results = await Promise.all(tasks.map(runTask));

Each agent runs in its own VM, so you can parallelize as wide as your rate limits and request pools allow. See Cloud agents for VM behavior, lifecycle, and artifact handling.

Per-task configuration

The SDK gives you the knobs eval harnesses usually need:

• Custom tool sets. Restrict or extend tools via MCP servers inline on Agent.create().
• Subagents. Define named subagents the main agent can spawn during a task.
• Cancellation and timeouts. Call run.cancel() to enforce wall-clock limits. Status becomes "cancelled" and partial output stays readable.
• Per-step callbacks. Use the onStep and onDelta options on agent.send() for finer-grained logging.

Privacy and billing

SDK runs follow the same pricing, request pools, and Privacy Mode rules as runs from the IDE and Cloud Agents. Eval traffic is tagged so it shows up under the SDK label in your team's usage dashboard. To keep eval data out of model training, turn on Privacy Mode for the account or team running the harness.

Higher rate limits

Next steps

• Browse the full Cursor SDK reference for every option, event type, and error class.
• Read about Grok 4.6 and the rest of Cursor's models.
• Explore Cloud agents for sandboxed, parallel runs.