Playwright's test agents in Claude Code, without losing control of the code

Ask an AI coding agent to "write tests for the cart" and you will get files that pass. That is the easy part. The hard part is knowing whether they test anything, and whether the next automatic fix will quietly turn a real bug into a green build.

Since version 1.56, released in October 2025, Playwright ships three agent definitions for exactly this work: a planner, a generator and a healer. They run inside Claude Code as subagents. This post covers what each one produces, how to install them, a worked example on a store cart, and where a person has to review before anything is merged. The commands and file names below were checked against the Playwright documentation and the source of release 1.64.1, the latest at the time of writing.

1. Three agents, three kinds of file

The agents can be used one at a time or in sequence. In sequence they form a loop, and each step leaves a file you can read.

• Planner. It takes a request ("plan the cart"), a seed test and, optionally, a requirements document. It runs the seed test so the app is in a known state, explores the pages with a real browser, and saves a Markdown plan in specs/: scenarios, numbered steps and expected results.
• Generator. It reads one plan and performs each step live in the browser, checking the locators and assertions as it goes. Then it writes one test per scenario under tests/, with a // spec: and a // seed: comment at the top and the step text as a comment before each action.
• Healer. It runs the suite, pauses on a failing test, takes a snapshot of the page and edits the test until it passes. If it concludes that the test is right and the app is wrong, it marks the test with test.fixme(), which skips it, and leaves a comment explaining why.

One line in the healer's own instructions explains why this post spends most of its time on review. The definition tells it: "Do not ask user questions, you are not interactive tool, do the most reasonable thing possible to pass the test." Making the test pass is the goal it is given. Deciding whether the test should pass is still your job.

2. Installing the agents in Claude Code

You need a Playwright Test project, which means a playwright.config.ts and @playwright/test 1.56 or later. If the repository has neither, start there. Then generate the agent definitions:

npm init playwright@latest              # only if Playwright is not set up yet
npx playwright init-agents --loop=claude

The command writes these files:

• .claude/agents/playwright-test-planner.md, playwright-test-generator.md and playwright-test-healer.md: Claude Code subagents, each with a name, a description, a list of allowed tools and model: sonnet.
• .mcp.json: one MCP server called playwright-test that runs npx playwright run-test-mcp-server. On Windows it is written as cmd /c npx playwright run-test-mcp-server. This server gives the agents the browser and the test runner.
• specs/README.md, if the specs/ folder does not exist yet.
• A seed test, seed.spec.ts in your test directory, but only if no test file with "seed" in its name exists.

Two options are worth knowing. --project <name> chooses which project in your config the seed belongs to, and --prompts also writes ready-made prompt templates to .claude/prompts.

Four details caught us out, or would have:

1. It replaces .mcp.json. In release 1.64.1 the file is written whole, containing only the playwright-test server. If your project already declared other MCP servers there, they are gone. Commit before running the command, then run git diff .mcp.json and merge your servers back.
2. It has to be rerun after every Playwright upgrade. The documentation says so, because the tools and instructions change between releases. Rerunning overwrites the agent files, so do not edit them. Put your own rules in CLAUDE.md or in the prompt.
3. Claude Code asks before it starts a project MCP server. The first time you open the project you approve playwright-test, and /mcp shows whether it is connected. If you decline by mistake, claude mcp reset-project-choices brings the question back.
4. Restart Claude Code after the first install. Claude Code picks up changed subagent files without a restart, but only in folders that existed when the session started. A brand new .claude/agents/ needs a new session.

One note on the docs: the "Agent Definitions" section of the Playwright page shows --loop=vscode under the heading for Claude Code and lists .github/ as the agent folder. That describes the VS Code setup. For Claude Code the command is --loop=claude, and the files go to .claude/agents/.

3. A worked example: the cart of a store

The store here is illustrative: a single product called "Ceramic mug" at $18.00, with the usual theme cart in a dialog. It is not a client, and the button copy and prices are invented for the example. Run the agents against a local server or a development store, never against production. They click real buttons.

The seed test

The seed tells the agents where the app starts and serves as the model for every generated file. Keep it short:

// tests/seed.spec.ts
import { test, expect } from '@playwright/test';

test.describe('Cart', () => {
  test('seed', async ({ page }) => {
    await page.goto('/products/ceramic-mug');
    await expect(page.getByRole('heading', { level: 1, name: 'Ceramic mug' })).toBeVisible();
  });
});

Planner, and the first review

In Claude Code, ask for the subagent by name and give it limits:

Use the playwright-test-planner subagent to plan the cart: add the product,
change the quantity, remove it. Seed: tests/seed.spec.ts. Do not cover checkout.

An excerpt of what it saves as specs/cart.md:

# Ceramic mug store - Cart Test Plan

## Test Scenarios

### 1. Cart

**Seed:** `tests/seed.spec.ts`

#### 1.1 Add a product to the cart

**Steps:**
1. On the "Ceramic mug" product page, click the "Add to cart" button

**Expected Results:**
- A cart dialog opens and lists "Ceramic mug" with quantity 1
- The cart link in the header reports 1 item
- The subtotal shows $18.00

#### 1.3 Remove the only item

**Expected Results:**
- The cart shows "Your cart is empty"

Gate 1: approve the plan. The plan is the contract for everything after it, so read it before generating anything. Check that every expected result is something a test can observe and is specific. "The subtotal shows $18.00" can fail; "the cart updates" cannot. Check that the rules of the business are there, such as a maximum quantity or a variant that is sold out, because the planner only knows what it saw on the screen. Delete scenarios that test the theme's demo content. Then edit the Markdown by hand and commit it. A plan that nobody approved is a description of the app as it is today, bugs included.

Generator, and the second review

Use the playwright-test-generator subagent to generate scenario 1.1 of specs/cart.md.

A file the generator could plausibly write, tests/cart/add-a-product-to-the-cart.spec.ts:

// spec: specs/cart.md
// seed: tests/seed.spec.ts

import { test, expect } from '@playwright/test';

test.describe('Cart', () => {
  test('Add a product to the cart', async ({ page }) => {
    await page.goto('/products/ceramic-mug');

    // 1. On the "Ceramic mug" product page, click the "Add to cart" button
    await page.getByRole('button', { name: 'Add to cart' }).click();

    // - A cart dialog opens and lists "Ceramic mug" with quantity 1
    const cart = page.getByRole('dialog', { name: 'Your cart' });
    await expect(cart.getByRole('link', { name: 'Ceramic mug' })).toBeVisible();
    await expect(cart.getByRole('spinbutton', { name: 'Quantity for Ceramic mug' })).toHaveValue('1');

    // - The cart link in the header reports 1 item
    await expect(page.getByRole('link', { name: /cart/i })).toContainText('1');

    // - The subtotal shows $18.00
    await expect(cart.getByText('Subtotal')).toBeVisible();
  });
});

Gate 2: read every locator and every assertion. The first three locators are good: a role plus the accessible name a customer hears from a screen reader. The last two lines are where it goes wrong, and they are the kind of thing a quick look approves.

• getByText('Subtotal') checks that a label exists. It passes with a subtotal of $0.00, or of $36.00. When a value has no accessible role, a test id is the honest choice: await expect(cart.getByTestId('cart-subtotal')).toHaveText('$18.00');
• { name: /cart/i } with toContainText('1') matches any link whose name contains "cart", and passes with 1, 10 or 12 items. Pin the exact accessible name your theme exposes, for example getByRole('link', { name: 'Cart (1 item)' }).

Also look for what should not be there: CSS classes, nth-child, generated ids, fixed waits with waitForTimeout. Then make the test fail once on purpose. Change '$18.00' to '$19.00', run npx playwright test tests/cart and confirm it goes red. A test you have never seen fail has not proven anything.

Healer, and the third review

A month later the button copy changes from "Add to cart" to "Add to bag". CI goes red with this error:

Error: locator.click: Test timeout of 30000ms exceeded.
Call log:
  - waiting for getByRole('button', { name: 'Add to cart' })

Locally, on a branch:

Use the playwright-test-healer subagent to fix "Add a product to the cart".

The healer runs the test, stops on the failure, reads the page snapshot, finds button "Add to bag" and proposes this change, then reruns until it passes:

-    await page.getByRole('button', { name: 'Add to cart' }).click();
+    await page.getByRole('button', { name: 'Add to bag' }).click();

Gate 3: validate each proposed fix. The patch is correct only if the rename was intended. Find the change that caused it, such as the theme's locale file or a pull request from the content team. If it was intended, accept it and update specs/cart.md as well, so the plan and the test say the same thing. If the button now said "Sold out" because a stock sync failed, the same move would retarget the test to a broken store, or mark it test.fixme() and skip it. Either way the build turns green while customers cannot buy. Only a person knows which case it is. Check four things on every healer diff:

• Did it change an expected value, such as a price, a count or a message? That is a change to what the store must do, not to how the test finds it.
• Did it widen a locator into a regular expression, such as /add to (cart|bag)/i? Accept that only if both texts are legitimate, for example during an A/B test.
• Did it add test.fixme()? That is a bug report. Open an issue instead of merging a skip.
• Did it touch anything outside tests/? Run git status.

4. The three risks, named

Tests that pass without testing anything. These are assertions on labels, visibility checks with no value behind them, and actions with no assertion after them. Watch for one subtler case. The generator checks its assertions against what the app does now, so if the app is wrong today, the test records the bug as the expected behavior. The remedy is gate 1: the plan should come from requirements, not only from exploring.

Fragile selectors. These are classes, positions and ids that change on every theme update. Text is fragile too when the store sells in several languages: "Add to cart" is "Agregar al carrito" on the Spanish market. Run the suite per locale with the right names, or use a test id for controls whose copy is translated.

Fixes that hide real bugs. Updated expected values, skipped tests and longer timeouts all turn a red signal into a green one. A longer timeout on add to cart can hide a performance problem customers feel, which we covered in why an add-to-cart button feels slow.

5. CI: the suite runs, the healer does not

The rule we use is that agents run on a developer's machine and propose, and CI only runs. CI uses the standard Playwright workflow, with no Claude Code, no MCP server and no agent:

- run: npm ci
- run: npx playwright install --with-deps
- run: npx playwright test
- name: Fail on skipped tests that nobody reviewed
  run: |
    if grep -rn "test.fixme(" tests/; then
      echo "test.fixme found: open an issue and review it"; exit 1
    fi

The fixme step is our addition, not part of the official workflow. Adjust it to your policy; some teams allow a fixme that links to an issue. Healer changes reach the main branch the same way as any other code: on a branch, in a pull request, with a reviewer who reads the diff.

Do not run the healer headless from CI with claude -p. In that mode Claude Code loads project MCP servers without the approval prompt, and the healer has edit and write tools. You would have a red build that turns itself green with nobody looking.

Locally, keep the session in the default permission mode, so Claude Code asks before each edit. Subagents inherit the session's permission rules, so you can also draw a hard line around app code while you heal, in .claude/settings.local.json:

{
  "permissions": {
    "deny": ["Edit(/src/**)"]
  }
}

Edit rules apply to every built-in tool that edits files. Change the path to wherever your app code lives; in a Shopify theme that is sections/, snippets/, templates/ and the rest.

Where this fits

The agents save the typing: exploring a flow, writing locators, chasing a renamed button. They do not decide what the store must do. That decision lives in the plan you approve, the assertions you read and the fixes you accept or reject. If you want the broader session around this, see my four-step workflow with Claude Code. For what to test on a store before launch, see a pre-launch QA pass for Shopify that finds real bugs.