ZYNLEX
Benchmark · browser MCPs for Claude Code

ZYNLEX vs Playwright MCP

Both give a coding agent a browser to check its work. We gave Claude Code the same bugs with each one, three runs per bug. Both fixed every run. ZYNLEX did it with about a third of the tokens.

176ktokens per bug with ZYNLEX
511ktokens per bug with Playwright MCP
$0.165per bug, vs $0.347
2browser calls per bug, vs 14.5

Medians over 8 bugs × 3 runs, Claude Code (Sonnet), 24/24 fixed by both. ZYNLEX before the fixes these runs led to: 222k tokens, $0.186, 5.5 browser calls. Harness and every result row.

The film: one bug, two real runs from the benchmark logs, played side by side at 2×.

One bug, step by step

The bug, as the agent got it: “Posting a comment on an event clears the box, but my comment never shows up.” The cause was the same in both runs: the page posted to /comment, the server only knew /comments. These are the middle runs of three for each tool.

ZYNLEXPlaywright MCP
Read the codeGlob, Read api.py, Glob, Read app.jsGlob, Read api.py and app.js
See the bug1 script call: open the event, type a comment, post it, read the listbrowser_navigate, browser_find, browser_type, browser_click
FixEdit app.jsEdit app.js
Check the fixThe same script again: the comment showsThe same four steps. The page still ran the old app.js, so the post failed again. The agent checked the console, grepped, opened app.js by URL, evaluated, reloaded, and did the four steps a third time.
Browser calls220
Turns826
Input tokens169,712598,255
Cost$0.155$0.366

Every bug

Median input tokens per bug, three runs each. “ZYNLEX” is the 0.19 build, made with these bugs in view; “ZYNLEX, blind” is the build before it, which had never seen them. Lower is better.

BugZYNLEXZYNLEX, blindPlaywright MCP
search shows unrelated events131k329k414k
wrong event page opens129k186k540k
wrong ticket price250k252k479k
overbooking allowed212k169k1,097k
comment never shows up169k128k598k
cancel booking does nothing156k257k467k
settings lost on reload211k180k499k
phone-size filters broken176k317k440k
Median, all 24 runs176k222k511k

Same day, same laptop, same model. The other tools in the round: OpenCode Browse 290k, agent-browser 541k, Chrome DevTools MCP 565k.

Why the gap

Turns cost more than replies. Claude Code sends its whole conversation again on every turn, so most of a run’s tokens are the same context re-read. A run of 8 turns is cheap; a run of 20 isn’t, even when each browser reply is short.

One script instead of one action per call. ZYNLEX’s script tool takes a few lines of Playwright-style code: open the page, do every step, return just what the agent needs to see. Playwright MCP can run code too, but in these runs agents stepped through it one action at a time, as its tool list invites.

await page.goto('http://run9313.localhost:9313/#/event/2');
await page.waitForTimeout(500);
await page.fill('#comment-text', 'Testing comment bug');
const [resp] = await Promise.all([
  page.waitForResponse(r => r.url().includes('/comment')),
  page.click('#post-comment'),
]);
const status = resp.status();
const url = resp.url();
await page.waitForTimeout(300);
const comments = await page.locator('#comments li').allTextContents();
return { status, url, comments };

The agent’s first call in the comments run, as it sent it. It returned the 404 and the URL it went to, which is the bug.

The same window, every time. ZYNLEX runs in the window you have open, and the agent’s script can be run again unchanged after a fix. You can watch every step, flag an element with Ctrl+Shift+E and it reaches the agent as a note, or pause it and take over.

Which one to use

ZYNLEX
  • You’re on Windows 10 or 11, building a web app on localhost
  • You want to see what the agent does, point at bugs and take over
  • Token cost matters: about a third of Playwright MCP’s here
  • You also want a dev browser: network log, mocks without a proxy, device sizes, an API client
Playwright MCP
  • You’re on macOS or Linux
  • The agent runs in CI or headless, with no one watching
  • You need Firefox or WebKit, not just Chromium
  • You want Microsoft’s project and the wider Playwright ecosystem
ZYNLEXPlaywright MCP
Where it runsWindows 10 / 11Windows, macOS, Linux
BrowserThe ZYNLEX window you have open (WebView2, Chromium)A browser it launches (Chromium, Firefox or WebKit, headed or headless), or your Chrome through an extension
Agents reachlocalhost and staging you allowAny site
Point at an element for the agentYes, Ctrl+Shift+E in the page you’re usingPartly: annotate in its own dashboard
Set up in Claude Codeclaude mcp add zynlex zynlex-mcpclaude mcp add playwright npx @playwright/mcp@latest
PriceFree, Apache-2.0Free, Apache-2.0

How we measured, and the limits

Each run starts a fresh copy of a small web app with one planted bug. Headless Claude Code (claude -p, Sonnet) gets the bug report, file tools and exactly one browser tool. A separate grader checks the fix in Edge and marks it pass or fail. Tokens are every input token of the run, cache reads included.

  • Our harness, our bugs, one machine, one model. That’s why everything is public to rerun.
  • ZYNLEX ran in a real window, the others headless, so time isn’t a fair comparison; tokens and cost are.
  • The 176k build was made with these bugs in view. The blind build, 222k, hadn’t seen them.
  • Playwright MCP 0.0.85, ZYNLEX 0.19 (blind: main at 0ffb7b2), 2026-11-17.