ZYNLEX vs Playwright MCP
Both give a coding agent a browser to check its work. We gave Claude Code the same bugs with each one, three runs per bug. Both fixed every run. ZYNLEX did it with about a third of the tokens.
Medians over 8 bugs × 3 runs, Claude Code (Sonnet), 24/24 fixed by both. ZYNLEX before the fixes these runs led to: 222k tokens, $0.186, 5.5 browser calls. Harness and every result row.
The film: one bug, two real runs from the benchmark logs, played side by side at 2×.
One bug, step by step
The bug, as the agent got it: “Posting a comment on an event clears the box, but my comment never shows up.” The cause was the same in both runs: the page posted to /comment, the server only knew /comments. These are the middle runs of three for each tool.
| ZYNLEX | Playwright MCP | |
|---|---|---|
| Read the code | Glob, Read api.py, Glob, Read app.js | Glob, Read api.py and app.js |
| See the bug | 1 script call: open the event, type a comment, post it, read the list | browser_navigate, browser_find, browser_type, browser_click |
| Fix | Edit app.js | Edit app.js |
| Check the fix | The same script again: the comment shows | The same four steps. The page still ran the old app.js, so the post failed again. The agent checked the console, grepped, opened app.js by URL, evaluated, reloaded, and did the four steps a third time. |
| Browser calls | 2 | 20 |
| Turns | 8 | 26 |
| Input tokens | 169,712 | 598,255 |
| Cost | $0.155 | $0.366 |
Every bug
Median input tokens per bug, three runs each. “ZYNLEX” is the 0.19 build, made with these bugs in view; “ZYNLEX, blind” is the build before it, which had never seen them. Lower is better.
| Bug | ZYNLEX | ZYNLEX, blind | Playwright MCP |
|---|---|---|---|
| search shows unrelated events | 131k | 329k | 414k |
| wrong event page opens | 129k | 186k | 540k |
| wrong ticket price | 250k | 252k | 479k |
| overbooking allowed | 212k | 169k | 1,097k |
| comment never shows up | 169k | 128k | 598k |
| cancel booking does nothing | 156k | 257k | 467k |
| settings lost on reload | 211k | 180k | 499k |
| phone-size filters broken | 176k | 317k | 440k |
| Median, all 24 runs | 176k | 222k | 511k |
Same day, same laptop, same model. The other tools in the round: OpenCode Browse 290k, agent-browser 541k, Chrome DevTools MCP 565k.
Why the gap
Turns cost more than replies. Claude Code sends its whole conversation again on every turn, so most of a run’s tokens are the same context re-read. A run of 8 turns is cheap; a run of 20 isn’t, even when each browser reply is short.
One script instead of one action per call. ZYNLEX’s script tool takes a few lines of Playwright-style code: open the page, do every step, return just what the agent needs to see. Playwright MCP can run code too, but in these runs agents stepped through it one action at a time, as its tool list invites.
await page.goto('http://run9313.localhost:9313/#/event/2');
await page.waitForTimeout(500);
await page.fill('#comment-text', 'Testing comment bug');
const [resp] = await Promise.all([
page.waitForResponse(r => r.url().includes('/comment')),
page.click('#post-comment'),
]);
const status = resp.status();
const url = resp.url();
await page.waitForTimeout(300);
const comments = await page.locator('#comments li').allTextContents();
return { status, url, comments };
The agent’s first call in the comments run, as it sent it. It returned the 404 and the URL it went to, which is the bug.
The same window, every time. ZYNLEX runs in the window you have open, and the agent’s script can be run again unchanged after a fix. You can watch every step, flag an element with Ctrl+Shift+E and it reaches the agent as a note, or pause it and take over.
Which one to use
- You’re on Windows 10 or 11, building a web app on localhost
- You want to see what the agent does, point at bugs and take over
- Token cost matters: about a third of Playwright MCP’s here
- You also want a dev browser: network log, mocks without a proxy, device sizes, an API client
- You’re on macOS or Linux
- The agent runs in CI or headless, with no one watching
- You need Firefox or WebKit, not just Chromium
- You want Microsoft’s project and the wider Playwright ecosystem
| ZYNLEX | Playwright MCP | |
|---|---|---|
| Where it runs | Windows 10 / 11 | Windows, macOS, Linux |
| Browser | The ZYNLEX window you have open (WebView2, Chromium) | A browser it launches (Chromium, Firefox or WebKit, headed or headless), or your Chrome through an extension |
| Agents reach | localhost and staging you allow | Any site |
| Point at an element for the agent | Yes, Ctrl+Shift+E in the page you’re using | Partly: annotate in its own dashboard |
| Set up in Claude Code | claude mcp add zynlex zynlex-mcp | claude mcp add playwright npx @playwright/mcp@latest |
| Price | Free, Apache-2.0 | Free, Apache-2.0 |
How we measured, and the limits
Each run starts a fresh copy of a small web app with one planted bug. Headless Claude Code (claude -p, Sonnet) gets the bug report, file tools and exactly one browser tool. A separate grader checks the fix in Edge and marks it pass or fail. Tokens are every input token of the run, cache reads included.
- Our harness, our bugs, one machine, one model. That’s why everything is public to rerun.
- ZYNLEX ran in a real window, the others headless, so time isn’t a fair comparison; tokens and cost are.
- The 176k build was made with these bugs in view. The blind build, 222k, hadn’t seen them.
- Playwright MCP 0.0.85, ZYNLEX 0.19 (blind: main at
0ffb7b2), 2026-11-17.