AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Bug0 Releases QA Agent to Automate Regression Testing with Cached Playwright Replays

Collected Oct 29, 2026

Linearbytes has released Bug0 QA Agent, a source-available library built on Playwright for agentic browser regression testing. The project was covered by InfoQ in a report by Bruno Couriol. Developers write declarative test steps in plain English inside ordinary Playwright specs, and an AI agent translates them into concrete browser interactions the first time, caching the result so later runs can replay it in CI without invoking a model.

The design sits between two established practices. Conventional web test suites rely on selectors such as CSS classes, attributes and data-testid hooks, which break when implementation details shift even if the behaviour under test has not changed, forcing rewrites. They also suffer from flakiness rooted in the asynchronous, distributed nature of web applications: race conditions, latency and dropped calls. Fully agent-driven suites avoid brittle selectors because the model interprets the page at run time, but each step can add seconds of latency, raise token costs and introduce its own non-determinism.

Bug0's hybrid approach separates the expensive exploration from routine execution. On a first run, the agent examines the accessibility tree and screenshots to understand the interface and carry out the requested action. Once a single-action step succeeds, the framework compiles that interaction into concrete browser operations and stores them in an in-memory Redis cache. Subsequent runs replay the cached action directly through Playwright, skipping the LLM inference that would otherwise accompany every step.

The caching has an important boundary. It applies only to single-action steps; complex, multi-action directives go back through the agent on every run, so those steps keep paying the latency and cost that caching is meant to avoid. This suggests teams get the biggest benefit from suites decomposed into granular steps, while long compound instructions remain roughly as expensive as a pure agent approach.

Recovery is built around failure. When the interface changes in ways that invalidate a cached action, such as moved button coordinates or restructured form hierarchies, the replayed Playwright operation may throw. Rather than failing the whole run, QA Agent detects the break, drops the affected cache entry and re-invokes the multimodal agent loop to work out the new interaction.

Assertions work differently from actions: they are never cached. Every pass routes them through vision models, with Claude and Gemini each evaluating the assertion independently and a third model breaking ties. Video models are used to catch transient UI defects that a single snapshot would miss, such as auto-dismissing toast alerts or flashing banner errors. The trade-off is that verification cost and latency persist even when actions themselves replay instantly, which matters for suites with many assertions.

The library also handles authentication flows. It embeds disposable email testing through mailsink and community-built adapters such as Mailinator, letting tests pull one-time passwords and magic links using prompts like {{email.otp:Extract the 6-digit code}}. For product teams testing sign-up and login journeys, that removes a common source of manual setup.

The stated approach connects to open research questions about agent reliability. A paper by Chevrot and colleagues, Are Autonomous Web Agents Good Testers?, published in April 2025 using GPT-4o, Sonnet and Gemini, found the best-performing autonomous testing architecture agreed with human professional testers in only about 60% of runs. The authors identified assertion verifiability and grounding, meaning correctly locating the relevant elements on the page, as the main levers for scaling such agents. Bug0's multi-model consensus on assertions and its cached, replayed actions can be read as engineering responses to exactly those two weak points, though the release does not publish accuracy figures of its own.

QA Agent ships under the Functional Source License, Version 1.1, with an ALv2 Future License. That is a source-available rather than open-source licence: it carries non-compete restrictions for the first two years, after which the code converts to Apache 2.0. Sentry, the licence's first proponent, framed it as granting freedom without harmful free-riding. For companies weighing adoption, the practical consequence is that the code is inspectable and usable now, but building a competing product on it during that window is restricted.

A likely trade-off for adopters is operational: an in-memory Redis cache and vision-model calls add infrastructure and external API dependencies to what would otherwise be a self-contained Playwright run, and model choices for tie-breaking are not specified. Background from the wider ecosystem is that self-healing test tools and AI test-generation products have proliferated, but most either regenerate selectors or run agents live; caching compiled actions is a less common middle path.

Why it matters: QA and platform teams maintaining large regression suites are the immediate audience, since replay-based caching could cut both CI time and token spend compared with agent-per-step tools, while keeping resilience to UI refactors that selector-based tests lack. Developers should note that caching covers only single-action steps and assertions still incur model calls every run, so the savings depend on how a suite is structured (an inference from the stated design). The Functional Source License also means product teams planning to build competing testing tools should check the two-year non-compete terms before adopting it.

Read at InfoQ · AI, ML & Data Engineering

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

QA Agent is a new source-available Playwright library for agentic browser regression testing. The agent associates test instructions expressed in natural language to actions on the user interface, caches those when appropriate for faster CI replays, remapping them to test specifications when the user interface changes. Verification is based on multi-model consensus and vision model evaluations. By Bruno Couriol