AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Building intent-driven UI testing with Amazon Nova Act

Collected Oct 29, 2026

Epsilon, a data, technology and marketing-services company owned by Publicis Groupe, has described how its internal Agentic Platform team built intent-driven UI testing as a managed service on top of Amazon Nova Act and the Amazon Bedrock AgentCore Browser Tool. The account, published on the AWS Machine Learning Blog, covers the architecture, integration code, a representative 18-step workflow, best practices and internal benchmarks. Epsilon says the capability is running in production across multiple product teams.

The context is additive rather than a replacement. Epsilon already had a Playwright-based regression suite covering its web applications, and that remains the regression backbone. Playwright tests are deterministic and effective, but they are bound to selectors and page structure, so cosmetic UI changes can break them even when functionality is intact. The platform team also faced duplication: each product team that wanted full coverage had to stand up its own browser infrastructure, manage dependencies and wire in identity systems. The goal was to layer natural-language testing on top of the existing suite and expose it through the same API surface teams already use for other agent capabilities.

Epsilon's platform already serves a skills-based agent builder, federated identity and pre-flight authorization, Model Context Protocol integration and cross-account provisioning. UI testing became one more managed capability. It is surfaced as a standard agent, ui-test-agent, with two skills: ui-test-recorder, which explores a scenario interactively and records natural-language steps, and ui-test-executor, which runs recorded steps and returns structured results.

Behind it sits a three-part design: a platform API for creating workflows, a background task runner for executing them, and the Nova Act plus AgentCore Browser integration that drives the browser. A user creates a workflow with a target_url and a list of plain-English steps. On creation the platform registers a Nova Act workflow definition on AWS. On execution, a Celery task runner opens a managed browser session and runs each step through Nova Act over a Chrome DevTools Protocol WebSocket, the standard interface for controlling Chromium browsers. The system captures a screenshot per step, tracks pass/fail per step, computes an overall pass rate, and stores the results JSON, screenshots in Amazon S3 and the session ID, with a completion hook recording the outcome back into the platform.

The layered stack runs from an Agent Cloud frontend through an API gateway with authentication and permission checks, into a Django REST API layer exposing endpoints for workflows, executions, automation suites and stats. Business logic spans a Workflow Manager, Execution Orchestrator and Suite Coordinator. Jobs queue in Amazon SQS and are picked up by a Runner Service on Amazon ECS running the Nova Act engine and headless Chromium; results, screenshots, trajectories and HTML reports land in Amazon S3.

The core integration is short: a boto3 session, a managed browser session that yields a WebSocket URL and headers, and a NovaAct context opened with that CDP endpoint, the session headers and a starting page, after which each instruction runs through act(). Epsilon wraps this primitive with its own multi-tenant orchestration, result capture and observability.

The test workflow Epsilon uses as an example exercises login, form entry, conditional handling, scrolling and verification across eight listed instructions, from typing a username and choosing Next through confirming a newly created agent appears in a list. Two of them show where the approach differs from scripted testing: skipping an onboarding tour if it appears, and scrolling until the Goal Statement field or skills search box is visible. In a traditional framework those need explicit branching; here they are expressed in language. A notable operational detail: Epsilon runs tests in a dedicated FastAPI sidecar that pins nova-act 3.x with Workflow and IAM authentication support, keeping dependencies away from the main backend's Strands agent stack, and falls back to in-process execution if the runner is disabled.

On security and tenancy, Nova Act authenticates through the standard AWS credential chain, so no separate API keys are distributed, and the service slots into existing AWS STS AssumeRole provisioning. Each onboarded team runs under a scoped role in its own account, which keeps isolation and auditability. Onboarding means provisioning a cross-account IAM role, registering target URLs and granting access to the agent.

Epsilon's internal benchmarks put authoring an 18-step workflow at 5–10 minutes versus 30–90 minutes for the equivalent Playwright script of roughly 80–150 lines, a 4–6x difference. After an initial explore phase of typically 4–5 prompt iterations, a successful trajectory is saved and replayed deterministically; Epsilon reports a 100 percent pass rate on those replays and says repeat runs skip redundant model reasoning, cutting latency and cost. Epsilon also credits built-in CloudTrail audit and session recording with easing debugging and compliance work, and per-step screenshots with faster diagnosis.

The team's advice: write steps as intent rather than mechanics; isolate the Nova Act runtime in its own service with a lockfile, noting that the Nova Act signing endpoint targets us-east-1 regardless of the browser session's region; reuse recorded trajectories; design for multi-tenancy from the start with cross-account AssumeRole and scoped policies; lean on managed observability instead of building custom tooling; and account for AgentCore Browser Tool viewport defaults when writing scrolling instructions.

This suggests the pattern is most valuable where interfaces churn often and where non-engineers should be able to contribute test coverage. A likely trade-off is that language-driven steps trade determinism for resilience: replay of saved trajectories gives repeatable runs, but first-pass authoring still depends on a model interpreting the page, and Epsilon's numbers are internal benchmarks rather than an independent comparison. Teams already invested in Playwright may find the additive framing apt, with scripted checks guarding precise behavior and intent steps absorbing layout drift.

Why it matters: product and platform teams weighing agentic test tooling now have a detailed reference architecture, including the AWS identity, isolation and runtime-pinning choices that make multi-tenant delivery workable. If Epsilon's authoring numbers generalize, the main shift is who writes tests, since plain-English steps let product managers and QA leads review coverage directly, though inference about broader cost savings should be validated against a team's own applications and failure modes.

Read at AWS Machine Learning Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Epsilon's Agentic Platform team delivers UI testing as a managed, multi-tenant service built on Amazon Nova Act and Amazon Bedrock AgentCore Browser Tool. Tests are written in plain English and run on browsers managed by AWS, giving every product team resilient, intent-driven testing with zero per-team infrastructure overhead.