5 production challenges in multi-agent systems and how Deep Insight solved them

AWS's Solutions Architecture team has published the first post in a three-part series describing Deep Insight, a production-grade multi-agent system it built and has now open sourced on GitHub. The series also documents how the team applied Context Engineering techniques proposed by Anthropic in a real production environment, along with the trade-offs and operational lessons from running the system on AWS. Part 2 will cover Context Engineering; Part 3 will cover harness engineering and the move from development to production.
Deep Insight turns data analysis into an automated pipeline. A user supplies a CSV file, a column definitions file and a natural-language prompt, and a hierarchy of eight agents collaborates to load the data, run analyses, generate charts, validate calculations and produce a professional .docx report with citations. The agents are Coordinator, Planner, Plan Reviewer, Supervisor, Coder, Validator, Reporter and Tracker. The demo in the post uses advertising data from a fictional food company, Yummy Food, with the prompt asking for a sales and marketing analysis with charts, insights and a report.
There are three deployment options. A Self-Hosted version runs locally through a CLI with full code-level access to every agent, prompt and workflow, and sets up in about 10 minutes. A Web version offers a browser UI for uploading data, reviewing plans and downloading reports without writing code. A Managed AgentCore version runs on Amazon Bedrock AgentCore runtime entirely inside a private VPC, with a custom Code Interpreter hosted separately on AWS Fargate and agents running concurrently in isolated per-user environments. All three share the same architecture, so an agent validated in development can deploy to production unchanged.
The post frames five challenges: controlling execution flow across agents, choosing a model for each agent, operating the finished agent reliably, safely executing AI-generated code, and monitoring the system. The answers map to Strands Agents SDK, Amazon Bedrock, Amazon Bedrock AgentCore, Fargate with an Application Load Balancer, and DynamoDB, Amazon SNS and Amazon Cognito for an admin dashboard and failure alerts.
The execution flow uses two Strands patterns. A Graph pattern defines agent order as a directed graph: Coordinator to Planner to Plan Reviewer to Supervisor, with a conditional branch looping back to the Planner when the user requests revisions. An Agents-as-Tools pattern lets the Supervisor invoke the Coder, Validator, Reporter and Tracker as tools only when needed, each working in its own context and returning only results. The point is to keep each agent's context small while preserving domain expertise.
Model choice is deliberately split. The Coordinator only classifies and routes, so it uses lightweight Claude Haiku to avoid expensive calls on simple greetings. The Planner uses Claude Opus with Extended Thinking enabled to decompose requests into a five-to-ten step checklist stored in a full_plan variable, assigning concrete tasks to each agent. The post notes the Supervisor, Coder and Reporter use Prompt Caching for system prompts and tool definitions, which it says cuts input token costs by up to 90 percent on repeated calls; the Coordinator runs only once and the Validator and Tracker see different contexts each time, so caching stays off for them.
The Plan Reviewer is the human-in-the-loop node. It shows the plan for approval or revision, loops back to the Planner on feedback, and caps revisions with a MAX_PLAN_REVISIONS setting to prevent infinite loops; the graph also sets a maximum of 25 node executions. In the Web version the plan appears in a modal with a 300-second countdown that auto-approves if no response arrives.
The Supervisor enforces strict ordering via its system prompt: Coder, Tracker, Validator, Tracker, Reporter, Tracker. It does not advance until every checklist item for the current agent is marked complete, and it passes the clues and full_plan variables to each tool agent. The Tracker, which updates checklist items without tools, runs after every agent. The Validator independently recomputes each calculation recorded in calculation_metadata.json from the source data; matches within 0.01 pass, mismatches are flagged, and verified calculations are written to citations.json for the Reporter.
For code execution, generated Python is base64-encoded, sent over HTTP and written to a file, then run through a subprocess inside per-session ephemeral Fargate containers in a private VPC subnet with no direct internet access. ALB sticky session cookies keep an agent on the same container across multiple executions, and containers terminate automatically after one hour. VPC endpoints connect to Amazon S3, Bedrock and Amazon ECR without public internet traffic, with security groups and scoped IAM roles. This design targets the obvious risks of running model-written code: destructive file operations, runaway loops and cross-user data access.
The post is candid that container isolation does not govern generated content. The Validator re-checks calculations but does not detect prompt-injection instructions hidden in uploaded data or screen generated text for harmful or ungrounded material; for production, the team recommends adding Amazon Bedrock Guardrails as a responsible AI control. This is the most important caveat for anyone adapting the design, since a CSV upload is an untrusted input channel by default.
Why it matters: teams moving agents past pilots often find prompt tweaks do not fix skipped validation or ignored business logic, and this post offers a concrete reference architecture for that gap, with open-source code and three deployment paths. Developers can inspect routing, planning, verification and sandboxing decisions and reuse them in their own systems. The trade-offs are explicit: more agents and orchestration add latency and cost, partly offset by tiered models and selective caching. Inference: expect the AgentCore and Fargate patterns to be the most borrowed pieces, since isolated code execution and human plan approval are the controls most teams lack.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Many teams build a multi-agent AI prototype in days but hit a wall in production. This post defines five production challenges (orchestration, per-agent model selection, reliable operation, secure code execution, and monitoring) and shows how Deep Insight, an open source multi-agent system on AWS, solves each.