Pluggable Agentic AI Framework: A seven-layer composable enterprise agentic AI reference architecture on Amazon Bedrock AgentCore

AWS solutions architects have published the Pluggable Agentic AI Framework, a reference architecture for multi-agent AI applications organized into seven layers, each with a distinct responsibility and a stable interface between it and its neighbours. The write-up appears on the AWS Machine Learning Blog and is credited to Sindhura Palakodety, Sujatha Kuppuraju and Mythili Annamalai Sekar. A companion hands-on implementation is offered through a Workshop Studio experience and a public sample repository, with the explicit caveat that it is a starting point rather than a substitute for production security hardening, testing and review.
The framing is that the hard part of agentic AI is not the model but everything around it. A proof of concept shows that a model can complete a task; it does not show how a system behaves under concurrent load, cost pressure or security review. Because every team tends to rebuild routing, guardrails and observability independently and bolt them on late, the framework proposes a shared blueprint with substitutable parts.
The seven layers, bottom to top:
Data and Knowledge owns retrieval and embeddings plus access to structured and unstructured data. Its contract is narrow: take a query, return relevant context. Reference building blocks include Amazon Bedrock Knowledge Bases, the Vector Engine for Amazon OpenSearch Serverless, Amazon S3 Vectors and Amazon DynamoDB.
Model and Inference is meant to be set by configuration rather than hardwired into application logic, so smaller and faster models can serve simple tasks while larger ones handle work needing nuanced reasoning. A routing proxy, with LiteLLM named as an example, presents one request-and-response contract across providers so models can be swapped after evaluation without rewriting code.
Orchestration decides how work is distributed. Simple tasks get a single agent; cross-domain workflows get a supervisor delegating to specialists, each scoped to a capability and a specific tool set. The post warns against a single do-everything agent: growing instructions, tools and context inflate the context window, raise latency, degrade accuracy and invite tool sprawl that can push an agent past its intended boundaries. Decomposition keeps context focused, makes latency more predictable, and lets teams test, scale and secure each specialist separately. Specialists are decoupled from supervisors and discovered through a centralized agent registry using standardized protocols such as MCP and A2A; the registry catalogs each agent's tools, skills and owner for governance and reuse. Suggested components are Strands Agents for specialists and the Amazon Bedrock AgentCore harness, AWS Agent Registry and AgentCore Gateway for orchestration and discovery.
Security and Guardrails is treated as a day-one design concern. Agentic systems face risks conventional frameworks were not built for, including prompt injection, tool misuse, and memory or data poisoning. External content is untrusted by default; every request is authenticated whether it originates from a user, another agent or an automated workflow, via something like AgentCore Identity, followed by fine-grained access control that decides whether a tool call may execute. Agents receive temporary scoped credentials rather than persistent secrets, sensitive data such as PHI and PII is masked in requests and responses, significant decisions land in a tamper-evident audit trail, high-impact actions pass a human approval gate, and circuit breakers halt anomalous behaviour. Amazon Bedrock Guardrails covers inputs, with AgentCore Identity and Policy in AgentCore plus AWS security and governance controls for defense in depth.
Application is deliberately thin. It presents the conversation and passes requests down; business logic belongs elsewhere so the interface can change without destabilizing the backend. A decision model can sit at this boundary as a cheap first call that returns probabilities for in-scope, needs-a-human or urgent, letting plain thresholds answer off-topic requests and flag escalations so the full agentic loop only runs when it adds value. Interfaces should also be designed for consumption by other agents, not just people.
Observability watches every layer from the start. Beyond logs and metrics, end-to-end agent traces are described as essential for judging whether a task was carried out correctly, alongside per-request cost and token usage, time to first token, whether a call is secure, and dashboards for latency, error rates and spend built before go-live. Amazon CloudWatch with OpenTelemetry, and CloudWatch Omni, are the reference choices.
Agentic Operations closes the loop with continuous evaluation: scoring every production session for groundedness, accuracy and safety, with a regression gate that catches quality drops before customers see them. AgentCore Evaluations handles scoring and AgentCore optimization handles tuning. Without it, you know only that the agent worked at launch; with it, you can show it still works after each release.
A ready-to-run reference implementation illustrates the whole stack with a retail customer support scenario covering order status and refund eligibility. A supervisor agent receives a refund request and delegates to an Order agent and a Refund agent. The Order agent looks up delivery and purchase details from DynamoDB; the Refund agent pulls tier-based policy rules from a knowledge base and computes eligibility from the delivery date, current date and the customer's loyalty tier. Every call passes through the AgentCore Gateway, which enforces fine-grained access control via Policy in AgentCore before a tool runs, and every call is traced end to end, so a wrong calculation can be traced to the layer and decision point responsible. Teams building a similar supervisor-and-specialist pattern can clone it and swap domain logic layer by layer.
The pluggability argument is that stable contracts at layer boundaries — identity checks, access policies, telemetry interfaces — let the underlying technology change without redesigning the stack. The post frames this as an architecture-first rather than technology-driven approach, and demonstrates substitutions from third-party providers across inference, search at the orchestration layer, decision APIs at the application layer, guardrails at the security layer and tracing and monitoring at the observability layer. For enterprises, the pitch is protecting existing investments: keep trusted observability, security and inference tools, fill gaps where needed and adopt incrementally rather than sourcing everything from one vendor.
Why it matters: teams moving agents from demo to production face the same recurring gaps — authorization before tool execution, tracing somewhere between the model and the tool, and evaluation after each release — and a published layer model with named reference services gives them a checklist and a vocabulary. The caveat is that the named AWS services are a reference implementation, not requirements, so the architectural guidance is separable from the vendor choice; teams on other stacks can adopt the layer boundaries while keeping their own components, though the worked examples and sample repository assume the AWS path. It is also presented as a starting point rather than a hardened system, so security review and load testing remain the team's own work.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
A seven-layer composable reference architecture for taking enterprise agentic AI from prototype to production on Amazon Bedrock AgentCore. Each layer has a stable contract, so you can swap technologies at any layer without redesigning the rest of the stack.