How Octave built a secure multi-tenant AI foundation on Amazon Bedrock

Octave built a multi-tenant AI foundation for Reliance, its cloud-native quality management system, on Amazon Bedrock rather than shipping a single AI feature first. The company detailed the architecture in an AWS Machine Learning Blog post. Reliance supports more than 650 enterprise customers across over 40 applications, including complaint management, CAPA (Corrective and Preventive Action) tracking, audit management and supplier quality, and Octave operates in 180 countries. Its platform serves more than 14,000 customers in medical devices, pharmaceuticals and food manufacturing, with each customer held as a separate tenant.
Planning workshops with AWS identified four candidate use cases: complaint summarization and triage, audit readiness monitoring, incident narrative generation, and predictive quality risk scoring. Because dozens of applications involve different document types and workflows, the team decided that building AI for one in isolation would mean rebuilding infrastructure for the next. It built the horizontal platform first and then cut vertical features into it. Complaint triage went first to prove the architecture. Octave says the foundation has since cut deployment time per use case by 50 percent.
The design was shaped by four constraints. Tenant data — complaints, CAPAs, standard operating procedures and audit findings — must be unreachable across tenants, including through an errant AI query. Agent permissions had to be scoped to the requesting user instead of a shared service role and live only during execution. Ingested content, such as email attachments, scanned PDFs and images, is untrusted and may carry prompt injection. And more than 2 billion documents had to be embedded while onboarding even the largest tenant within a single working day, without sacrificing quality.
The resulting foundation has three layers. Data preparation embeds documents with Cohere Embed v4 on Amazon Bedrock and writes them to a per-tenant vector store on Amazon OpenSearch Service. Secure execution routes requests through AgentCore Gateway, a capability of Amazon Bedrock AgentCore, which applies authentication and policy before handing work to an isolated microVM session that calls an Anthropic Claude model on Bedrock. Amazon Bedrock Guardrails wrap both, filtering input on the way in and output before it reaches the handler. Keeping retrieval, execution and screening in separate stages lets each be enforced and evidenced independently.
Cohere Embed v4 was chosen against Octave's own corpus rather than public leaderboards. The model places text and images in a single vector space, so a photo of a cracked housing and a paragraph describing that defect are retrievable from one query without a separate image pipeline, and native multilingual support removes translation preprocessing for mixed-language records. In-Region inference was a precondition because customer data cannot leave its AWS Region; Embed v4 supports in-Region use in us-east-1, eu-west-1 and ap-northeast-1, covering three of Octave's regional compliance requirements.
Vector storage cost scales linearly with dimension count on OpenSearch Service, so the team tested output dimensions rather than accepting the default. At 512 dimensions, representative production queries held target accuracy while storage fell roughly 66 percent against the 1,536 default. The model's 128K token context window helped elsewhere: packing many chunks of about 4K tokens into one request drove ingestion throughput.
Isolation is enforced at the runtime rather than in application code, which the team reasoned would require every future feature team to get tenant checks right on every code path. AgentCore runtime is framework- and model-independent, so the isolation decision did not constrain the agent framework or models later chosen. Each user session runs in a dedicated microVM with its own CPU, memory and filesystem, stopped and sanitized when it ends. Per-user OAuth token vaults verify the agent acts for that specific user, credentials are scoped per agent and expire with the session, and AgentCore Gateway authorizes at the tool, operation and parameter level. Automated reasoning provides formal verification against defined rules.
On untrusted content, Octave applied Guardrails over the agent logic rather than inside it, at three points: text extracted from attachments before embedding, retrieved documents and tool outputs before they enter the agent's context, and the synthesized brief before a handler sees it. Attachment-derived content is input-tagged to mark it as data rather than instructions, which is what makes an injection buried mid-document detectable, and personally identifiable information is masked on the way in. Bounded agency sits underneath: Gateway governs which tools an agent can invoke, and extraction runs in the AgentCore Code Interpreter sandbox with no network egress and no persistence beyond the session. Detection and masking rates are tracked per tenant, so screening results can be evidenced.
Ingestion fans document events through Amazon SNS and Amazon SQS into an AWS Step Functions state machine, which orchestrates parsing, chunking and embedding across concurrent Lambda executions inside a dedicated VPC. Parsing preserves semantic hierarchy from Word files and spreadsheets, applies OCR to scanned PDFs without a text layer, and treats embedded images as first-class content. Chunks carry metadata — tenant, document type, source application, language and a content hash — which drives retrieval filtering and per-tenant namespace enforcement. Interleaving many chunks into single embedding requests cut API calls for a large corpus by about an order of magnitude. Octave reports Embed v4 delivered 28 million tokens per minute in its internal Bedrock testing, and that the pipeline sustained 1.4 million documents per 24 hours per tenant workload after batch tuning with Cohere's solutions architects.
The Complaint Triage Assistant is built with the Strands Agents SDK, which AWS open-sourced and which composes agentic workflows as graphs of tool and model calls. A Claude model summarizes each incoming record, and the summary rather than the raw multi-thousand-token source becomes the retrieval query against the tenant's OpenSearch index. A separate model call scores retrieved matches by confidence and discards results that share vocabulary but differ in root cause. Anthropic Claude Sonnet then assembles a brief with key facts, sentiment, urgency indicators, closest historical records with outcomes and a recommended priority. Completed briefs are cached in Amazon DynamoDB with a per-environment TTL. Requests reach AgentCore runtime through a private Amazon API Gateway and Lambda function, with attachments in Amazon S3.
For auditability, AgentCore captures OpenTelemetry-compatible traces across every graph node, so months later Octave can show which documents and prompts produced a given brief. Bedrock model invocation logging records every request and response per tenant, with costs attributed to the initiating tenant in Amazon CloudWatch. AgentCore Evals reruns the workflow offline against expert-annotated ground truth after changes to prompts, models or retrieval settings, reporting summarization faithfulness, retrieval precision at K and output completeness.
Octave says the assistant has run in production for three months and that it measures 4 to 6 hours per investigation as the work the pipeline absorbs: extraction across formats, the search for precedent, and the reading needed to tell a genuinely similar defect from a wording coincidence. By its own accounting, development cycles for new AI capabilities went from months to weeks.
Why it matters: Teams building AI on SaaS platforms with strict data isolation get a concrete reference architecture — per-tenant vector stores, hardware-level session separation, multi-point guardrails and embedding-dimension tuning — rather than a single feature design. The reported throughput and quality figures come from Octave's own testing and production window, so treat them as vendor accounts until independently reproduced. For regulated industries, the notable shift is that isolation and audit evidence were treated as platform concerns, which is likely to lower the cost of every subsequent AI feature.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Octave built a secure, multi-tenant AI foundation on Amazon Bedrock, using Cohere Embed v4 for embeddings and Amazon Bedrock AgentCore for isolated agent execution, to power a Complaint Triage Assistant that searches 2 billion regulated documents without crossing tenant boundaries.