AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Natural language security queries at Orca Security using Amazon Bedrock

Collected Oct 13, 2026

Orca Security built a natural language query agent called Discovery AI on Amazon Bedrock that turns plain-English security questions into valid Sonar queries, the domain-specific filter language behind its cloud security platform. The company reports that against its initial proof-of-concept baseline, the production system cut query costs by 90 percent, input token usage by 93 percent and end-to-end latency by 75 percent. Discovery AI has handled thousands of queries per day in production since March 2026, according to the AWS Machine Learning Blog post describing the work, which Orca produced with AWS.

The problem Discovery AI addresses is a schema-familiarity barrier. Sonar spans more than 900 cloud infrastructure and application models, the resource types it tracks, such as an Amazon EC2 instance or an Amazon S3 bucket. Querying that data means writing JSON objects that name models exactly, choose among 18 operators (comparison functions such as equals or contains), and traverse parent-child relationships across cloud providers. Analysts who lack deep knowledge of the Sonar schema cannot easily self-serve answers.

Orca's first attempt took the obvious route: load the entire Sonar schema into a single prompt on the largest available model, including all 900-plus model definitions, every relationship type and a large set of example queries. Output quality was good and drew positive responses from design partners and internal users, but three problems appeared at production scale. Each workload consumed roughly 2 million input tokens, driving up cost; responses took around 60 seconds end to end; and accuracy on complex multi-model queries was limited because nested relationships and multi-condition requests often needed extra refinement.

The production architecture is a nine-stage pipeline that blends retrieval augmented generation with schema-aware query construction, with each stage assigned to the Amazon Bedrock model best suited to it. A request first picks up context such as the user's organization, connected cloud providers and request metadata. A deterministic quick matcher then uses regex and rule-based matching to spot structured inputs, including IP addresses, CVEs, ARNs, file hashes, Amazon Machine Image IDs, Oracle Cloud Identifiers, EC2 IDs, URLs and alert IDs. A high-confidence hit can return a predefined filter immediately, skipping model inference entirely.

Requests that survive that fast path pass a guardrail stage that checks whether the input is safe, relevant, technical and supported, blocking prompt injection attempts, off-topic requests and unsupported inputs before they reach the rest of the pipeline. Approved requests go to example retrieval, which uses Amazon Titan Text Embeddings to pull the 20 nearest previously validated query examples from a vector store by semantic similarity. Cohere Rerank 3.5, also on Amazon Bedrock, then reranks those candidates down to the five highest-signal examples. If an exact or high-confidence match exists, the pipeline can reuse a known-good filter directly.

Otherwise a model selector running on Anthropic Claude Haiku 4.5 narrows the 900-plus Sonar models to the small relevant set, drawing on the request, connected providers and the retrieved examples. It passes each selected model to the generator at an appropriate schema granularity, from compact micro schemas to fuller mini schemas, to keep context small. The generator is a tool-augmented agent, also on Claude Haiku 4.5, that reasons over the selected schemas, retrieved examples, user context and available tools, iteratively refining a filter. Every model, field and operator it emits must resolve back to those schema definitions.

Validation runs twice: static syntactic and semantic checks, then verification against Orca's Sonar API. Failures return to the generator with the offending element identified, so retries are re-grounded rather than re-guessed. If the maximum retry count is exhausted without a valid filter, the pipeline returns an invalid-filter response. Across the pipeline, Amazon Nova 2 Lite handles the guardrail and validator, where fast, cheap inference matters more than deep reasoning, while Titan supplies embeddings and Cohere Rerank 3.5 handles relevance.

Example retrieval turned out to be the single biggest accuracy lever. Orca benchmarked retrieval strategies: pure top-K semantic search on Titan embeddings was fast but returned near-duplicates that starved the generator of diversity, while Maximal Marginal Relevance improved diversity but lost relevance when the top candidate was the right answer. The winning approach combined top-20 nearest-neighbor retrieval with Cohere Rerank 3.5 down to five, giving a small, diverse, highly relevant example set without trading coverage against precision.

The team also built a continuous evaluation harness early, combining a curated dataset reflecting production question types, model families and complexity tiers with an LLM-as-judge layer that grades generated queries on correctness, completeness and semantic equivalence to a reference query. Deltas are broken down by slice so regressions can be traced to their cause. Every change to a prompt, model, retrieval parameter or example passes through the harness before shipping.

For long-term quality, Orca built an internal labeling tool used by product managers, support engineers and security researchers to flag queries, correct outputs and push corrections into the example bank in one action. Production queries with low-confidence example matches are routed automatically to the right domain expert for daily labeling; the next pipeline run retrieves the new example through Cohere Rerank 3.5, improving accuracy on that question class immediately and without a deployment. Or Gabay, Director of AI at Orca Security, is quoted calling the labeling interface the single most important investment the team made for long-term quality, turning Discovery AI from a static deployment into a living system.

The measured results: input tokens per comparable workload dropped from roughly 2 million to 140,000, a 93 percent reduction; overall cost fell 90 percent, helped by per-stage model allocation as well as lower token consumption; and end-to-end response time went from about 60 seconds to 15 seconds, a 75 percent improvement. Accuracy on complex multi-model queries also improved, tracked continuously through the evaluation harness. Orca chose Amazon Bedrock in part because a single API gives multi-model access, so stages can run different foundation models without separate integrations, and because it inherits AWS governance controls through IAM, VPC endpoints and CloudTrail. The model-agnostic design also lets the team swap in newer model versions without rearchitecting.

Why it matters: teams building natural language interfaces over large, structured schemas now have a concrete, measured template. The pattern of widening the candidate pool cheaply, reranking to a handful of diverse examples, selecting only the schema fragments needed, and routing each pipeline stage to a model chosen for its cost and reasoning profile is the reason the reported token and latency figures fell so far, and the human-in-the-loop labeling loop is what keeps such a system from decaying as environments and question patterns change. This suggests that for developer and product teams, the durable investment is not the prompt or the base model but the retrieval corpus, validation feedback and evaluation harness around them. Inference: the approach should transfer to other domain-specific query languages with comparable schema complexity, though the reported gains depend on having a body of previously validated queries to retrieve from and an API that can verify generated output.

Read at AWS Machine Learning Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Orca Security built Discovery AI, a natural language query agent on Amazon Bedrock that converts plain-English questions into Sonar queries. The post shares the multi-stage architecture and the results: 90% lower query cost, 93% fewer input tokens, and 75% lower latency than the initial baseline.