AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Production Patterns for Agents, Evals, and LLM Costs: QCon AI New York 2026

Collected Oct 13, 2026

QCon AI New York 2026 will take place on December 15 and 16 at The Westin Jersey City Newport, roughly a five-minute PATH train ride from the World Trade Center. The program committee has confirmed a slate of sessions aimed at senior engineers and technical leaders who are making production AI decisions, and the through-line is hindsight: speakers describe systems they have already run and the numbers behind them.

The framing the conference offers is that once AI reaches production, several pressures arrive together. Code shows up faster than teams can review it, token costs climb, evaluations have to be trustworthy enough to act on, and agents need permissions and guardrails. The confirmed talks are presented as accounts from senior engineers who have worked through those constraints in live systems rather than vendor pitches.

On agent identity and authorization, the opening keynote describes what it calls two bad options for granting an agent production access: issuing a long-lived service account and hoping, or scoping access so narrowly that the agent cannot complete its task. Nancy Wang, who founded and scaled AWS Data Protection, addresses the problem in a session titled When Software Becomes a User: Identity and Authorization for Agents in Production. A companion talk from Ronak Nathani, who led the move to Kubernetes at LinkedIn, covers Inside LinkedIn's Kubernetes Ops Agent: Skills, Tools, and Guardrails. That operations agent works across more than 500,000 nodes and more than 5 million pods, and the session covers server-side guardrails including rate limits and peer approval for production changes.

For practitioners, the pairing is useful because authorization models and guardrail implementations tend to be decided separately and then collide. A server-side rate limit or a peer-approval gate is only as good as the identity the agent acts under, so teams evaluating one of these talks will likely need the other to make the approach coherent. That is my reading of how the two sessions complement each other, not a claim the conference makes.

Evaluation and cost form a second cluster. Fabio Flores, who leads foundation development for Agentic Ordering at DoorDash, argues in From Signal To Impact: The Eval Loop Behind DoorDash's Agentic Ordering System that engineers should apply a single judge to both simulated and production sessions, so that improvements measured before launch carry through into production. The underlying idea is that an eval is only useful when engineers trust the score enough to ship on it or hold a release because of it.

Cost gets its own session from Tejas Chopra, who previously worked on distributed caching infrastructure at Netflix. In Make Each Token Count: Compression, Caching, and Retrieval for Production LLM Context, he covers prompt compression and ordering intended to increase provider cache hits, tiered storage for conversation history, and the failure modes his team encountered. Agent workloads that behave acceptably as prototypes can become expensive once context grows, which is a common pattern for teams moving from demo to sustained traffic.

Code review is the third cluster. Min Chen, who leads the move to AI-native software engineering at LinkedIn, presents High-Signal AI Code Review That Adapts to Your Codebase at Scale. Her suggestion is to filter low-confidence findings before developers see them. The system she describes reviews tens of thousands of pull requests per week across roughly 10,000 repositories, and she explains how the team measures whether suggestions actually make it into merged code. That last detail matters more than it might appear: a review bot's usefulness is normally judged by how many of its comments a developer accepts, and measuring insertion into merged code is a stricter test than counting comments posted.

A broader organizational view comes from Nathen Harvey, who leads DORA at Google Cloud, in Building in the AI-First World: Scale Brilliance, Not Bottlenecks. He explores assessing AI maturity beyond adoption metrics, identifying which existing problems AI is likely to make worse, and argues that AI amplifies what an organization already does well and what it does badly. Adoption numbers alone, on this view, say little about whether the underlying delivery process is in a state to benefit.

According to the session description, the talks are vendor-neutral with no hidden product pitches, and the ratio is one speaker for every eleven attendees, leaving room for follow-up questions with the engineers who built the systems. The stated aim is for teams to leave with patterns they can adopt and mistakes they can avoid. All confirmed talks are listed on the QCon AI New York 2026 sessions page.

Why it matters: Teams putting agents, evals or AI-generated code into production are the audience for this material, and the scheduled speakers hold relevant roles at AWS, LinkedIn, DoorDash, Netflix and Google Cloud. For engineering leaders, the practical value is comparing concrete approaches on agent permissions, guardrails, eval loop design, caching and review filtering against their own constraints before committing. Since these are conference sessions rather than documentation, the detail available beforehand is mostly the session abstracts, and the numbers cited for the LinkedIn systems are the headline figures rather than the full results.

Read at InfoQ · AI, ML & Data Engineering

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Once AI is in production, code arrives faster than it can be reviewed, token costs rise, evals have to be good enough to act on, and agents need permissions and guardrails. QCon AI New York 2026 (December 15-16) has confirmed sessions from engineers sharing how they handle these constraints. By Artenisa Chatziou