AivexaNewsSearch
AI news for builders and product teamsChecked every hour

AI’s hidden storage bill: What inference leaves behind

Collected Oct 22, 2026

WD has published a sponsored analysis, distributed through MIT Technology Review's AI section, arguing that storage is the under-planned half of AI infrastructure cost. The company's central evidence is its own internal measurement of an AI video workload: an 8-second clip delivered to a user as a 3.97MB 720p file left more than 500MB of related data on the back end. That figure covers logs, cached context, intermediate renders, backups and safety records, and works out to roughly 130 times the size of the file the user actually sees. WD says enterprise workloads producing longer or higher-resolution video could push the back-end footprint into the hundreds of gigabytes per clip.

The piece draws its case from the difference between training and inference. Training is finite, a run that starts, consumes compute and ends, so it can be budgeted and retired. Inference keeps running every time a model answers a prompt, and each request leaves artifacts behind. "You can think of having more storage as an optimization for keeping the GPUs busy," a senior product manager at a hyperscale cloud provider is quoted as saying in the report, adding that idle GPUs waiting on data become expensive quickly. In practice that framing matters more than it first appears: an accelerator sitting idle because a retrieval step is slow costs about the same as one idle for lack of allocation, but only the second shows up on the capital budget.

Several external figures support the growth argument. McKinsey projects inference demand growing at a 35% compound annual rate through 2030, against 22% for training, with inference accounting for more than 40% of total data center demand by the end of the decade. A WD-sponsored IDC white paper surveying 763 IT and business decision makers found nearly 95% reported data volume growth from generative AI over the previous 12 months, and about 61% reported growth of 25% or more. The same survey found 96% expect to need faster archive retrieval to support inference or retrieval-augmented generation.

The asymmetry WD highlights is that compute demand is cyclical while retained data is not. Model logs, synthetic outputs and inference results keep accumulating, and organizations are holding them longer because archived material has become an input to future work rather than dead weight. A retrieval pipeline that pulls old documents into a prompt turns a cold archive into a hot dependency. That changes what storage has to do: capacity still matters, but so does the latency of pulling something back from a tier that was previously designed to be rarely touched. WD's prescription is tiering, with hard disk drives carrying the bulk of retained data on cost per terabyte while faster media handle the hot path.

This suggests the budgeting point is the most portable takeaway. A team that treats storage as a rounding error next to compute may find, within 12 to 18 months on WD's reckoning, that it has become a meaningful share of program cost, arriving just as leadership weighs expansion. The retailer generating catalog videos and the insurer running claims through RAG both have predictable per-unit compute costs; storage compounds with every generation or query. The three questions the piece poses are worth borrowing regardless of vendor: what the retention policy for AI-generated content is and who owns it, whether storage purchasing is tied to training-time performance or to the actual lifecycle of the data, and whether doubling AI usage would double storage cost or worse. A likely trade-off is that aggressive retention cuts storage bills but degrades retrieval quality and audit trails, so retention decisions become product decisions. The obvious caveat is that the measurements and the survey come from a storage vendor with an interest in the conclusion; the McKinsey projection is the only cited figure not tied to WD.

Why it matters: teams shipping RAG systems, agents or generative media should treat storage growth as a first-class line item with an owner and a retention policy, since it scales with usage rather than with training cycles and surfaces as retrieval latency and operating cost rather than a visible hardware purchase. The specific 130x ratio is WD's own measurement of one video workload and should be read as illustrative rather than universal; the underlying pattern, that generated artifacts outlive the request that created them, is likely to hold across most inference products.

Read at MIT Technology Review · AI

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Ask most executives to picture the cost of AI and they’ll likely see GPUs—racks of accelerators with gigawatts of power—and a waitlist for silicon. But that picture leaves something out: Data storage costs. What happens to data after a model responds? Much of the back-end data that’s generated—from billions of daily prompts, images, and videos—needs…