AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Exowatt beats frontier models on accuracy with SageMaker serverless model customization

Collected Oct 27, 2026

Exowatt, which builds modular solar systems that store energy as heat and dispatch electricity on demand — including behind-the-meter power for AI data centers — published an account on the AWS Machine Learning Blog of how it built task-specific fine-tuned models on Amazon SageMaker AI serverless model customization. The company reports that two compact Qwen3 models, a 1.7B-parameter version for edge alarm triage and a 14B version for cloud-side energy-market analysis, each outperformed frontier Claude models on its narrow task, at up to 49 times lower cost per answer. The engagement ran from first experiment to a model in production in nine days.

The business problem was operational. Exowatt's energy management system raises alarms continuously across a growing fleet of sites — components out of spec, dropped connections, out-of-range voltages — and the company expects to scale toward thousands of field units. A human operator cannot watch every site in real time, and a cloud-hosted frontier model is a poor fit for three stated reasons: the triage logic is proprietary to Exowatt's hardware and data, many sites have no GPU and intermittent or no internet, and routing every repeating decision through a hosted model across a fleet is uneconomical. Before adopting the serverless service, each experiment required picking an open-weight model, building a container, writing a training recipe and waiting for a chosen GPU instance type to become available.

SageMaker's serverless model customization supplies ready-made fine-tuning recipes for public-hub open-weight models, so the team trains with LoRA (low-rank adaptation, which freezes the base weights and learns small adapter matrices) and supervised fine-tuning without managing a container or requesting capacity. The pipeline spans seven managed AWS services: S3 for data and artifacts, serverless fine-tuning, serverless evaluation, the SageMaker Model Registry, Bedrock Custom Model Import for cloud serving, Bedrock-hosted Qwen3 and Claude models for data paraphrasing and baselines, and Lambda plus EventBridge for a serverless scorer and a scheduled keep-warm call.

For edge alarm triage, the fine-tuned Qwen3-1.7B reads an alarm and site telemetry and returns one of a small fixed set of classifications with cited evidence, or abstains. The training set was 2,522 examples: 684 verified decisions on generated scenarios plus 1,838 certified paraphrases generated by Qwen3-235B on Bedrock for about $3.72, with each rewrite kept only if deterministic checks confirmed decisions, labels, citations and numbers were unchanged (1,838 of 2,218 candidates passed). One fault family was held out entirely and 320 cases were used for testing. Training took about 32 minutes for roughly $20–$25. The model is quantized to 8 bits at about 1.71 GiB and runs offline in a 6 GiB llama.cpp pod on the site's existing CPU server, inside Kubernetes, or on a t3.2xlarge instance.

The edge results: the fine-tuned Qwen3-1.7B scored 0.911 macro-F1 on 320 test cases against 0.698 for Claude Sonnet 4.6, 0.680 for Claude Opus 4.7 and 0.519 for Claude Haiku 4.5. The base Qwen3-1.7B failed a validity gate. Cost per 1,000 decisions is listed as $0 because the quantized model uses existing CPU with no metered API or GPU inference.

For energy-market analysis, the 14B model reads a six-hour packet of roughly 90 rows of real-time ERCOT prices, day-ahead forecasts and grid notices, and returns one of five market events in about four seconds, with every claim tied to a specific data row. The fine-tuning data came from five years of public ERCOT data, about 163 million rows across 12 tables, rule-labeled into 25,000 examples (22,000 train, 1,000 validation, 2,000 test) and expanded to 86,281 rows via Claude Sonnet 4.6 paraphrasing. Training was one LoRA epoch at rank 16 and alpha 32, about 8.5 hours, with up to four concurrent jobs; 23 jobs totaling roughly 155 hours ran in the six days from plan to a serving model. The merged checkpoint imported to Bedrock in about 14 minutes, with no idle cost and $6.86 per hour only while active. Paraphrase diversity lifted macro-F1 from 0.950 to 0.975.

The cloud results: 0.975 macro-F1 against 0.625 for Claude Opus 4.7 (up 0.35, 95% confidence interval 0.324 to 0.371), 0.538 for Claude Sonnet 4.6, and 0.415 for base Qwen3-14B. Cost was $0.60 per 1,000 answers versus $29.26 for Opus, about 49 times lower, and the model held 0.968 on a later test set.

A control experiment is the most instructive detail: training the 1.7B on only the 684 verified rows for 11 epochs, matching step count, reached just 0.879, against 0.911 with the paraphrase-expanded set and 0.584 with the original rows alone. The team also reports that 13 fine-tuned Qwen3-8B candidates across 12 dataset versions never beat the prompted 8B baseline of 0.735, though the 1.7B with better data did. Both models emit strict JSON, and an evidence fence drops any claim whose cited source, field or value does not literally match the input.

This reads as a well-instrumented case for a familiar pattern: narrow, repetitive, high-volume classification tasks reward small fine-tuned models over general-purpose ones, and the economics favor owning the artifact. Two caveats are worth keeping in mind. Macro-F1 on held-out sets built from the same rule-labeling and scenario-generation process measures agreement with Exowatt's labels, not field outcomes; the company says the edge model ran autonomously in shadow mode on a cloud replica of a site, which is a safe first step but not a deployment track record across thousands of units. And the $0 edge figure covers inference only, excluding quantization, packaging and the operational cost of fleet-wide model distribution. The 49x cloud comparison also depends on measured token counts per answer against Bedrock list prices, so it will move if answer length or pricing changes.

Why it matters: Teams building agents for physical or offline environments now have a concrete template — serverless fine-tuning, paraphrase-expanded certified data, strict cited JSON, and split serving between quantized edge and elastic cloud endpoints. The reported numbers suggest that for repetitive, well-labeled tasks, a 1.7B model can beat frontier APIs on accuracy and eliminate per-call cost, an inference that pushes architecture choices toward owned checkpoints and away from hosted general models where connectivity and volume are constraints.

Read at AWS Machine Learning Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

In this post, we show how Exowatt used Amazon SageMaker AI serverless model customization to build a repeatable AI pipeline that produces task-specific fine-tuned models for two deployment targets, edge and cloud, achieving higher accuracy than frontier models at up to 49x lower cost.