[AINews] Agents disprove Polysemanticity; greatly improve Interpretability & Alignment

Meta released Muse Spark Open on 15 October 2026, an open-weight mixture-of-experts model with 816B total parameters and 42B active per token, plus a context window of one million tokens. It accepts text, image, video and audio as input and is positioned for coding, long-horizon agents and multi-step reasoning. The weights ship under a custom Muse Model License, which requires a "Built with Muse Spark Open" attribution and a separate agreement for Model-as-a-Service businesses earning more than $20M in annual revenue.
Jack Rae says it was the first base model trained under Meta's new MSL org, and that additional training was done for the open release. Several observers describe the model as roughly Muse Spark 1.2 level. Nathan Lambert was blunt about the license, calling it "trash" — a reminder that open weights and permissive terms are separate questions, and that the $20M MaaS clause effectively reserves commercial hosting for parties willing to negotiate.
Architecturally, vLLM reports 62 decoder layers, all of them MoE, with 256 experts per layer and 8 active per token. A latent-MoE pooler projects the hidden state from 8,192 down to 4,096 for expert computation and back again. Attention is hybrid: 46 layers use a 2,048-token sliding window while 16 attend across the full context. Images and video pass through a Perception Encoder, while audio uses a separate 118M-parameter encoder. Elie Bakouch notes Muse is the only major US open model without shared experts; it also uses gated residuals, roughly nine norms per layer and muP-like multipliers, including a router factor of 1/sqrt(number of experts).
Artificial Analysis scores it 39 on its Intelligence Index. That puts it first among US open-weight models but behind GLM-5.3 at 45 and Kimi K3 at 44. It uses 40k output tokens per index task, the fewest among the nine highest-scoring open models, and leads open models on AA-Omniscience at 25. Its weak spot is agentic work: it ranks last among those nine on the index's four agentic evaluations, and GLM-5.3's entire six-point lead comes from that segment.
Launch pricing on Together, Fireworks and Baseten is $1.25 per million input tokens and $4.25 per million output, with cached input at $0.15. Artificial Analysis calculates that at $0.84 per index task, against $2.01 for GLM-5.3 and $0.27 for DeepSeek V4.1 Flash. Together offers a free endpoint until 18 October. The BF16 weights are about 1.6TB; the NVFP4 checkpoint quantizes only the experts and comes to roughly 530GB, which runs on 4× Blackwell Ultra, 8× H200, or two 512GB M5 Ultra Mac Studios via MLX.
The ecosystem moved fast. llama.cpp had GGUF support on day 0, and Unsloth published quants, with UD-IQ1_S at 176GB. vLLM's day-0 stack covers NVIDIA, AMD MI355X in FP8 and MXFP4, and TPU. On TPU, megakernels with DSpark speculation at acceptance length 6 exceed 650 tokens per second on 4 TPU v7 chips and 960 tok/s on 8. Inferact trained a block-diffusion DSpark drafter that proposes up to 7 tokens per draft.
For teams weighing an open model, the picture is nuanced. A 39 on the Intelligence Index at roughly a fifth of GLM-5.3's cost per task is a genuinely different trade-off from chasing the top score, and the 1M context plus native audio and video opens use cases that text-only rivals cannot serve directly. The agentic gap is the number to watch: my reading is that a model ranking last among its peer group on multi-step tool use will disappoint anyone reaching for it specifically for long-horizon agents, which is one of its stated targets. The license clause matters similarly — under 20M in MaaS revenue the attribution requirement is mild, but product teams planning to resell inference should treat the negotiation as part of the roadmap. This suggests Meta is optimizing for adoption and cost efficiency rather than benchmark leadership.
Separately, Goodfire's Nick Cammarata published an essay arguing that neural networks are already clean — that many known cases of polysemanticity and superposition, which produce tangled knots of weights that resist human interpretation, fall away under further agentic analysis. Superposition is the idea that a network encodes more features than it has dimensions, overlapping them so single neurons fire for unrelated concepts; polysemanticity is the visible symptom. The claim that scaling interpretability tooling with agents resolves these cases would, if it holds, lower the assumed difficulty of alignment work substantially.
Why it matters: developers choosing an open-weight model now have a cheaper, long-context, multimodal option with day-0 support in llama.cpp, vLLM and Unsloth, but should test agentic workloads themselves before assuming the headline score transfers. Product teams planning to host Muse Spark Open commercially need to read the $20M MaaS clause carefully. The interpretability argument, if it survives scrutiny, would suggest that agent-driven analysis — not new theory — is where alignment progress is coming from, which in turn implies tooling investment is a reasonable bet for labs. That last point is my inference, not a reported result.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
Nick Cammarata out of nowhere with some major claims!