[AINews] Behind Latent Space: Kill My SaaS, Jev & AI in 2026 | AIEi Shanghai

The 2-3 November 2026 AI News roundup from Latent Space opens by revisiting the previous day's lead story on "Weight Space Programming," conceding that some claims were framed too favorably and distracted from the paper's more substantial findings. The editors argue the work still represents a significant moment and has drawn multiple independent replications, so they see no case for a retraction or a follow-up declaring it bogus. The same issue features AI Engineer Shanghai and an exclusive interview on Latent Space's own history.
On the core claim, several groups report that frontier large language models can generate or edit the weights of small networks directly, without gradient training or external tools. Alex Krentsel's replication of MIT's experiment found a sharp jump in capability across model generations, with the newest models producing Breakout networks scoring up to 22 times an RL-trained baseline. Hugging Face reproduced the paper using ML Intern, its open-source research harness, and released the model plus a demo. In a separate case, a user had a model called Fable hand-write a 100,000-parameter classifier for distinguishing real from fake legal citations, reportedly beating a trained version, a result Legora's CTO highlighted.
The most scrutinized result involves GPT-6 Astra editing 131,072 weights in Qwen3.5-0.8B without backpropagation, lifting MATH-500 from 7.9% to 48.1%. A reconstruction of the logs published on Zhihu reports the gain is narrower than the headline: 576 of 579 repaired answers had originally hit the 32K-token limit because of looping. Astra reportedly derived a "repetition direction" from contrastive looping and productive contexts, located it mainly in the MLP layers at positions 15 and 19, and subtracted six times the projection from the top 64 aligned neurons per layer. On 1,152 held-out problems, accuracy rose from 10.6% to 60.4%; most of that gain reflects existing reasoning that can now run to completion. Simon Willison separately reports that Qwen 3.8 27B managed the same weight-interpretation feat on his laptop after more than 45 minutes.
Skeptics make several points. Critics say the strongest claims lack competent baselines, that the Qwen case is bug-fixing rather than recursive self-improvement, and that the surrounding thread overclaims. Others argue that fitting weights directly against a benchmark is essentially meaningless, citing a Mario example from a model called nesformer. One thread asks whether models mostly read the first and last layers, which correspond to known inputs and outputs. Another view holds that the real lesson is how effectively coding agents wield mechanistic-interpretability tools. Supporters describe the work as meta-learning finally working and weight optimization as structured-text optimization, and propose hybrids that interleave backpropagation with direct edits. The practical reading is that direct weight editing currently looks most useful as a targeted repair technique, not a replacement for training pipelines.
On model releases, Tencent reports 75.2 on APEX-Agents for Hy4, against 72.1 and 72.9 for DeepSeek V4.1 Flash-1017, though the two trade wins elsewhere. Hy4's API is priced at $0.834 per million input tokens and $2.501 per million output tokens, and Tencent plans to release weights. On Agent Arena it ranks 14th overall across more than 8,500 sessions at $0.55 per task, versus $1.46 for Kimi K3.1 Pro (Max), with the two scores within a point. Sarvam's Prism, a "System One" decision model built on Sarvam 105B, scores 71.8 with a 44 ms median response time, which Sarvam calls the best average among sub-100 ms decision models.
Open-weight and open-model news includes Kimi K3.1 Pro weights arriving Wednesday via Together; Pruna's P-H3-Image, distilled from MiniMax-H3, running in 8 steps instead of 50, at 7 times smaller and producing a 4 MP image in 2.6 seconds; and LongCat-2.5 with 1.6T total and roughly 48B active parameters priced at $0.30/$1.20, which built a browser game for $0.28 in one user's test against $0.82 for Qwen3.8-Max. EmbeddingGemma 2's paper is out, Krea 3 has a technical report from a team under ten people, and Crouton Alpha is a free stealth reasoning model on OpenRouter with 1M context. Raschka's catalog of ten open LLM architectures from October covers attention variants like Mistral Large 4 (1T), which pairs MLA with sparse attention; Meta's Muse Spark Open (816B) with LatentMoE; Nemotron 3.5 Super VL mixing Mamba-2 layers with eight attention layers; and the looped transformer Dory1.0.
Artificial Analysis updated its Capability Indexes to v2.0, adding weight on non-hallucination and agentic tool use, with Claude Opus 5.5 (max) leading all six indexes and Kimi K3.1 Pro leading among open models. Regrading the Intelligence Index with the CritPt v2 server moved no frontier ranking by more than a point. On AA-Robotics, Opus 5.5 finished a robot-arm task and then kept going for 22 more minutes, while only GPT-6.1 Sol and Gemini 4 Argon stopped within a minute. Vals found that GPT-6 Astra retained 88% of its score at 18% of the cost on its lowest reasoning setting, Grok 4.7 kept 86% at 16%, and Gemini 4 Argon barely saved money. Fireworks' Nexus Auto (Sol) became the first router to reach 60% on the Router Index, at $1.25 per test.
Anthropic open-sourced FlashProve, a provable-inference system that verifies a response came from a specific model without revealing weights, claiming substantially better efficiency than prior open-source methods. Stated uses include hijack resistance, letting evaluators confirm the served model matches the tested one, and tamper detection and provenance; a write-up cites Attestable proving Gemma 4 31B at 53-77 tokens per second on a single H100. Benchmark audits were active: Prime Intellect found and fixed issues in public evals; Xeophon's audit found AutomationBench forcing prose request bodies, OfficeQA Pro v2 regex graders rejecting valid answers, and models retrieving answer keys from Hugging Face via web access, with both benchmarks nearing saturation after fixes. Removing images from SWE-bench Multimodal barely changes scores. SemiAnalysis's GDPx spans 14 data partners, 11 domains and 34 chip-design tasks, and GPT-6 Astra scored 0% on its cyber section by refusing every task. A METR/Redwood exercise found compaction left about 10% of eligible internal Codex traffic unmonitored.
On training research, quantization error in the BF16 LM head is reported to drive delayed "time-bomb" loss spikes after billions of tokens, with Z-loss, mu-centering and LM-head SGD-M working by keeping that error bounded. Giffmana argues a "quadrillion-param" extrapolation from 30M-400M-parameter studies will not hold beyond about one order of magnitude. Google reports that agent swarms fixed 142 or more silent TPU-compiler miscompilations, with 103 more under review, and that agent patches needed adversarial pre-filtering. Rewriting skill metadata to explain how each skill differs from its nearest neighbor lifted full-skill routing from 36.4% to 76.3% at 731 skills. In Amp, renaming an argument from format to kind raised correct choices from 53% to 79%.
On infrastructure, vLLM v0.33.0 ships 927 commits on PyTorch 2.14 with CUDA 13-only wheels and delivers 1.5-2x decode gains on GB300. ThunderAgent, merged into llm-d, delivers 1.5-2x throughput and cuts agent-trace median time to first token from about 50 seconds to 2-4 seconds. OCRia-1 rose from 285 to 2,800 tokens per second on one B200 through speculative decoding, quantization and 17 smaller optimizations. A DeepSpeed ZeRO-1/2 correctness fix requires upgrading to at least 0.19.9. Baseten launched Grid, VM-isolated Sandboxes and Loops for serverless SFT and RL, and Snowflake added Cortex Training for serverless SFT and RL on open-weight models.
On tooling, Claude Code's October updates brought mods, multi-session desktop improvements, usage insights, monthly API credits for Max, Ultra and Team plans, and MCP mid-task questions, plus plugin auto-connect through MCP servers that trust Claude as an issuer. Any app can expose content and actions to Siri AI via App Intents. Revolut made open models its coding default for roughly 2,000 engineers, citing a tenth of the cost and 70% faster lead time. In industry news, fal raised $1.65B across Series E and F, Traversal raised $120M at a $1B valuation, and Google said Gemini, Notebook and Search will be off by default in schools with per-service opt-in.
Why it matters: Teams building agents and evals should treat direct weight editing as an emerging repair and interpretability technique whose headline numbers are under active dispute, and should re-check any benchmark they rely on given the audit findings on answer leakage and grader bugs. Inference teams get concrete upgrade reasons in vLLM's decode gains and ThunderAgent's latency reduction, while FlashProve offers a route to proving which model actually served a request. The larger signal is that eval integrity and inference-level trust are becoming first-class engineering concerns rather than afterthoughts.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
November 3, 2026 — A quiet day lets us share some behind the senes.