AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Import AI 476: Creativity and RSI; worse-than-China open weight models; AI goes to space

Collected Oct 12, 2026

Elorian and Scale AI have released Humanity's Sixth Sense (HSS), a benchmark measuring how a model reasons about information a visual scene implies but does not explicitly show. HSS contains 522 questions covering 288 images and 234 short video clips totaling 17.6 hours of video, organized into four categories: temporal and causal dynamics, physical and spatial logic, social understanding, and abstract and contextual inference. Sample tasks include looking at a bookcase and judging whether more books fit, or viewing horses pulling a plough and asking whether the chains bear the plough's drag weight.

The human baseline is not perfect but is very high, with people scoring aggregate pass@1 of 93.1 (image 94.1, video 91.9, time 97, space 94.9). The best-scoring AI system, GPT 6 Astra, reaches 53.6 aggregate (image 55.6, video 51.1, time 58.2, space 53.6). Chinese open weight models trail far behind on this test, with GLM-5.3-flash at 26.8 pass@1 and Kimi K3 at 25.5. The benchmark authors report that the two most common single mistakes were missing a detail that decides the answer, at 21% of failures, and misidentifying an object, person, or role, at 20%.

My reading is that HSS cuts two ways. It documents deficiencies, but it also shows how far general-purpose multimodal LLMs have come on tasks many researchers assumed would need spatially aware 2D and 3D world models. That frontier LLMs do as well as they do is some evidence against the claim that intuitive visual reasoning requires a separate class of model, though the large gap to the human baseline leaves plenty of room for architectures that build explicit spatial representations.

Separately, P-Zero Research has released TasteVal, a benchmark for what it calls research taste: the ability to pick interesting problems, design experiments, and interpret results. TasteVal covers eight tasks representative of frontier AI development, spanning pretraining data curation, pretraining and fine-tuning language models, preference modeling, and robustness to adversarial prompts. It isolates the experimental component, giving a model a fixed research problem and measuring how well it iteratively designs experiments and draws conclusions from outcomes. Results are checked against a baseline drawn from 24 experts, at least two per task, who have recently worked at organizations including OpenAI, Google DeepMind, NVIDIA and Microsoft Research.

P-Zero reports that the experimental research taste of frontier models has doubled every 3.0 months since December 2025, with a 95% confidence interval of 1.7 to 5.0 months. The best model, Opus 5.5, significantly exceeds the human baseline with a compute multiplier of 2.30x (95% CI 1.15-4.37x), at roughly 1/30 of the expert baseliners' average cost per attempt. The growth slope changed around the end of 2025 and beginning of 2026, after slower movement in prior years. The framing matters because research taste is a candidate bottleneck for recursive self-improvement, the idea that AI systems could accelerate their own development. Import AI's author now puts full RSI as more likely in 2027, moving up from an earlier 2028 expectation.

Reflection has revealed Beam, a 501 billion parameter mixture-of-experts LLM with 23 billion active parameters, promoted on token efficiency. Benchmark scores put it behind Chinese frontier models: DeepSWE 1.1 at 44.4 versus 61 for GLM-5.3, GPQA Diamond at 90.5 versus 91.7, and MCP Atlas at 78.7 versus 84.2. Reflection says Beam achieves scores comparable to GLM-5.2 on advanced reasoning benchmarks while using 3-4x less inference compute. The model was pretrained on 6,144 GB300 NVL72 GPUs for four weeks, with an RL run using roughly 10.5K of the same GPUs for four more weeks, over 23.8 trillion tokens from web and proprietary licensed datasets. Reflection sourced one million coding, agentic and STEM RL environments and says the weights, technical report, model card and developer artifacts will follow later this month.

Mistral has announced Mistral Large 4, an open weight model it plans to release by the end of the month, tuned for cyber tasks with fewer refusals than proprietary Anthropic and OpenAI models. It was trained on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's European datacenters. Scores show it likely becoming the strongest Western open weight model on release while remaining behind Chinese ones: Artificial Analysis DeepSWE 1.1 at 62 versus 68 for Kimi K3 and 61 for GLM-5.3; Terminal-Bench 4.0 at 28.3 versus 41.9 for GLM-5.3; Finance Agent V2 at 54.7 versus 55.8. In internal preference testing, ML4 was preferred in CAD and STEM and performed on par or close to GLM-5.3 in finance and coding.

The US government has run two multi-supercomputer AI training projects. Sandia, LLNL and Los Alamos trained a LLaMA-based model across LLNL's El Capitan (AMD MI300A) and NVIDIA H100 machines at Sandia and Los Alamos, keeping access-restricted data at its originating site, using torchtitan and comparing the NVFlare, Flower and APPFL federated frameworks on LLaMA 3.x at 1B, 8B and 70B scales; the workflow is described as production-ready and deployed on larger classified machines. A second project federated fine-tuned LLaMA-2 7B on the SMolInstruct chemistry dataset across about 1,700 Intel, NVIDIA and AMD GPUs at Argonne (Aurora, Polaris), Oak Ridge (Frontier) and Berkeley (Perlmutter). Researchers came from Oak Ridge, Sandia, Argonne, Flower Labs, Arizona State University and the Ontario Brain Institute. The scale remains roughly 100x off the frontier.

Why it matters: HSS, TasteVal and the Beam and ML4 releases give developers concrete yardsticks for reasoning, research autonomy and inference cost, and the cost-per-attempt figures on TasteVal are the kind of number that reshapes how teams budget evaluation. The open weight results also show the Western frontier still trailing Chinese labs on several coding and agentic benchmarks, which affects anyone choosing a self-hostable model. For public-sector teams, the DOE work is a template for pooling distributed compute under data-residency constraints, though the roughly 100x scale gap is inference on my part that pooling alone will not close the frontier gap soon. The space essay is the wildcard here; Import AI's argument that latency will push solar-system activity toward autonomous machines is a forecast, not a result.

Read at Import AI

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Will we give robots the moon?