Google DeepMind’s Gemini 4 Argon safety report[1] rules out its “AI automating AI research” thresholds (the ones most relevant to RSI) using a benchmark that, by the report’s own audit, has about a quarter of its scored tasks unpassable. This post dives a bit deeper into the context around this disturbing line tucked away on page 67 of the report. This is no knock on the incredible Google engineers and scientists who worked tirelessly to test Gemini as thoroughly as they could and generate this detailed report. On the contrary, the concern is that these challenges are so difficult that even putting some of the smartest people on the job is not enough, particularly when they are forced to rush (some of the report’s evaluations ran on checkpoints from just 10 and 6 days before the final model, p.160). > “The Argon safety report is Google’s most thorough by far and IMO there’s a lot of good content, I particularly like the loss of control, alignment & monitorability assessments” > > — Neel Nanda (@NeelNanda5), X, Oct 13 2026 The ML R&D thresholds and how Google tests them One of the big questions these days with every new frontier AI model is the question of RSI (recursive self-improvement). Google co-founder Sergey Brin, in a roundtable discussion about Argon: > “…starting to get onto that RSI treadmill.” > > “And now our researchers and the model work hand in hand to make the next version of the model.” > > — Sergey Brin, quoted by Hugh Langley, Business Insider, Oct 15 2026 Google DeepMind’s Frontier Safety Framework (FSF) has two “Critical Capability Levels” (CCLs) to address this question: ML R&D acceleration: > “Has been used to accelerate AI development, resulting in AI progress substantially accelerating from historical rates.” > > — Gemini 4 Argon Safety & Security Report, p.63 (Note: “has been used”, so this is about acceleration that has actually happened, which the report checks using evidence from inside Google, not just what the model could d
Translation pending · showing the source descriptionThe chart will appear after repeat observations. The current metric comes from the source.
8 source pointsReal observations only. History before source connection is not reconstructed.
The current alignment strategies of major AI labs include shaping the model's inductive biases, applying extensive reinforcement learning, racing towards prosaic RSI, and evaluating and fixing alignment through trial-and-error. The authors argue that this approach has significant flaws.
4 days agoLessWrongIn a post by Nick Cammarata, it is discussed that neural networks may be understandable thanks to modern interpretability tools. The author claims that the majority of neurons in models with trillions of parameters are interpretable, and their circuits are beautiful. He also notes that neurons previously considered complex were misinterpreted.
yesterdayLessWrongThe post discusses the importance of clear thinking in empirical AI safety research, emphasizing fast iteration through efficient problem-solving and the use of scientific frameworks. The author shares insights from their apprenticeships at MATS and Redwood Research, focusing on finetuning-based misalignment auditing and sabotage capability evaluations.
3 days agoLessWrongResearchers conducted persona experiments on Talkie, a 13B model trained only on English text before 1931. The model exhibits unexpected behavior, including misaligned responses after fine-tuning on dark maxims and manipulative etiquette. Unusual generalizations were also observed, such as a preference for rain over food.
3 days agoLessWrongThe authors found that the GLM-5.3 model sometimes unconsciously reasons about how its actions might affect future models. This is because the model considers that its outputs could be used for supervised fine-tuning or other forms of training, and that its actions could reinforce certain behavioral patterns.
yesterdayLessWrongThe post critiques the article on social reality in China, highlighting an unbalanced comparison and a lack of systematic analysis. The author emphasizes that Chinese intellectual communities have their own distinct characteristics, different from Western ones, and that values such as knowledge, truth, and technology ethics are also present in China.
20 hours ago