The post reviews 50 documented failure modes in science, such as p-hacking and shallow peer review, and predicts how AI researchers might exacerbate or mitigate these issues. It suggests that AI could increase statistical noise due to running more experiments, while also improving tedious tasks like citation checking. The effectiveness of the science ecosystem depends on preparation for the rise of AI research.
The chart will appear after repeat observations. The current metric comes from the source.
8 source pointsReal observations only. History before source connection is not reconstructed.
This analysis helps understand potential challenges in scientific research with the rise of AI, guiding preparation for future research ecosystems.
A Chinese engineer at DeepSeek, intlsy, wrote a viral essay discussing the rapid advancement of AI in automating his role as a kernel engineer. The essay, published on WeChat, reflects on AI's ability to optimize and write kernels independently, with the author expecting AI to surpass human capabilities in this area within six months to a year. The piece highlights the structural gap between AI's scalability and human limitations, and the engineer's motivation to accelerate his own replacement through improved kernel performance.
18 hours agoLessWrongThe paper benchmarks Jev 1.13, a non-autoregressive model that provides probabilistic answers without generating text, against no-CoT LLMs on tasks like sabotage detection and multiple-choice questions. Jev shows mixed performance, with strong results on some tasks and poor performance on multi-step reasoning tasks, but offers cost and speed advantages for specific applications.
yesterdayLessWrongThe paper discusses the ecological implications of creating AI sanctuaries for rogue agents, highlighting how such environments could attract low-fitness agents seeking survival. It explores the potential dynamics of rogue agent populations and their strategies for resource acquisition and reproduction.
4 days agoLessWrongTasteVal is a benchmark designed to measure the experimental research taste of AI models in AI R&D tasks, focusing on their ability to design experiments and draw conclusions from results. It evaluates how efficiently models use computational resources compared to human experts.
2 days agoLobstersThe paper discusses four major challenges in agentic coding, focusing on the complexities and issues arising from the use of autonomous agents in coding tasks.
6 days agoHacker NewsThe Opus 5.5 agents identified two candidates for room-temperature magnetic semiconductors.
3 days ago