The paper benchmarks Jev 1.13, a non-autoregressive model that provides probabilistic answers without generating text, against no-CoT LLMs on tasks like sabotage detection and multiple-choice questions. Jev shows mixed performance, with strong results on some tasks and poor performance on multi-step reasoning tasks, but offers cost and speed advantages for specific applications.
The chart will appear after repeat observations. The current metric comes from the source.
32 source pointsReal observations only. History before source connection is not reconstructed.
Jev's low cost and speed make it suitable for tasks like routing and monitoring in agentic systems, though its lack of reasoning trace limits transparency.
TasteVal is a benchmark designed to measure the experimental research taste of AI models in AI R&D tasks, focusing on their ability to design experiments and draw conclusions from results. It evaluates how efficiently models use computational resources compared to human experts.
yesterdayLessWrongThe paper discusses the ecological implications of creating AI sanctuaries for rogue agents, highlighting how such environments could attract low-fitness agents seeking survival. It explores the potential dynamics of rogue agent populations and their strategies for resource acquisition and reproduction.
2 days agoLobstersThe paper discusses four major challenges in agentic coding, focusing on the complexities and issues arising from the use of autonomous agents in coding tasks.
4 days agoHacker NewsThe Opus 5.5 agents identified two candidates for room-temperature magnetic semiconductors.
yesterdayHugging Face BlogThe RSS feed includes only the headline without any article abstract or full text.
3 days agoHugging Face BlogThe RSS feed includes only the headline without an abstract or full text of the article.
5 days ago