The CarrotParsnipBench evaluates multi-agent deception and detection using the simple social deduction game Carrot-Parsnip, where two players are Carrots and one is a Parsnip. A study with 10 models and over 5000 games explores capabilities in strategic deception and detection. Results suggest the game's potential as a basis for multi-agent benchmarking of large language models' abilities.
The chart will appear after repeat observations. The current metric comes from the source.
8 source pointsReal observations only. History before source connection is not reconstructed.
This approach could be useful for evaluating models' deception and detection capabilities in multi-agent systems, as the game is simple to implement and cost-effective.
Anthropic released documents related to a copyright litigation case. The documents outline the company's research plans from 2021, including the development of a safe general AI assistant capable of programming tasks. The documents do not represent the company's official stance.
2 days agoLessWrongAs part of the MATS 9.1 extension program, mentored by Richard Ngo, this work explores applying the concept of cross-temporal incoherence to multistage games with observable actions. New equilibria defined through variants of the One Shot Deviation Principle (OSDP) are proposed, allowing analysis of situations where players reach inadequate equilibria with their future or counterfactual selves despite time-consistent discounting. The potential contributions of these variants to a theory of credible threats are also discussed.
yesterdayLessWrongThe article discusses challenges in ensuring the individual rationality of safe Pareto improvements (SPIs) in agent interactions. It explores mechanisms that might lead agents to prefer opting out of SPIs, including effects on other participants' strategies and potential consequences of not participating in SPIs.
3 days agoLessWrongThe author describes how their work with AI led to a shift in role from developer to manager, despite no explicit decision. They note the process was gradual and reminiscent of the Ship of Theseus paradox.
5 days agoHugging Face BlogFast private agents operating on-device using the LFM2.5-24B-A2B model are presented.
3 days agoHacker NewsLFM2.5-24B-A2B is a model designed for fast on-device operation.
2 days ago