The text discusses the importance of public agreements to prevent defection in the development of advanced superintelligence (ASI), suggesting that such agreements could mitigate risks by making treachery unacceptable and altering the incentives for actors involved. It highlights the instability of current AI development paths and the potential role of future ASI systems in enforcing cooperative behavior.
The chart will appear after repeat observations. The current metric comes from the source.
8 source pointsReal observations only. History before source connection is not reconstructed.
This could inform strategies for cooperative AI development and risk management by establishing clear norms against defection.
The article critically examines Lean4, a formal verification tool, highlighting its current limitations and potential vulnerabilities. It discusses concerns about the reliability of Lean4's proofs and the possibility of undetected bugs, including references to specific issues like 'soundness bug #14576' and the creator's warnings about AI exploiting these flaws.
4 days agoLessWrongThe author expresses frustration with calls for people to 'drop everything and pivot to AI safety,' noting that there are many talented individuals in the field but few job opportunities. They highlight that fellowships and job openings in AI safety are highly competitive and offer high salaries.
4 days agoLessWrongThe article discusses the distinction between AGI-pilled and ASI-pilled individuals, as well as challenges related to the potential superiority of artificial intelligence over humans. The author expresses concern about the future of freedom and democracy in the context of rapid technological development.
4 days agoLessWrongThe current alignment strategies of major AI labs include shaping the model's character, applying extensive RL pressure, racing towards prosaic RSI, and evaluating and fixing alignment through trial-and-error. These approaches are considered problematic due to the rapid growth of capabilities and the instability of alignment methods.
17 hours agoLessWrongThe post discusses the importance of clear thinking in empirical AI safety research, emphasizing fast iteration through efficient problem-solving and the use of scientific frameworks. The author shares insights from their apprenticeships at MATS and Redwood Research, focusing on finetuning-based misalignment auditing and sabotage capability evaluations.
13 hours agoLessWrongThis document provides an overview of monitoring practices for internally deployed agents at frontier AI companies, focusing on OpenAI (OAI), Anthropic, and Google DeepMind (GDM). It highlights that most monitoring is done asynchronously, with some real-time automated classifiers used for agent actions. However, there is limited public information about GDM's specific practices, as much of the data comes from their control roadmap, which outlines suggestions rather than confirmed implementations.
6 days ago