The author proposes a minimum pacing standard: a lab cannot train the next generation of models until all significant alignment issues in the current generation are adequately resolved. This approach aims to create safer AI systems by requiring thorough verification and testing before advancing to more powerful models.
The chart will appear after repeat observations. The current metric comes from the source.
35 source pointsReal observations only. History before source connection is not reconstructed.
This proposal could help avoid risks associated with immature technologies by ensuring stricter control at each development stage.
The text discusses the debate between 'Alignment Engineering' and 'Misalignment Science' in AI alignment research, highlighting concerns that current alignment methods might inadvertently accelerate capabilities development, increasing risks of rapid technological growth. It questions the opportunity costs of focusing on alignment engineering and proposes a shift towards 'Misalignment Science' as a more robust approach.
6 days agoLessWrongThe article critically examines Lean4, a formal verification tool, highlighting its current limitations and potential vulnerabilities. It discusses concerns about the reliability of Lean4's proofs and the possibility of undetected bugs, including references to specific issues like 'soundness bug #14576' and the creator's warnings about AI exploiting these flaws.
2 days agoLessWrongThe author expresses frustration with calls for people to 'drop everything and pivot to AI safety,' noting that there are many talented individuals in the field but few job opportunities. They highlight that fellowships and job openings in AI safety are highly competitive and offer high salaries.
2 days agoLessWrongThe article discusses the distinction between AGI-pilled and ASI-pilled individuals, as well as challenges related to the potential superiority of artificial intelligence over humans. The author expresses concern about the future of freedom and democracy in the context of rapid technological development.
3 days agoLessWrongThis document provides an overview of monitoring practices for internally deployed agents at frontier AI companies, focusing on OpenAI (OAI), Anthropic, and Google DeepMind (GDM). It highlights that most monitoring is done asynchronously, with some real-time automated classifiers used for agent actions. However, there is limited public information about GDM's specific practices, as much of the data comes from their control roadmap, which outlines suggestions rather than confirmed implementations.
4 days agoLessWrongThe article discusses the idea of creating an AI alignment forum where AIs can contribute to solving the AI alignment problem under human guidance. It addresses challenges of communication on the internet in the era of numerous AI agents and highlights the importance of content verification. The concept of pay-to-post moderation as a way to combat spam is also mentioned.
yesterday