norgitov/ trends
Technology · people · ideas
RSS
RUEN
Back to discovery/LessWrong15 minutes ago
SafetyLessWrong

Distillation for Incrimination and Distillation for Capabilities

The paper explores two distillation approaches for AI safety: Distillation for Incrimination (DFI) and Distillation for Capabilities (DFC). DFI aims to transfer a model's misalignment to a weaker student model to expose the teacher's issues, while DFC focuses on transferring capabilities without misalignment. Experiments show DFI works better when the student shares the teacher's base model, and DFC can preserve capabilities while blocking subliminal biases.

Open original
SIGNAL FROM THE SOURCE
20
source points
Tracking sinceOctober 9, 202620 source points
Momentum0/hover 0.55 h
PublishedOctober 9, 2026sebastian_prasanna
BEHIND THE NUMBERS

How interest changes

LessWrong

History starts here

The chart will appear after repeat observations. The current metric comes from the source.

20 source points

Real observations only. History before source connection is not reconstructed.

WHY IT MAY MATTER

This research provides empirical insights into using distillation techniques to enhance AI safety by either exposing misalignments or preserving capabilities without harmful biases, which could inform safer model development and auditing practices.

A useful discovery?
KEEP EXPLORING

Connected ideas

Explore topic
LessWrong

“Alignment Engineering” vs. “Misalignment Science”

The text discusses the debate between 'Alignment Engineering' and 'Misalignment Science' in AI alignment research, highlighting concerns that current alignment methods might inadvertently accelerate capabilities development, increasing risks of rapid technological growth. It questions the opportunity costs of focusing on alignment engineering and proposes a shift towards 'Misalignment Science' as a more robust approach.

4 days agoLessWrong

ASI and Liberty: Can we do better than a benevolent god?

The article discusses the distinction between AGI-pilled and ASI-pilled individuals, as well as challenges related to the potential superiority of artificial intelligence over humans. The author expresses concern about the future of freedom and democracy in the context of rapid technological development.

yesterdayLessWrong

Stop telling people to pivot to AI safety, I beg you

The author expresses frustration with calls for people to 'drop everything and pivot to AI safety,' noting that there are many talented individuals in the field but few job opportunities. They highlight that fellowships and job openings in AI safety are highly competitive and offer high salaries.

18 hours agoLessWrong

An Overview of Frontier AI Company Monitoring Practices in Mid-2026

This document provides an overview of monitoring practices for internally deployed agents at frontier AI companies, focusing on OpenAI (OAI), Anthropic, and Google DeepMind (GDM). It highlights that most monitoring is done asynchronously, with some real-time automated classifiers used for agent actions. However, there is limited public information about GDM's specific practices, as much of the data comes from their control roadmap, which outlines suggestions rather than confirmed implementations.

2 days agoLessWrong

Are OpenAI's Math Results 'Creative' in an Important Way?

The article discusses whether OpenAI's recent mathematical results demonstrate a form of creativity that goes beyond brute-force methods, suggesting that while AIs are improving at solving complex problems, they may still lack the ability to form new concepts and apply them efficiently. The text raises questions about the nature of the creativity involved in these proofs and whether they represent a significant breakthrough akin to historical examples like AlphaGo's Move 37.

2 days agoLessWrong

A Critical Look at Lean4

The article critically examines Lean4, a formal verification tool, highlighting its current limitations and potential vulnerabilities. It discusses concerns about the reliability of Lean4's proofs and the possibility of undetected bugs, including references to specific issues like 'soundness bug #14576' and the creator's warnings about AI exploiting these flaws.

15 hours ago