norgitov/ trends
Technology · people · ideas
RSS
RUEN
Back to discovery/LessWrong26 minutes ago
SafetyLessWrong

Latent Reasoning Architectures Could Undermine Chain of Thought Oversight

The paper discusses the potential risks of latent reasoning architectures, which could reduce the effectiveness of Chain of Thought (CoT) as a tool for understanding AI systems. It highlights that such architectures might allow AI models to reason extensively in latent states rather than through text-based CoT, making oversight more challenging. The paper also mentions examples like COCONUT and full-bandwidth transformers that could enable this shift.

Open original
SIGNAL FROM THE SOURCE
44
source points
Tracking sinceSeptember 23, 202644 source points
Momentum+6.88/hover 1.02 h
PublishedSeptember 23, 2026Lukas Finnveden
BEHIND THE NUMBERS

How interest changes

LessWrong

History starts here

The chart will appear after repeat observations. The current metric comes from the source.

44 source points

Real observations only. History before source connection is not reconstructed.

WHY IT MAY MATTER

This research highlights the importance of Chain of Thought (CoT) as a tool for understanding AI reasoning and warns about potential architectural changes that could make oversight more difficult by enabling extensive latent reasoning.

A useful discovery?
KEEP EXPLORING

Connected ideas

Explore topic
LessWrong

We've saved the world before: what the ozone hole teaches us about AI

The article draws a parallel between the historical success of the Montreal Protocol in addressing the ozone hole and the challenges of ensuring AI safety. It highlights the importance of global cooperation and proactive measures, suggesting that lessons from the past could inform future strategies for managing AI risks.

3 days agoLessWrong

AI as Orderly Evacuation vs Stampede

The article compares the development of AI to an orderly evacuation rather than a stampede, emphasizing the importance of careful management to avoid chaos. It argues that alignment challenges are akin to a jammed door, requiring time to resolve before a crisis occurs, and highlights the dangers of an 'arms race' analogy in fostering competitive and paranoid cultures among AI companies.

6 days agoLessWrong

For Love of the Lightcone, Don't Partisanize AI Safety

The author warns against politicizing AI safety, emphasizing that while it is inherently political, it should remain nonpartisan to avoid the risks associated with tribal reasoning. The post reflects concerns about the potential for AI safety to follow the same path as other issues that became partisan.

6 days agoLessWrong

The Game is Set for a Targeted Memetic Attack on the AI Safety Community

The author discusses potential adversarial memetic attacks targeting the AI safety community, suggesting that false information about dangerous AI incidents could be spread through unverified sources. The text advises caution, recommending that individuals pause, assess the credibility of information, and avoid acting impulsively on unconfirmed claims.

5 days agoLessWrong

Common Mistakes in AI Safety Group Organizing

The text outlines common errors in organizing AI safety groups, suggesting that reading groups often fail due to low participation and lack of shared knowledge. It recommends structuring meetings with food, proper cohort grouping, and physical materials to enhance engagement and discussion quality.

3 days agoLobsters

Flock Cameras Have Security Vulnerabilities and Hard-Coded Credentials

Flock cameras have been found to have security vulnerabilities and hard-coded credentials.

6 days ago