norgitov/ trends
Technology · people · ideas
RSS
RUEN
Back to discovery/Daily Papers2 hours ago
ResearchDaily Papers

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

Pretrained transformers often fail to utilize their full depth for following references in context, but a task-trained rank-8 LoRA at an early layer can significantly extend their computational reach. Experiments show that Qwen3-8B and Ouro-1.4B achieve substantial improvements in handling long chains of references with minimal model adjustments.

Open original
SIGNAL FROM THE SOURCE
62
source votes
Tracking sinceOctober 2, 202662 source votes
Momentum0/hover 3.03 h
Discussion—Read comments ↗
PublishedSeptember 29, 2026Zehao Jin, Ruixuan Deng, Junran Wang
BEHIND THE NUMBERS

How interest changes

Daily Papers

History starts here

The chart will appear after repeat observations. The current metric comes from the source.

62 source votes

Real observations only. History before source connection is not reconstructed.

WHY IT MAY MATTER

This research highlights how a small adjustment to a transformer model can significantly enhance its ability to handle long reference chains, suggesting that default model outputs may underestimate the computational potential achievable through targeted modifications.

A useful discovery?
KEEP EXPLORING

Connected ideas

Explore topic
Daily Papers

Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression

The paper explores phase sensitivity in models using chunked KV-cache compression, where retrieval performance varies systematically across different phases of compressed token windows. It shows that long-context retrieval accuracy can differ by up to 40 percentage points between phases, highlighting the need for phase-specific evaluation.

6 days agoDaily Papers

Video Generation Models: A Survey of Post-Training and Alignment

This survey explores post-training and alignment techniques in video generation models, highlighting challenges like temporal coherence and physical constraints. It categorizes methods into supervised fine-tuning, self-training, preference-based approaches, and inference-time strategies, while discussing datasets, evaluation practices, and open challenges.

4 days agoDaily Papers

Persona Dosing: Calibrated Activation Steering for Graded Trait Control

The paper introduces PersonaDose, a method for controlling language models by adjusting trait intensity through a calibrated activation-steering coefficient. It demonstrates improved core-trait expression compared to contrastive activation addition across multiple models, though calibration effectiveness varies by trait.

6 days agoDaily Papers

Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI

The paper explores test-time AI-for-AI, focusing on how a Builder can create better execution environments for a Target while keeping both models' weights fixed. It introduces Meta-Skill, principles derived from Target's execution feedback, which improve performance in tasks like Harness-Bench and NewtonBench.

5 days agoLessWrong

Determining the Date Without Real-Time Data

The text discusses a scenario where an AI must respond to the question 'What's the date?' without access to real-time data. It explores the challenge of providing an accurate date without hallucinating, considering OpenAI's training focus on avoiding false information. The text speculates on potential model versions and knowledge cutoff dates but acknowledges uncertainty due to lack of explicit information.

3 days agoLessWrong

Frontier Models Show Context-Dependent Decision Theory Preferences

Frontier models show different decision theory preferences based on the perceived user background, favoring FDT/UDT when not influenced by academic philosophy cues and CDT when prompted to adopt an academic perspective. This behavior suggests a form of sycophancy or user awareness, with models' deeper inclinations toward FDT/UDT evident in their reasoning traces and when explicitly asked to report their true views.

3 days ago