The paper explores how deep learning models efficiently store more information than their dimensionality allows, a phenomenon called superposition. It builds on prior research that studied this in simple neural networks, focusing on reconstructing inputs from a narrow bottleneck layer using mean squared error as a metric.
The chart will appear after repeat observations. The current metric comes from the source.
7 source pointsReal observations only. History before source connection is not reconstructed.
This research contributes to understanding how AI models compress and represent information efficiently, which could inform future model design and interpretation.
The paper explores the linearity in Large Language Models (LLMs) by showing that combining inputs from different text streams leads to a superposition of next-token distributions. It suggests that this linearity is an inherent property of the Transformer architecture and can be restored through fine-tuning, allowing for generating two coherent continuations from one forward pass.
4 days agoLessWrongThe post explores the author's confusion about how neural networks, particularly large language models (LLMs), perform and learn computations. It challenges the conventional 'mechinterp' (mechanistic interpretation) approach that relies on circuit-based explanations, arguing that 'circuits' and 'computation' may not be the most suitable framework for understanding LLMs. The essay discusses representational drift as a key obstacle to weight-based circuit analysis and suggests a 'co-selectionist' view of circuits as emergent units. The author also reflects on how the concept of 'universality' should influence explanations of LLM function.
6 days agoDaily PapersThe paper introduces Taste-Bench, a benchmark for evaluating an agent's 'taste' in making long-horizon decisions. It measures the ability to choose the better path in decision forks without knowing future outcomes, revealing that current models perform poorly and that taste can be improved through distillation.
6 days agoDaily PapersThis paper introduces FuseReg, a method that replaces heuristic layer fusion in representation autoencoders (RAEs) with training over random subsets of encoder layers. The approach reduces the reconstruction-generation gap by improving robustness to layer fusion choices, achieving higher PSNR and lower generation FID scores without modifying the pretrained encoder.
3 days agoDaily PapersThe study explores JEV-as-a-Judge, a cost-effective method for evaluating language models by making decisions based on confidence levels, showing comparable performance to state-of-the-art judges at a fraction of the cost. It highlights that while JEV performs well on standard tasks, it struggles with complex judgments requiring deeper analysis, but a cascade system that escalates uncertain cases maintains most of the accuracy at lower cost.
6 days agoDaily PapersThis paper introduces StableVQ, a method for improving the training stability of vector-quantized tokenizers by addressing the entanglement between encoder-decoder and codebook training. It proposes three key components: Dynamic STE for encoder stability, Region VQ Loss for codebook learning, and Decoupled Schedule for separate optimization dynamics. Experiments on ImageNet show improved training stability, codebook utilization, and reconstruction quality.
6 days ago