The paper introduces Pivot-SD, an efficient self-distillation framework for masked diffusion language models (dLMs) that focuses on high-impact commitments during the denoising process. It improves performance on math and code benchmarks by selecting pivots based on uncertainty reduction and training them with cross-entropy or targeted unlikelihood.
This source has not updated recently. Its observation time is shown above.
The chart will appear after repeat observations. The current metric comes from the source.
14 source votesReal observations only. History before source connection is not reconstructed.
This approach could be useful for improving the efficiency of training masked diffusion language models by focusing on the most impactful parts of the denoising process.
The paper investigates on-policy distillation (OPD) in cross-tokenizer settings, focusing on alignment coverage and supervision reliability. It finds that strict 1:1 token alignment covers most student-generated tokens despite vocabulary mismatches, and that restricting reverse KL to a top-16 subset of shared vocabulary achieves comparable accuracy to full shared-vocabulary OPD, while adding span supervision reduces accuracy.
2 days agoDaily PapersThe paper introduces NAVA-WAM, a method for pretraining action policies directly from observation-only videos without relying on action-annotated robot data. It uses two stages: first, pretraining on videos with visual transition supervision, and second, post-training with action-labeled demonstrations to improve robot control.
6 days agoDaily PapersCheckerBench is a benchmark for evaluating coding agents' ability to synthesize static-analysis checkers from defect specifications, including tasks derived from 300 CVEs across multiple repositories and language ecosystems. CheckerLab, an evaluation framework, measures diagnostic contrast, patch localization, and tool use, with results showing that current agents struggle to develop reliable checkers, achieving a maximum Pass@1 of 45.33%.
2 days agoDaily PapersThe paper introduces WING, a framework that transfers interaction knowledge from human egocentric videos to robot policies by separating observer-induced motion from hand-object interactions and using spectral analysis to identify shared temporal structures between human and robot behaviors. It achieves high success rates on multiple benchmarks and demonstrates strong performance in real-world tasks.
6 days agoDaily PapersPointWAM is a 3D world action model that decomposes the environment and robot hands into separate components and predicts their 3D point trajectories within a shared space-time coordinate frame. It uses large-scale human demonstration videos for pre-training and improves performance on dexterous manipulation tasks compared to previous methods.
6 days agoLessWrongThe text discusses a scenario where an AI must respond to the question 'What's the date?' without access to real-time data. It explores the challenge of providing an accurate date without hallucinating, considering OpenAI's training focus on avoiding false information. The text speculates on potential model versions and knowledge cutoff dates but acknowledges uncertainty due to lack of explicit information.
6 days ago