This work investigates how absolute positional embeddings (APE) and rotary positional embeddings (RoPE) encode spatial position in Vision Transformers (ViTs). APE store position in explicit features, while RoPE build it within attention and reconstruct it when withheld. Experiments show both methods affect classification: permuting APE's positional features reduces accuracy by 24-31 points, and ablating RoPE's rotations harms performance more than other heads.
This official source does not publish popularity metrics. The story is refreshed from its RSS feed.
Real observations only. History before source connection is not reconstructed.
This research helps understand how different positional encoding methods affect model behavior, which can be useful for optimizing and improving architectures.
The study presents AgentSWE, a benchmark evaluating coding agents' ability to build agents aligned with user requirements. The research found that existing agents fail to accurately fulfill tasks, as there is a significant gap between demonstrated and actual performance.
5 days agoDaily PapersThe paper introduces S-Space, a linear subspace in multimodal language models where object-token representations encode continuous spatial coordinates. It demonstrates that S-Space serves as an internal workspace for spatial reasoning, enabling accurate spatial judgments through perception, internal reasoning, and selective verbal access, while highlighting limitations in spatial transformations across viewpoints.
4 days agoDaily PapersThe paper introduces TerminalHorizon, an automated data synthesis engine for long-horizon terminal agents. It constructs 1.5K task environments and fine-tunes Qwen3.5-27B to create TerminalHorizon-27B, achieving performance gains across multiple benchmarks. The model demonstrates improved ability to experiment, refine methods, and retain effective improvements in final results.
5 days agoDaily PapersUrbex is an agentic vision-language framework that reformulates grounding as active spatial search. It represents a city-scale 3D scene as an interactive multi-view environment, enabling a VLM agent to query landmarks, zoom into local regions, and commit grounded bounding boxes. Experiments show Urbex achieves the best Hit@1 and IoU on CityRefer.
yesterdayDaily PapersThe study explores Adversarial Heuristic Learning (AHL), where AI agents refine game strategies with fixed model weights. The AAArena benchmark includes 12 games and 1,920 archived programs, evaluated through protocols mimicking real competitions. Results show agents can improve strategies but face challenges in complex games.
6 days agoDaily PapersMulti-Stream Attention Residuals (MSAR) extends the AttnRes method by generalizing the attention mechanism to multi-head depth, achieving significant performance improvements with minimal additional overhead. MSAR leverages multi-scale causal convolutions to draw stream diversity from sequence context, allowing different heads to retrieve historical representations through distinct contextual views.
5 days ago