The study investigates how layer order affects computations in transformers. It shows that layer order creates a measurable process influencing task outcomes like lexical tasks and can be used to control internal trajectories.
This official source does not publish popularity metrics. The story is refreshed from its RSS feed.
Real observations only. History before source connection is not reconstructed.
The research can help in understanding the inner workings of transformers and developing methods to control their behavior.
MemT-1.0 is a new architecture of memory foundation models that internalizes memory as part of model computation. It uses Gated DeltaNet-2 with a stable token-level memory update mechanism and Mixture-of-Storage to scale native memory. MemT-1.0-Nano and MemT-1.0-Flash models outperformed parametric memory baselines and were competitive with retrieval-augmented methods.
2 days agoDaily PapersWaypoint-OPD uses an intermediate teacher checkpoint to compare early and late shifts in action preferences during policy transfer. It evaluates the consistency between early and late shifts and suppresses the endpoint shift when they conflict.
6 days agoDaily PapersThis paper investigates the extent to which sparse autoencoder (SAE) representations support behavioral compositionality. The study introduces SAE Additivity Residuals (SAR) to measure unexplained behavioral variation above a chosen interaction order. Experiments show that learned SAE coordinates exhibit lower first-order residuals compared to random rotations of the same subspace.
6 days agoDaily PapersWe study how vision-language models (VLMs) can learn to think and actively explore in 3D space. Instead of providing multi-view images, videos, or 3D representations to the model, we preserve the native visual encoding of VLMs, i.e., a single image, one at a time, just like humans, and treat the 3D representation only as an environment with which the model can interact through actions (e.g., move left) and observations (rendered images).
6 days agoDaily PapersThis work introduces Scene-Designer, an agentic framework that learns design intelligence through complementary design and critique experiences. Two datasets, Scene-OTA-35K and Scene-OTC-15K, were created for scene design and quality critique. A Scene-GRPO methodology was proposed to improve design decisions and reduce unnecessary corrections. Experiments demonstrate superior performance in physical validity, functional correctness, and layout coherence.
5 days agoDaily PapersReliable multi-view forecasting requires videos from different cameras to describe the same evolving 3D scene. To this end, we introduce Video-Point Model (VPM), a world model that conditions on multi-view video histories and a language instruction to jointly forecast future videos from arbitrary viewpoints and the 3D trajectories of persistent scene points. We extend a pretrained world model with camera-conditioned multi-view processing that integrates complementary observations across cameras, and add a trajectory head that predicts how physical points move through time. Each trajectory tracks a single point across all views, and supervising these trajectories alongside video generation therefore anchors appearance changes in every camera to a shared description of 3D motion. To support training and evaluation, we build VPM-Data, a simulation corpus pairing synchronized multi-view videos with persistent point trajectories, and VP-WorldBench, a forecasting benchmark spanning simulated and real environments. We further extend VPM to robot control by adding robot actions as a token modality in the generator. Action tokens share the point-supervised backbone and are denoised jointly with future videos, thereby grounding the policy in the same 3D-aware representation. On VP-WorldBench and existing 4D generation benchmarks, point supervision improves both video fidelity and 3D motion accuracy over a video-only counterpart, and in simulated and real-robot experiments, it also raises the manipulation success of the action-enabled model.
yesterday