norgitov/ trends
Technology · people · ideas
RSS
RUEN
Back to discovery/arXiv1 hour ago
ResearcharXiv

Do Small Language Models Know What They Don't Know?

The study investigates if entropy-based confidence signals can enhance the accuracy of small language models (SLMs) with fewer than 3 billion parameters. It finds that token-level entropy is ineffective, but semantic entropy, which involves generating multiple samples and measuring distributional uncertainty, provides a viable confidence signal. Routing uncertain queries to larger expert models improves accuracy significantly, with cross-family routing showing greater benefits.

Open original
SIGNAL FROM THE SOURCE
21
September 2026
Tracking sinceSeptember 21, 2026
Momentum—More observations needed
Discussion—No comment count provided
PublishedSeptember 21, 2026Prashant Mudgal
BEHIND THE NUMBERS

How interest changes

arXiv

The publication is the signal

This official source does not publish popularity metrics. The story is refreshed from its RSS feed.

Real observations only. History before source connection is not reconstructed.

WHY IT MAY MATTER

This research provides insights into improving the reliability of small language models by using semantic entropy to identify uncertain predictions and route them to more capable models, enhancing overall accuracy without requiring extensive computational resources.

A useful discovery?
KEEP EXPLORING

Connected ideas

Explore topic
Daily Papers

Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model

This paper evaluates the ability of the MiniMax-H3 multimodal generative model to reason about the physical world. The study introduces a new framework for assessing physical reasoning through four dimensions, using tasks that require integrating information across multiple modalities. The model achieves an overall success rate of 41.97% across 517 evaluation instances, with video-based reasoning performing best and audio-based reasoning least effectively.

5 days agoDaily Papers

LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence

LimiX-2 is a new model in the LimiX family that uses the Contextual Mechanism Networks (CMNs) paradigm, pretrained with Context-Conditional Masked Modeling (CCMM). It focuses on learning joint representations of data generation processes rather than target-centric prediction, showing superior performance on tabular data benchmarks and promoting causal awareness.

6 days agoDaily Papers

Verifiable Social Reasoning for LLM Assistants

This paper introduces Fuse, a multi-agent simulation framework for evaluating social reasoning in LLM assistants. The framework uses a target agent with a hidden motive interacting with other agents, including a user agent, to create verifiable ground truth for social reasoning tasks. The study applies Fuse to 12 LLMs and identifies key factors affecting social reasoning performance.

6 days agoDaily Papers

Self-Evolving Search Index

The paper introduces SELF-INDEX, a framework that enables an index to self-evolve without human intervention by autonomously diagnosing retrieval issues, revising index keys, and validating changes. It also uses a Query Simulator to proactively explore new retrieval demands, improving performance across various corpora and downstream applications.

4 days agoDaily Papers

Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents

The paper introduces XConf, a method for estimating confidence in language models by leveraging the model's accumulated experience. XConf uses past episodes, including tasks, reflections, confidence levels, outcomes, and lessons learned, to inform current confidence estimates through a recall and reflect process. It outperforms existing methods in discrimination and calibration with lower computational cost.

6 days agoDaily Papers

ScienceIDE: Turning the World's Scientific Codebase into Agent Learnable Environments

The paper introduces ScienceIDE, a framework that transforms scientific code repositories into programmable environments for training scientific agents. These environments enable task generation, execution, and verification, supporting supervised fine-tuning and reinforcement learning. The approach demonstrates improvements in scientific code repair and general-purpose benchmarks.

5 days ago