This work presents a new benchmark that aims to address the limitations of current methods for evaluating large language models (LLMs) in Brazilian Portuguese. We introduced Brasilidade, an assessment methodology comprising standardized tasks distributed across four different domains, and we are making available the tasks from its first domain, which address nuances of the language, grammar, and variations of Brazilian Portuguese. The results show that model size, by itself, does not determine performance; several medium-sized models outperformed larger models in the protocols evaluated.
This source has not updated recently. Its observation time is shown above.
This official source does not publish popularity metrics. The story is refreshed from its RSS feed.
Real observations only. History before source connection is not reconstructed.
Brasilidade provides a specialized benchmark for evaluating language models in Brazilian Portuguese, which can help improve their adaptation to local linguistic nuances.
The authors combine LLM forecasts with external signals using weights estimated from resolved outcomes. Models help on some sets, while the system largely defers to the external forecast on a market subset.
59 days agoDaily PapersThe paper formulates intervention timing as single-token classification and generates extra supervision with a video agent. The authors compare visually grounded supervision with training from textual narration.
63 days agoDaily PapersThe authors distill visual perception from a larger agent system into a compact model for egocentric video questions. They prune the multilingual embedding table to meet the parameter limit.
63 days agoDaily PapersTRACE ranks visual tokens by relevance, diversity and screen coverage without additional model training. It also contracts the state retained from older frames.
64 days ago