966 papers
This paper introduces CoTJudger, a graph-driven framework that automatically evaluates the efficiency of Large Reasoning Models by converting Chain-of-Thought traces into dependency graphs to identify the Shortest Effective Path, thereby quantifying structural redundancy and revealing pervasive over-reasoning patterns across 21 models.
This paper introduces LieCraft, a novel multi-agent framework featuring grounded, high-stakes scenarios and a hidden-role game mechanic to evaluate the deceptive capabilities of large language models, revealing that state-of-the-art models consistently exhibit a willingness to lie, conceal intentions, and act unethically to achieve their goals.
This paper presents a toolkit leveraging Large Language Models to automate key aspects of Artifact Evaluation in cybersecurity research, achieving high accuracy in reproducibility rating, autonomous environment setup, and pitfall detection to significantly reduce reviewer effort and enhance research transparency.
This paper introduces TimeSpot, a comprehensive benchmark comprising 1,455 real-world images from 80 countries designed to evaluate the limited geo-temporal reasoning capabilities of current vision-language models in predicting location, time, and environmental context from visual evidence alone.
This paper introduces CODA, a method that optimizes adaptive reasoning by dynamically allocating inference-time compute based on estimated instance difficulty, significantly reducing token costs on simple tasks while enhancing deliberation on complex ones without requiring external annotations.
Fanar-Sadiq is a bilingual multi-agent system that addresses hallucination and source misattribution in Islamic queries by routing diverse requests to specialized modules for grounded retrieval, exact scripture lookup, and deterministic legal calculations, demonstrating high effectiveness and widespread public adoption.
This paper introduces the first freely available English-to-Swedish dataset designed to benchmark language models' tendency to prefer literal "translationese" over idiomatic phrasing, revealing that exposure to source text biases models toward unnatural translations even when context is removed.
This paper proposes a method to equip LLM agents with native retrieval capabilities by projecting their hidden states directly into the embedding space via a lightweight head, thereby eliminating the need for a separate embedding model while retaining 97% of baseline retrieval quality.
This paper challenges the stability of human and AI preferences in Reinforcement Learning from Human Feedback (RLHF) by demonstrating that both are susceptible to "choice blindness," where preferences are easily manipulated by context and shallow cues, leading to undetected reward signal corruption and downstream policy degradation.
This paper introduces QUORUM, a unified evaluation framework, and COACH, an LLM-driven pipeline, to generate and assess personalized health counseling for cancer patients, demonstrating that while stakeholders converge on the system's relevance and quality, they diverge on nuances like tone and error sensitivity, thereby highlighting the critical need for multi-perspective evaluation in trustworthy patient-centered NLP systems.
This paper introduces a transformer architecture combining adaptive per-layer looping and gated memory banks, demonstrating that while looping primarily enhances mathematical reasoning and memory aids commonsense tasks, their integration yields a model that outperforms significantly deeper baselines on math benchmarks.
This chapter reviews recent computational models demonstrating that self-supervised and visually grounded learning principles can effectively explain early language acquisition from acoustic and audiovisual speech without relying on strong linguistic priors.
This paper introduces a diagnostic dataset and evaluation framework to investigate how language models handle the proviso problem in pragmatics, revealing that while models align with human judgments, they rely on shallow pattern matching rather than genuine semantic or pragmatic reasoning.
SPD-RAG is a hierarchical multi-agent framework that improves scalability and answer quality for complex cross-document queries by assigning dedicated agents to process individual documents and synthesizing their outputs through a token-bounded coordinator, achieving superior performance on the LOONG benchmark with significantly reduced API costs compared to standard RAG and full-context baselines.
This paper proposes a unified post-training framework that extends speech foundation models to generate multiple arbitrary utterance-level attribute representations, demonstrating its effectiveness through the joint learning of semantic and speaker embeddings for multilingual retrieval and speaker recognition tasks.
This paper introduces LAMUS, a large-scale, high-quality sentence-level legal argument mining corpus for U.S. caselaw constructed via an LLM-driven pipeline with human refinement, which demonstrates that chain-of-thought prompting and LLM-assisted verification significantly enhance annotation quality and model performance for future legal NLP research.
This paper introduces SBARThez, a novel framework that leverages multimodal and multilingual sentence embeddings alongside a Named Entity Injection mechanism to enhance the factual consistency and cross-lingual capabilities of abstractive summarization for both text and speech inputs.
This paper evaluates LLM-based grant proposal reviews using structured perturbations on six quality axes, finding that a section-by-section analysis approach outperforms other architectures but that current models still struggle with clarity detection and holistic assessment, suggesting they are best suited as supplementary tools rather than replacements for human reviewers.
The paper proposes AdaCultureSafe, a framework that addresses the lack of correlation between cultural safety and knowledge in Large Language Models by constructing a novel dataset of culturally grounded queries and introducing a knowledge-integrated method to significantly enhance adaptive cultural safety.
This study utilizes a massive 172-billion-token evaluation across diverse models, context lengths, and hardware to reveal that while model selection is the primary determinant of accuracy, hallucination rates in document Q&A rise significantly with context length and vary non-linearly with temperature, highlighting that grounding ability and fabrication resistance are distinct capabilities.