1071 papers
This paper challenges the reliability of language model micro-benchmarks by demonstrating that they often fail to consistently rank models with small performance differences, frequently requiring as many as 250 examples to achieve accuracy comparable to random sampling, thereby offering actionable guidance on the trade-off between evaluation efficiency and reliability.
This paper argues that LLM internal states primarily reflect the retrieval of parametric knowledge rather than the truthfulness of outputs, demonstrating that hallucinations driven by spurious statistical associations are mechanistically indistinguishable from factual recall, thereby limiting the effectiveness of standard detection methods.
This paper introduces "Just-In-Time Objectives," a framework that passively observes user behavior to infer and rapidly optimize for specific, real-time goals, enabling large language models to generate specialized tools and responses that significantly outperform standard generic interactions.
This paper demonstrates that chain-of-thought reasoning significantly enhances large language models' ability to handle inter-sentential dependencies in English-French translation, with top-performing models like GPT-4 and Phi achieving high accuracy and showing a "wise get wiser" effect where reasoning benefits already strong models the most.
This study demonstrates that ChatGPT-based coding of communication data performs consistently across gender and racial/ethnic subgroups, matching human rater reliability and validating its potential for large-scale collaborative assessments.
This paper introduces the Collaborative Battleship task to evaluate language models' information-seeking abilities and proposes Bayesian Experimental Design-inspired Monte Carlo inference strategies that significantly enhance both question-asking and answer-accuracy, enabling weaker models to outperform humans and frontier models in strategic decision-making tasks.
This paper introduces DETECT, the first German-specific metric for evaluating automatic text simplification that is trained on synthetic LLM-generated data and validated on a large human-annotated dataset, demonstrating superior correlation with human judgments across simplicity, meaning preservation, and fluency compared to existing general-purpose metrics.
This paper introduces AfriMTEB, a comprehensive benchmark covering 59 African languages across 14 tasks and 38 datasets, alongside AfriE5, a state-of-the-art text embedding model adapted for these languages through cross-lingual contrastive distillation.
This paper proposes a novel hybrid layer selection framework that extracts Big Five personality traits from LLM hidden states via low-rank subspace discovery to enable stable, precise behavioral steering without compromising fluency or general capabilities.
This paper proposes "critical confabulation," a framework where LLMs are guided to generate evidence-bound, speculative narratives to fill archival gaps regarding marginalized historical figures, demonstrating that controlled hallucinations can support ethical knowledge production without sacrificing historical fidelity.
This paper presents Co-Layout, a novel framework that integrates large language models with grid-based integer programming and a coarse-to-fine optimization strategy to jointly optimize room layouts and furniture placement, significantly outperforming existing two-stage pipelines in both solution quality and computational efficiency.
The paper proposes SPINE, a token-selective test-time reinforcement learning framework that improves reasoning model performance by updating only high-entropy decision-critical tokens with entropy-band regularization, thereby preventing response collapse and enhancing stability without requiring external labels or reward models.
This paper identifies and systematically studies "Tools Orchestration Privacy Risk" (TOP-R), a novel vulnerability where autonomous agents inadvertently synthesize sensitive information from non-sensitive tool fragments, and addresses it by introducing the TOP-Bench benchmark, the H-Score metric, and effective mitigation strategies that significantly improve the safety-utility trade-off.
This paper introduces WBC (Window-Based Comparison), a novel membership inference attack that significantly outperforms existing global-averaging methods against fine-tuned Large Language Models by exploiting localized memorization signals through a sliding window approach with sign-based aggregation.
This paper introduces a framework for finetuning Large Language Models to generate age-appropriate, factually accurate educational content across six grade levels, which significantly improves grade-level alignment by 35.64 percentage points compared to standard prompting methods.
This paper introduces the PyMUSAS framework, which presents the largest evaluation of the USAS semantic tagging system across five languages and demonstrates how a hybrid approach combining rule-based methods with neural networks, trained on a newly created silver-standard dataset, significantly enhances multilingual semantic annotation performance.
This paper introduces Latent Exploration Decoding (LED), a training-free decoding strategy that leverages high-entropy intermediate layer posteriors to counteract exploration collapse in post-trained Large Reasoning Models, thereby significantly improving accuracy across multiple benchmarks.
The paper introduces COMI, a coarse-to-fine adaptive context compression framework that utilizes a novel Marginal Information Gain metric to jointly optimize semantic relevance and diversity, significantly outperforming existing baselines in long-context tasks under high compression rates.
This paper presents a collection of case studies demonstrating how researchers successfully collaborate with Google's Gemini models to solve open problems and generate new proofs in theoretical computer science and other fields, while extracting common techniques for effective human-AI partnership in scientific discovery.
This paper introduces Aletheia, an autonomous AI research agent powered by advanced reasoning models and tool use that successfully generates, verifies, and revises mathematical proofs from Olympiad problems to PhD-level research, achieving milestones such as fully AI-generated papers and the autonomous solution of open problems while proposing new frameworks for quantifying AI autonomy and transparency.