2073 papers
The paper introduces Exclusive Self Attention (XSA), a modification that constrains attention to information orthogonal to a token's own value vector, thereby improving Transformer performance in language modeling tasks, particularly as sequence length increases.
This paper proposes learning lightweight, reinforcement-trained decoding adapters that dynamically select sampling strategies at both the sequence and token levels based on prompt features and compute budgets, significantly improving the accuracy-efficiency tradeoff on math and coding benchmarks compared to fixed hyperparameter baselines.
This paper introduces DynaME, a novel hybrid framework for online time series forecasting that redefines concept drift into recurring and emergent types, utilizing specialized historical experts for the former and a stable general expert for the latter to significantly outperform existing baselines.
SCALAR is a bidirectional framework that couples LLM-guided symbolic planning with deep RL to iteratively refine skill specifications through execution feedback, significantly outperforming prior methods in complex environments like Craftax by correcting initial planning errors and improving sample efficiency.
The paper introduces EPIC, a hardware- and physics-co-guided distributed scientific machine learning framework that significantly reduces communication latency and energy consumption while preserving physical fidelity by performing lightweight local encoding and physics-aware decoding with cross-attention for tasks like full-waveform inversion.
This paper introduces CALIPER, a data-only, detector-agnostic test that determines the sufficient post-drift data size for stable model retraining by analyzing the trend of a one-step proxy error against a locality parameter, thereby bridging the gap between drift detection and effective adaptation in streaming learning.
This paper proposes an exact, parameterization-aware flatness measure tailored to the geometric structure of convolutional neural networks with global average pooling, demonstrating its effectiveness as a robust proxy for estimating and comparing generalization performance across various CNN architectures.
This paper introduces Normalized Flow Matching (NFM), a novel method that distills quasi-deterministic couplings from pretrained auto-regressive normalizing flow models to train student flow models, achieving superior performance over both traditional flow matching approaches and the teacher models themselves.
MAPLE introduces a unified training paradigm that enhances medical large language models by integrating Test-Time Reinforcement Learning with expert-aligned Med-RPMs to replace unreliable majority voting with fine-grained process rewards, thereby significantly improving clinical reasoning accuracy and reliability across multiple benchmarks.
This paper proposes MAcPNN, a decentralized Mutual Assisted Learning paradigm inspired by Vygotsky's Sociocultural Theory that enables autonomous IoT devices to collaboratively address concept drifts and temporal dependence in data streams using Continuous Progressive Neural Networks while minimizing communication overhead compared to traditional Federated Learning.
This paper introduces the inequality to demonstrate that Mixture-of-Experts (MoE) models suffer from a structural "double penalty" of routing fragmentation and memory constraints during inference, often rendering them significantly less efficient than quality-matched dense models for long-context serving despite their training-time FLOP advantages.
This paper introduces a comprehensive multi-vector privacy evaluation framework that quantifies memorization risks in Genomic Language Models by integrating perplexity-based detection, canary sequence extraction, and membership inference, revealing that these models exhibit measurable data leakage dependent on architecture and training dynamics.
This paper theoretically and empirically demonstrates that hybrid sequence models, which combine Transformers and state-space models, can provably solve core synthetic tasks with significantly fewer parameters and less memory than non-hybrid models while also achieving superior length generalization and out-of-distribution robustness.
This paper introduces SoftJAX and SoftTorch, open-source libraries that provide feature-complete, drop-in soft relaxations for hard, non-differentiable primitives in JAX and PyTorch, thereby enabling informative gradients for optimization tasks involving operations like thresholding, sorting, and Boolean logic.
This paper introduces the Temporal Markov Transition Field (TMTF), a novel time series representation that overcomes the limitations of the global Markov Transition Field by partitioning data into temporal chunks to preserve regime-specific dynamics, thereby creating a structured image suitable for convolutional neural networks.
The paper introduces SPREAD, a geometry-preserving framework for lifelong imitation learning that utilizes singular value decomposition to align policy representations within low-rank subspaces and a confidence-guided distillation strategy to mitigate catastrophic forgetting while achieving state-of-the-art performance on the LIBERO benchmark.
This paper proposes the Online-Within-Online Fair Multi-Task Learning (OWO-FMTL) framework, which leverages a dual-loop mechanism with primal-dual updates to ensure long-term equitable inference performance for heterogeneous users in AI-RANs while maintaining low computational overhead.
This paper introduces \textsc{Gome}, a gradient-based MLE agent that outperforms traditional tree search methods on MLE-Bench by mapping diagnostic reasoning to gradient computation, demonstrating that as LLM reasoning capabilities improve, gradient-based optimization becomes increasingly superior to exhaustive enumeration.
The paper introduces Coupled Discrete Diffusion (CoDD), a hybrid framework that overcomes the "factorization barrier" in diffusion language models by replacing fully factorized outputs with a lightweight probabilistic inference layer, thereby enabling efficient parallel generation of coherent, high-quality text without the prohibitive costs of full joint modeling or reinforcement learning.
This paper proposes a curriculum-based continual learning framework that decomposes complex robust control problems with multiple uncertainties into sequential tasks, combining a model-based controller with deep reinforcement learning to achieve efficient, non-forgetting policy updates and successful sim-to-real transfer for automotive powertrain vibration control.