Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learning
This paper introduces two novel model-free algorithms, Q-EarlySettled-LowCost and FedQ-EarlySettled-LowCost, for single-agent and federated reinforcement learning that simultaneously achieve near-optimal regret, linear burn-in costs in state and action spaces, and logarithmic policy switching or communication costs, while also providing improved gap-dependent theoretical guarantees.