H. Dilpriya's Momentum (HDM) : A Multi-Strategy Gradient-Aligned Optimizer with Adaptive Per-Parameter Corrections, Cosine-Annealed Scheduling, and Convergence Guarantees for Deep Neural Networks
This paper introduces H. Dilpriya's Momentum (HDM), a novel multi-strategy optimizer that combines adaptive per-parameter corrections with cosine-annealed scheduling to achieve rigorous convergence guarantees and state-of-the-art gradient alignment, demonstrating superior performance on both ill-conditioned synthetic problems and deep learning benchmarks like MNIST and CIFAR-10.