Functionality-Oriented LLM Merging on the Fisher--Rao Manifold
This paper proposes a functionality-oriented model merging method that computes a weighted Karcher mean on the Fisher--Rao manifold via a practical fixed-point algorithm, effectively overcoming the representation collapse and scalability limitations of existing Euclidean-space approaches when combining multiple heterogeneous LLMs.