FLoRG: Federated Fine-tuning with Low-rank Gram Matrices and Procrustes Alignment
The paper proposes FLoRG, a federated fine-tuning framework that utilizes single low-rank Gram matrix aggregation and Procrustes alignment to eliminate aggregation errors and decomposition drift, thereby achieving superior downstream accuracy and significantly reduced communication overhead compared to existing state-of-the-art methods.