Distributed Training using an Intelligent Network
This paper proposes an intelligent network framework for distributed training across wide area networks that combines multicast and in-line FPGA aggregation with optimized synchronization schedules to overcome bandwidth and latency constraints, thereby narrowing the performance gap with colocated training.