Decentralized federated learning and network topologies: an empirical study on convergence

Federated Learning is a well-known learning paradigm that allows the distributed training of machine learning models. Federated Learning keeps data in the source devices and communicates only the model's coefficients to a centralized server. This paper studies the decentralized flavor of Federated Learning. A peer-to-peer network replaces the centralized server, and nodes exchange model's coefficients directly. In particular, we look for empirical evidence on the effect of different network topologies and communication parameters on the convergence in the training of distributed models. Our observations suggest that small-world networks converge faster for small amounts of nodes, while xx are more suitable for larger setups.