Physical-Layer Arithmetic for Federated Learning in Uplink MU-MIMO Enabled Wireless Networks Tao Huang',Baoliu Yef,Zhihao Qu,Bin Tang',Lei Xief,Sanglu Lut National Key Laboratory for Novel Software Technology,Nanjing University,China College of Computer and Information,Hohai University,China Email:schrodinger.huang@gmail.com,yebl@nju.edu.cn,quzhihao@hhu.edu.cn,[tb,Ixie,sanglu)@nju.edu.cn Abstract-Federated learning is a very promising machine To improve the communication efficiency,methods based learning paradigm where a large number of clients cooperatively on reducing the size of updates needed to transmit have been train a global model using their respective local data.In this paper,we consider the application of federated learning in widely studied [5]-[9],by performing techniques like quanti- wireless networks featuring uplink multiuser multiple-input and zation,sparsification.etc.In addition to simply compressing multiple-output (MU-MIMO),and aim at optimizing the commu- client-side updates,there is also work focusing on improving nication efficiency during the aggregation of client-side updates the communication efficiency of federated learning by utiliz- by exploiting the inherent superposition of radio frequency ing the characteristics of networks.Specifically,for wireless (RF)signals.We propose a novel approach named Physical- Layer Arithmetic (PhyArith),where the clients encode their local networks where uplink multiuser multiple-input and multiple- updates into aligned digital sequences which are converted into output (MU-MIMO)is enabled,i.e.,clients can concurrently RF signals for sending to the server simultaneously,and the send their updates to the server in the same time-frequency server directly recovers the exact summation of these updates resource,recent work like [10]-[13]study exploiting the as required from the superimposed RF signal by employing a superposition of radio frequency (RF)signals to effectively customized sum-product algorithm.PhyArith is compatible with aggregate updates by averaging them,based on the technique commodity devices due to the use of full digital operation in both the client-side encoding and the server-side decoding processes. called over-the-air computation (AirComp).These AirComp and can also be integrated with other updates compression based based work use uncoded analog transmission to send aligned acceleration techniques.Simulation results show that PhyArith client-side updates in the form of vector,each of which has further improves the communication efficiency by 1.5 to 3 times the same number of elements placed in the same order.Clients for training LeNet-5,compared with solutions only applying in these work are assumed to be equipped with a special updates compression. device featuring linear-analog-modulation and pre-channel- compensation (or simply assumed have no fading channel). I.INTRODUCTION such that the superimposed RF signal can be well utilized for Federated learning and related decentralized learning are fast update aggregation.However,these work are not dedicated the kind of machine learning paradigm where the goal is to to obtaining the exact average of updates.While the channel train a high quality centralized model,while training data noise is not negligible and the pre-channel-compensation of remains distributed over a large number of clients [1]-[4]. each client is not perfect,their obtained average may contain In each round of learning algorithms for this paradigm,each significant aggregation error,which may seriously affect the client independently computes an update to the current model convergence of the global model to train. based on its local data,and sends this update to a central In this paper,we study utilizing the superposition of RF server,where the client-side updates are aggregated to compute signals to efficiently aggregate client-side updates in federated a new global model.Communicating the model updates in learning by using commodity devices dedicated to modern each round has been observed to be a significant performance wireless networks featuring uplink MU-MIMO,e.g.,802.11ax bottleneck [5][6]for this paradigm.It is particularly serious [14].where neither linear-analog-modulation nor pre-channel- for federated learning,since its typical clients are smart phones compensation is necessarily available.We target at reliably ob- and loT devices,which are with unreliable and relatively slow taining the exact average of updates,such that the convergence network connections. of the global model to train is guaranteed for realistic channel conditions.Inspired by the fact that the amount of information This work was supported in part by the National K&D Program of China of all updates is greater than that of their summation,our under Grant 2018YFB1004704,in part by the National Natural Science Foun- idea is that we can directly recover the exact summation dation of China under Grant 61832005.Grant 61872171,and Grant 61872174 in part by the Key R&D of Jiangsu Province under Grant BE2017152,in based on the received superimposed RF signal,and then part by the Natural Science Foundation of Jiangsu Province under Grant calculate their average.It should have lower outage probability BK20190058.in part by Project funded by China Postdoctoral Science compared with the conventional multiuser detection (MUD) Foundation under Grant 2019M661709,and in part by the Collaborative Innovation Center of Novel Software Technology and Industrialization. based solution to deal with such mutual interference in uplink Baoliu Ye and Bin Tang are corresponding authors. MU-MIMO,i.e.,separating these colliding data streams basedPhysical-Layer Arithmetic for Federated Learning in Uplink MU-MIMO Enabled Wireless Networks Tao Huang† , Baoliu Ye† , Zhihao Qu‡ , Bin Tang† , Lei Xie† , Sanglu Lu† †National Key Laboratory for Novel Software Technology, Nanjing University, China ‡College of Computer and Information, Hohai University, China Email: schrodinger.huang@gmail.com, yebl@nju.edu.cn, quzhihao@hhu.edu.cn, {tb, lxie, sanglu}@nju.edu.cn Abstract—Federated learning is a very promising machine learning paradigm where a large number of clients cooperatively train a global model using their respective local data. In this paper, we consider the application of federated learning in wireless networks featuring uplink multiuser multiple-input and multiple-output (MU-MIMO), and aim at optimizing the commu￾nication efficiency during the aggregation of client-side updates by exploiting the inherent superposition of radio frequency (RF) signals. We propose a novel approach named Physical￾Layer Arithmetic (PhyArith), where the clients encode their local updates into aligned digital sequences which are converted into RF signals for sending to the server simultaneously, and the server directly recovers the exact summation of these updates as required from the superimposed RF signal by employing a customized sum-product algorithm. PhyArith is compatible with commodity devices due to the use of full digital operation in both the client-side encoding and the server-side decoding processes, and can also be integrated with other updates compression based acceleration techniques. Simulation results show that PhyArith further improves the communication efficiency by 1.5 to 3 times for training LeNet-5, compared with solutions only applying updates compression. I. INTRODUCTION Federated learning and related decentralized learning are the kind of machine learning paradigm where the goal is to train a high quality centralized model, while training data remains distributed over a large number of clients [1]–[4]. In each round of learning algorithms for this paradigm, each client independently computes an update to the current model based on its local data, and sends this update to a central server, where the client-side updates are aggregated to compute a new global model. Communicating the model updates in each round has been observed to be a significant performance bottleneck [5] [6] for this paradigm. It is particularly serious for federated learning, since its typical clients are smart phones and IoT devices, which are with unreliable and relatively slow network connections. This work was supported in part by the National K&D Program of China under Grant 2018YFB1004704, in part by the National Natural Science Foun￾dation of China under Grant 61832005, Grant 61872171, and Grant 61872174, in part by the Key R&D of Jiangsu Province under Grant BE2017152, in part by the Natural Science Foundation of Jiangsu Province under Grant BK20190058, in part by Project funded by China Postdoctoral Science Foundation under Grant 2019M661709, and in part by the Collaborative Innovation Center of Novel Software Technology and Industrialization. Baoliu Ye and Bin Tang are corresponding authors. To improve the communication efficiency, methods based on reducing the size of updates needed to transmit have been widely studied [5]–[9], by performing techniques like quanti￾zation, sparsification, etc. In addition to simply compressing client-side updates, there is also work focusing on improving the communication efficiency of federated learning by utiliz￾ing the characteristics of networks. Specifically, for wireless networks where uplink multiuser multiple-input and multiple￾output (MU-MIMO) is enabled, i.e., clients can concurrently send their updates to the server in the same time-frequency resource, recent work like [10]–[13] study exploiting the superposition of radio frequency (RF) signals to effectively aggregate updates by averaging them, based on the technique called over-the-air computation (AirComp). These AirComp based work use uncoded analog transmission to send aligned client-side updates in the form of vector, each of which has the same number of elements placed in the same order. Clients in these work are assumed to be equipped with a special device featuring linear-analog-modulation and pre-channel￾compensation (or simply assumed have no fading channel), such that the superimposed RF signal can be well utilized for fast update aggregation. However, these work are not dedicated to obtaining the exact average of updates. While the channel noise is not negligible and the pre-channel-compensation of each client is not perfect, their obtained average may contain significant aggregation error, which may seriously affect the convergence of the global model to train. In this paper, we study utilizing the superposition of RF signals to efficiently aggregate client-side updates in federated learning by using commodity devices dedicated to modern wireless networks featuring uplink MU-MIMO, e.g., 802.11ax [14], where neither linear-analog-modulation nor pre-channel￾compensation is necessarily available. We target at reliably ob￾taining the exact average of updates, such that the convergence of the global model to train is guaranteed for realistic channel conditions. Inspired by the fact that the amount of information of all updates is greater than that of their summation, our idea is that we can directly recover the exact summation based on the received superimposed RF signal, and then calculate their average. It should have lower outage probability compared with the conventional multiuser detection (MUD) based solution to deal with such mutual interference in uplink MU-MIMO, i.e., separating these colliding data streams based