Efficient and Resilient Packet Recovery for Federated Learning via Approximation

초록

To alleviate network congestion resulting from packet retransmission in Federated Learning (FL) systems, the User Datagram Protocol (UDP) has been adopted. However, data loss under UDP inevitably degrades FL performance, as learning is sensitive to incomplete model parameters. This paper addresses such degradation by proposing a low-rank approximation-based parameter transmission method for FL. This approach decomposes model parameters at the transmitter to extract dominant singular values, which are essential for accurate approximation of the model parameters at the receiver. Although the selective delivery of singular values and vectors reduces communication overhead, their loss would cause severe performance degradation, making it essential to employ protection mechanisms. Therefore, to protect the extracted singular values and vectors, we use Systematic Network Coding (SysNC). The SysNC with low-rank approximation can recover original information under a packet loss environment, enhancing robustness against partial loss during the transmission of model parameters. Theoretical analysis and experiments confirm its effectiveness. As an example from our experiments, with a packet loss ratio of p=0.1, the proposed method achieves over 96% of the accuracy observed in the packet loss-free case, while reducing the end-to-end delay for model parameter transmission by approximately 50% compared to a UDP baseline.

키워드