论文标题
常见问题解答：沟通高效的联合DNN体系结构和量化搜索个性化硬件偏好
FAQS: Communication-efficient Federate DNN Architecture and Quantization Co-Search for personalized Hardware-aware Preferences
论文作者
论文摘要
由于用户隐私和监管限制，联合学习（FL）被提议作为分散的神经网络（DNN）的分布式学习框架（DNN）。 FL的最新进展已应用神经体系结构搜索（NAS）来替换预定义的一尺拟合的ALL DNN模型，该模型并不适用于各种数据分布的所有任务以及可搜索的DNN架构。但是，以前的方法遭受了昂贵的通信成本，而服务器和客户端之间的频繁大型模型参数会损失。当结合NAS算法时，这种难度通常会进一步放大，NAS算法通常需要过度计算和巨大的模型存储。为此，我们提出了一个常见问题解答，这是一个有效的个性化FL-NAS定量框架，可以通过三个功能降低通信成本：体重分享超级内核，位分类量化和掩盖的传输。常见问题解答有一个负担得起的搜索时间，并且每回合的传输消息大小都非常有限。通过为本地客户设置不同的人性化帕累托功能损失，常见问题解答可以为各种用户偏好产生异质的硬件感知模型。实验结果表明，与正常的FL框架相比，常见问题解答的平均每轮频率为1.58倍，而与FL+NAS框架相比，平均降低了。
Due to user privacy and regulatory restrictions, federate learning (FL) is proposed as a distributed learning framework for training deep neural networks (DNN) on decentralized data clients. Recent advancements in FL have applied Neural Architecture Search (NAS) to replace the predefined one-size-fit-all DNN model, which is not optimal for all tasks of various data distributions, with searchable DNN architectures. However, previous methods suffer from expensive communication cost rasied by frequent large model parameters transmission between the server and clients. Such difficulty is further amplified when combining NAS algorithms, which commonly require prohibitive computation and enormous model storage. Towards this end, we propose FAQS, an efficient personalized FL-NAS-Quantization framework to reduce the communication cost with three features: weight-sharing super kernels, bit-sharing quantization and masked transmission. FAQS has an affordable search time and demands very limited size of transmitted messages at each round. By setting different personlized pareto function loss on local clients, FAQS can yield heterogeneous hardware-aware models for various user preferences. Experimental results show that FAQS achieves average reduction of 1.58x in communication bandwith per round compared with normal FL framework and 4.51x compared with FL+NAS framwork.
