论文标题
用于联合学习的单发超参数优化：一般算法和分析
Single-shot Hyper-parameter Optimization for Federated Learning: A General Algorithm & Analysis
论文作者
论文摘要
我们解决了联合学习（FL-HPO）的超参数优化（HPO）的相对未开发的问题。我们引入了联合损失表面聚集（Flora），这是一个通用的FL-HPO解决方案框架，可以解决表格数据的用例和任何机器学习（ML）模型（包括梯度增强培训算法），因此进一步扩大了FL-HPO的范围。 Flora启用了单发FL-HPO：识别一组良好的超参数，随后在单个FL训练中使用。因此，与没有HPO的FL培训相比，它可以实现具有最小的额外通信开销的FL-HPO解决方案。从理论上讲，我们表征了FL-HPO的最佳差距，该差距明确说明了当事方本地数据分布的异质性非IID性质，这是FL系统的主要特征。我们对七个OpenML数据集对多种ML算法的菌群进行的经验评估表明，对所考虑的基线进行了显着的模型准确性改进，以及对FL-HPO培训涉及的各方增加的稳健性。
We address the relatively unexplored problem of hyper-parameter optimization (HPO) for federated learning (FL-HPO). We introduce Federated Loss SuRface Aggregation (FLoRA), a general FL-HPO solution framework that can address use cases of tabular data and any Machine Learning (ML) model including gradient boosting training algorithms and therefore further expands the scope of FL-HPO. FLoRA enables single-shot FL-HPO: identifying a single set of good hyper-parameters that are subsequently used in a single FL training. Thus, it enables FL-HPO solutions with minimal additional communication overhead compared to FL training without HPO. We theoretically characterize the optimality gap of FL-HPO, which explicitly accounts for the heterogeneous non-IID nature of the parties' local data distributions, a dominant characteristic of FL systems. Our empirical evaluation of FLoRA for multiple ML algorithms on seven OpenML datasets demonstrates significant model accuracy improvements over the considered baseline, and robustness to increasing number of parties involved in FL-HPO training.
