论文标题
PP-MARL：有效保存隐私的多代理强化学习，用于交流中的合作智能
PP-MARL: Efficient Privacy-Preserving Multi-Agent Reinforcement Learning for Cooperative Intelligence in Communications
论文作者
论文摘要
预计合作智能（CI）将成为下一代网络中不可或缺的元素，因为它可以汇总多个设备的功能和智能。多代理增强学习（MARL）是通过使代理之间有效的协作解决顺序问题的流行方法来解决CI的CI。但是，确保MARL隐私保护是一项艰巨的任务，因为存在通过共享信息相互依存的异质代理人的存在。实施隐私保护技术，例如数据加密和联合学习来介绍著名的开销（例如，计算和带宽）。为了克服这些挑战，我们提出了PP-MARL，这是MARL的有效保护隐私学习计划。 PP-MARL利用同态加密（HE）和差异隐私（DP）来保护隐私，同时引入分裂学习以减少共享消息的量减少开销，然后提高效率。我们在两个与通信相关用例中应用和评估PP-MARL。仿真结果表明，PP-MARL可以实现有效且可靠的合作，其隐私保护更好的1.1-6倍，并且间接费用较低（例如，带宽减少84-91％）是最先进的方法。
Cooperative intelligence (CI) is expected to become an integral element in next-generation networks because it can aggregate the capabilities and intelligence of multiple devices. Multi-agent reinforcement learning (MARL) is a popular approach for achieving CI in communication problems by enabling effective collaboration among agents to address sequential problems. However, ensuring privacy protection for MARL is a challenging task because of the presence of heterogeneous agents that learn interdependently via sharing information. Implementing privacy protection techniques such as data encryption and federated learning to MARL introduces the notable overheads (e.g., computation and bandwidth). To overcome these challenges, we propose PP-MARL, an efficient privacy-preserving learning scheme for MARL. PP-MARL leverages homomorphic encryption (HE) and differential privacy (DP) to protect privacy, while introducing split learning to decrease overheads via reducing the volume of shared messages, and then improve efficiency. We apply and evaluate PP-MARL in two communication-related use cases. Simulation results reveal that PP-MARL can achieve efficient and reliable collaboration with 1.1-6 times better privacy protection and lower overheads (e.g., 84-91% reduction in bandwidth) than state-of-the-art approaches.
