论文标题
使用随机补丁置换的视觉变压器的多任务分布式学习
Multi-Task Distributed Learning using Vision Transformer with Random Patch Permutation
论文作者
论文摘要
目前，人工智能在健康研究中的广泛应用受到数据可用性限制的阻碍。引入了分布式学习方法，例如联合学习（FL）和共享学习（SL），以解决此问题以及数据管理和所有权问题，并具有不同的优势和劣势。联邦分裂任务不可替代（Festa）的最新建议通过通过Vision Transformer（VIT）体系结构启用参与者之间的多任务协作来调和FL和SL的独特优点，但它们遭受了较高的沟通范围。为了解决这个问题，我们在这里提出了使用带有随机贴片置换的VIT的多任务分布式学习。 P-Festa没有像Festa一样使用基于CNN的头部，而是采用随机置换的简单补丁嵌入器，从而改善了多任务学习的性能而无需牺牲隐私。实验结果证实，该提出的方法显着提高了多任务协作，沟通效率和隐私保护的好处，从而阐明了医学成像领域中实用的多任务分布式学习。
The widespread application of artificial intelligence in health research is currently hampered by limitations in data availability. Distributed learning methods such as federated learning (FL) and shared learning (SL) are introduced to solve this problem as well as data management and ownership issues with their different strengths and weaknesses. The recent proposal of federated split task-agnostic (FeSTA) learning tries to reconcile the distinct merits of FL and SL by enabling the multi-task collaboration between participants through Vision Transformer (ViT) architecture, but they suffer from higher communication overhead. To address this, here we present a multi-task distributed learning using ViT with random patch permutation. Instead of using a CNN based head as in FeSTA, p-FeSTA adopts a randomly permuting simple patch embedder, improving the multi-task learning performance without sacrificing privacy. Experimental results confirm that the proposed method significantly enhances the benefit of multi-task collaboration, communication efficiency, and privacy preservation, shedding light on practical multi-task distributed learning in the field of medical imaging.
