论文标题
使用子图近似的图形卷积网络的分布式培训
Distributed Training of Graph Convolutional Networks using Subgraph Approximation
论文作者
论文摘要
现代的机器学习技术已成功地适应了以图形为模型的数据。但是，许多现实世界图通常非常大，并且不适合记忆，通常会使他们在它们上训练机器学习模型的问题。分布式培训已成功地用于缓解记忆问题并加快机器学习域中的培训，其中假定输入数据是独立相同的分布式（I.I.D）。但是，分布非I.I.D数据的培训，例如用作图形卷积网络（GCN）中训练输入的图形训练会导致准确性问题，因为在图形分区边界上丢失了信息。 在本文中，我们提出了一种培训策略，该策略通过子图近似方案来减轻图形多个分区的丢失信息。我们提出的方法用少量的边缘和顶点信息增强了每个子图，这些信息均来自所有其他子图。子图近似方法有助于分布式训练系统以单机器的准确性收敛，同时保持记忆足迹低，并最小化机器之间的同步开销。
Modern machine learning techniques are successfully being adapted to data modeled as graphs. However, many real-world graphs are typically very large and do not fit in memory, often making the problem of training machine learning models on them intractable. Distributed training has been successfully employed to alleviate memory problems and speed up training in machine learning domains in which the input data is assumed to be independently identical distributed (i.i.d). However, distributing the training of non i.i.d data such as graphs that are used as training inputs in Graph Convolutional Networks (GCNs) causes accuracy problems since information is lost at the graph partitioning boundaries. In this paper, we propose a training strategy that mitigates the lost information across multiple partitions of a graph through a subgraph approximation scheme. Our proposed approach augments each sub-graph with a small amount of edge and vertex information that is approximated from all other sub-graphs. The subgraph approximation approach helps the distributed training system converge at single-machine accuracy, while keeping the memory footprint low and minimizing synchronization overhead between the machines.
