论文标题
开发21厘米宇宙学的高吞吐量基于云的数据管道
Development of a High Throughput Cloud-Based Data Pipeline for 21 cm Cosmology
论文作者
论文摘要
我们提出了一个基于云的计算工作流程的案例研究，用于从Murchison广场阵列（MWA）宇宙学实验处理大型天文数据集。云计算非常适合大规模的，情节计算，因为它在付费模型中提供了极端的可扩展性。这有助于快速周转时间来测试计算昂贵的分析技术。我们描述了如何使用Amazon Web Services（AWS）云平台进行有效，经济测试和实施我们的数据分析管道。我们讨论了与AWS现货市场合作的挑战，该市场以更长的处理时间降低了成本，我们通过蒙特卡洛模拟探索了这种权衡。
We present a case study of a cloud-based computational workflow for processing large astronomical data sets from the Murchison Widefield Array (MWA) cosmology experiment. Cloud computing is well-suited to large-scale, episodic computation because it offers extreme scalability in a pay-for-use model. This facilitates fast turnaround times for testing computationally expensive analysis techniques. We describe how we have used the Amazon Web Services (AWS) cloud platform to efficiently and economically test and implement our data analysis pipeline. We discuss the challenges of working with the AWS spot market, which reduces costs at the expense of longer processing turnaround times, and we explore this tradeoff with a Monte Carlo simulation.
