论文标题
用于高能物理分布的无服务器引擎
A Serverless Engine for High Energy Physics Distributed Analysis
论文作者
论文摘要
在过去的十年中,CERN的大型强子对撞机(LHC)生成了高能物理(HEP)领域的前所未有的数据。有兴趣分析此类数据的科学合作通常需要一台机器以外的计算能力。传统上,通过使用状态托管批处理计算系统在分布式环境中进行分析来解决此问题。尽管到目前为止,这种方法已经有效,但目前对该领域的计算需求的估计提出了巨大的扩展挑战。这样的托管方法可能不是解决这些方法的唯一可行方法,而无服务器体系结构可以提供有趣的替代方法,从而实现更大的扩展潜力。 这项工作描述了一种通过分布式无服务器计算引擎运行真实HEP科学应用程序的新颖方法。该引擎是基于Root建立的,该引擎是一个已建立的HEP数据分析软件,并将其计算分发给Amazon Web Services Lambda无服务器平台上的大量并发执行。多亏了开发的工具,物理学家才能够访问存储在CERN(也受到限制访问策略的数据集),并在其典型环境之外的远程基础架构上进行处理。在运行时监视无服务器功能的分析,以收集用于数据和计算密集型工作负载的性能指标。
The Large Hadron Collider (LHC) at CERN has generated in the last decade an unprecedented volume of data for the High-Energy Physics (HEP) field. Scientific collaborations interested in analysing such data very often require computing power beyond a single machine. This issue has been tackled traditionally by running analyses in distributed environments using stateful, managed batch computing systems. While this approach has been effective so far, current estimates for future computing needs of the field present large scaling challenges. Such a managed approach may not be the only viable way to tackle them and an interesting alternative could be provided by serverless architectures, to enable an even larger scaling potential. This work describes a novel approach to running real HEP scientific applications through a distributed serverless computing engine. The engine is built upon ROOT, a well-established HEP data analysis software, and distributes its computations to a large pool of concurrent executions on Amazon Web Services Lambda Serverless Platform. Thanks to the developed tool, physicists are able to access datasets stored at CERN (also those that are under restricted access policies) and process it on remote infrastructures outside of their typical environment. The analysis of the serverless functions is monitored at runtime to gather performance metrics, both for data- and computation-intensive workloads.