论文标题
表达,并行化和优化大数据应用程序的新框架
A New Framework for Expressing, Parallelizing and Optimizing Big Data Applications
论文作者
论文摘要
最初引入了前肢框架,作为使用为编译器开发的优化技术优化数据库查询的一种手段。自从引入以来,预兆已被证明更具用途,并且适用于数据库应用程序。在本文中,我们表明,更具体地说:K-Means聚类和Pagerank,可以使用原始的前肢框架来表达和优化大数据应用程序,从而自动生成这些应用程序的实现。这些实现比最新的,手写的MPI MPI C/C ++实现了K-均值和Pagerank的实现,并且明显胜过最先进的Hadoop实现。
The Forelem framework was first introduced as a means to optimize database queries using optimization techniques developed for compilers. Since its introduction, Forelem has proven to be more versatile and to be applicable beyond database applications. In this paper we show that the original Forelem framework can be used to express and optimize Big Data applications, more specifically: k-Means clustering and PageRank, resulting in automatically generated implementations of these applications. These implementations are more efficient than state-of-the-art, hand-written MPI C/C++ implementations of k-Means and PageRank, as well as significantly outperform state-of-the-art Hadoop implementations.