acm-header
Sign In

Communications of the ACM

ACM TechNews

Big Data Reaches to the Stratosphere


View as: Print Mobile App Share: Send by email Share on reddit Share on StumbleUpon Share on Hacker News Share on Tweeter Share on Facebook
Stratosphere is an open source platform for analyzing very large data sets in parallel computing environments such as clouds or clusters.

A position paper developed at the Big Data and Extreme-scale Computing workshop emphasizes the challenges of big data analytics.

Credit: EIT ICT Labs

A position paper by Berlin Technical University professor Volker Markl developed at the recent Big Data and Extreme-scale Computing workshop emphasizes the goals and challenges of big data analytics.

"Today's existing technologies have reached their limits due to big data requirements, which involve data volume, data rate and heterogeneity, and the complexity of the analysis algorithms, which go beyond relational algebra, employing complex user-defined functions, iterations, and distributed state," Markl writes.

To correct this requires deploying declarative language concepts for big data systems. However, the effort presents several challenges, including designing a programming language specification that does not demand systems programming skills; plotting out programs expressed in this language to a computing platform of their own choosing, and performing them in a scalable fashion.

Markl says next-generation big data analytics frameworks such as Stratosphere can enable deeper data analysis. Stratosphere integrates the advantages of MapReduce/Hadoop with programming abstractions in Java and Scala and a high-performance runtime to facilitate massively parallel in-situ data analytics. Markl says Stratosphere is so far the only system for big data analytics featuring a query optimizer for advanced data analysis programs that transcend relational algebra, and the goal is to enable data scientists to concentrate on the main task without spending too much time on instilling scalability.

From HPC Wire
View Full Article

 

Abstracts Copyright © 2014 Information Inc., Bethesda, Maryland, USA


 

No entries found

Sign In for Full Access
» Forgot Password? » Create an ACM Web Account