Haupt-Navigation

Inhalt des Dokuments

Publications

Bridging the Gap: Towards Optimization Across Linear and Relational Algebra
Citation key	KunftAKM16
Author	Andreas Kunft Alexander Alexandrov Asterios Katsifodimos Volker Markl
Year	2016
Journal	International Workshop on Algorithms and Systems for MapReduce and Beyond (BeyondMR), in conjunction with SIGMOD/PODS in San Francisco
Volume	2016
Abstract	Advanced data analysis typically requires some form of pre-processing in order to extract and transform data before processing it with machine learning and statistical analysis techniques. Pre-processing pipelines are naturally expressed in data ow APIs (e.g., MapReduce, Flink, etc.), while machine learning is expressed in linear algebra with iterations. Programmers therefore perform end-to-end data analysis utilizing multiple programming paradigms and systems. This impedance mismatch not only hinders productivity but also prevents optimization opportunities, such as sharing of physical data layouts (e.g., partitioning) and data structures among di erent parts of a data analysis program. The goal of this work is twofold. First, it aims to alleviate the impedance mismatch by allowing programmers to author complete end-to-end programs in one engine-independent language that is automatically parallelized. Second, it aims to enable joint optimizations over both relational and linear algebra. To achieve this goal, we present the design of Lara, a deeply embedded language in Scala which enables authoring scalable programs using two abstract data types (DataBag and Matrix) and control ow constructs. Programs written in Lara are compiled to an intermediate representation (IR) which enables optimizations across linear and relational algebra. The IR is finally used to compile code for different execution engines.

Link to publication [1] Link to original publication [2] Download Bibtex entry [3]

Back [4]

[5]

------ Links: ------

[1] https://www.researchgate.net/publication/303902033_
Bridging_the_gap_towards_optimization_across_linear_and
_relational_algebra

[2] http://dl.acm.org/citation.cfm?id=2926534.2926540

[3] https://www.dima.tu-berlin.de/menue/publications/pu
blications/parameter/maxhilfe/?no_cache=1&tx_sibibt
ex_pi1%5Bdownload_bibtex_uid%5D=1248289&tx_sibibtex
_pi1%5Bcontentelement%5D=tt_content%3A126920

[4] https://www.dima.tu-berlin.de/menue/publications/pu
blications/parameter/maxhilfe/

[5] https://www.dima.tu-berlin.de/menue/publications/bi
g_data_management_report/parameter/maxhilfe/