TU Berlin

Database Systems and Information Management GroupManaging Very Large Distributed State for Scalable Stream Processing

Logo FG DIMA-new  65px

Page Content

to Navigation

The EDADS Software Campus Project

Lupe

Despite recent advances in distributed computing and the availability of big data platforms, such as Apache Flink and Apache Spark, datasets continue to grow in magnitude. The advent of emerging technologies, such as the Internet of Things (IoT) further press the need for the development of novel solutions to expedite data analysis, particularly for streaming data. To overcome this challenge, computer scientists utilize varying approaches to cope with the data deluge. One approach involves computing sketches on large datasets, which enables us to approximate certain characteristics of the original data, such as the average, variance or extrema. In the EDADS (Efficient Data Analysis Based on Data Summaries) Software Campus Project our principal aim is to design and implement sketch algorithms for streaming data in modern dataflow engines. Consequently, this would serve to reduce the size of data streams. Furthermore, in contrast to examining the entire dataset, the sketch could then be used by data analytics (e.g., for anomaly detection) and thereby shorten the data analysis execution time.

Lupe

The EDADS project is funded by the Federal Ministry of Education and Research (BMBF) as part of the Software Campus program, and is supported by Huawei Technologies.

[1] Software Campus

Sponsored by the German Federal Ministry of Education and Research (BMBF), Software Campus (SC) is an executive development program aimed at developing tomorrow’s senior IT executives.

The SC program combines scientific leading-edge research with hands-on management practice in an entirely new and innovative concept. It is directed at outstanding computer science doctoral students who are interested in taking over executive management functions in industry. Awardees lead their own research projects in cooperation with industry partners over a one to two year period.

Kickoff des Jahrgangs 2017

Industry Partner

cooperation of Holtzbrinck Publishing Group

Project Duration: 06/2019 - 05/2020

Supervisor: Prof. Dr. Volker Markl

Advisor: Martin Kiefer

Funded by

Lupe

Navigation

Quick Access

Schnellnavigation zur Seite über Nummerneingabe