direkt zum Inhalt springen

direkt zum Hauptnavigationsmenü

Sie sind hier

TU Berlin

Inhalt des Dokuments

Publikationen

Analyzing Efficient Stream Processing on Modern Hardware
Zitatschlüssel DBLP:journals/pvldb/ZeuchBRMKLRTM19
Autor Steffen Zeuch, Sebastian Breß, Tilmann Rabl, Bonaventura Del Monte, Jeyhun Karimov, Clemens Lutz, Manuel Renz, Jonas Traub, Volker Markl
Seiten 516-530
Jahr 2019
DOI /10.14778/3303753.3303758
Journal Proceedings of the VLDB Endowment
Jahrgang 12(5)
Monat January
Zusammenfassung Modern Stream Processing Engines (SPEs) process large data volumes under tight latency constraints. Many SPEs execute processing pipelines using message passing on shared-nothing architectures and apply a partition-based scale-out strategy to handle high-velocity input streams. Further-more, many state-of-the-art SPEs rely on a Java Virtual Ma-chine to achieve platform independence and speed up system development by abstracting from the underlying hardware. In this paper, we show that taking the underlying hard-ware into account is essential to exploit modern hardware efficiently. To this end, we conduct an extensive experimen-tal analysis of current SPEs and SPE design alternatives optimized for modern hardware. Our analysis highlights po-tential bottlenecks and reveals that state-of-the-art SPEs are not capable of fully exploiting current and emerging hard-ware trends, such as multi-core processors and high-speed networks. Based on our analysis, we describe a set of design changes to the common architecture of SPEs to scale-up on modern hardware. We show that the single-node throughput can be increased by up to two orders of magnitude compared to state-of-the-art SPEs by applying specialized code genera-tion, fusing operators, batch-style parallelization strategies, and optimized windowing. This speedup allows for deploy-ing typical streaming applications on a single or a few nodes instead of large clusters.
Link zur Originalpublikation Download Bibtex Eintrag

Zusatzinformationen / Extras

Direktzugang:

Schnellnavigation zur Seite über Nummerneingabe