TU Berlin

Fachgebiet Datenbanksysteme und InformationsmanagementPublikationen

Logo FG DIMA-new  65px

Inhalt

zur Navigation

Publikationen

Efficient Migration of Very Large Distributed State for Scalable Stream Processing
Zitatschl├╝ssel DelMonte17
Autor Bonaventura Del Monte
Jahr 2017
Journal Proceedings of the VLDB 2017 PhD Workshop
Zusammenfassung Any scalable stream data processing engine must handle thedynamic nature of data streams and it must quickly react toevery fluctuation in the data rate. Many systems successfullyaddress data rate spikes through resource elasticity and dynamicload balancing. The main challenge is the presence of stateful op-erators because their internal, mutable state must be scaled outwhile assuring fault-tolerance and continuous stream processing.Both rescaling, load balancing, and recovering demand statemovement among work units. Therefore, how to guarantee thosefeatures in the presence of large distributed state with minimalimpact on the performance is still an open issue. We propose anincremental migration mechanism for fine-grained state shardsthrough periodicincremental checkpointsandreplica groups.This enables moving large state with minimal impact on streamprocessing. Finally, we present a low-latency hand-over protocolthat smoothly migrates tuples processing among work units.
Link zur Originalpublikation Download Bibtex Eintrag

Navigation

Direktzugang

Schnellnavigation zur Seite über Nummerneingabe