Inhalt des Dokuments
The continuously increasing volume of data has had a huge impact on information systems and businesses. With the forthcoming of Internet of things (IoT), the amount of information available will be even greater. Gartner is expecting a 30-fold increase in devices and sensors by 2020 and foresees the emergence of new business models that take advantage of real-time streaming data from these devices. New kind of technological tools, called Steam Processing Engines (SPEs), have risen to facilitate the processing of large-scale data streams. The real-time handling of information, however, introduces unique challenges in terms of resiliency and fault tolerance that affect the implementations as well as the operation of such solutions.
Our main contributions are threefold. First, a survey about the impacts these new technologies have on the operations side is presented. Next, three distinct alternative implementations for resilient online machine learning applications are proposed. Our focus is on finding a solution to handle shared state in a SPE that gains its fault tolerance through linear, deterministic workflows. Finally, a complementary analysis about the integration test that were conducted on a cluster of servers is given.