Inhalt des Dokuments
The contribution of this thesis is to explore the integration between Parallel Processing frameworks (PPF) with Software Defined Network (SDN) in order to exploit knowledge in the PPF side about data statistics for better routing of packets in data-intensive workloads. The TCP/IP network stack was born with data and control plane on the same channel. By control plane we understand the routing protocols like BGP and OSPF. SDN is a new concept for TCP/IP networks which proposes more flexible networks by separating the data plane from the control plane. In this work, we propose an architecture and algorithms to support such integration and developed some components as a prototype, specially a component called Network Scheduler. Based on PPF statistics the Network Scheduler provisions a suitable network for a job that will run on the PPF. Instead of relying on the standard network protocols to define the path of data to be transferred, the network scheduler relies on the amount of data that will be transferred and allocates the proper optimized paths on the network, increasing the bandwidth utilization.