Please use this identifier to cite or link to this item:
Title: CoMan : managing bandwidth across computing frameworks in multiplexed datacenters
Authors: Li, WX
Guo, DK
Liu, AX
Li, KQ
Qi, H
Guo, S 
Munir, A
Tao, XY
Keywords: Data-parallel computing frameworks
Multiplexed datacenter
Bandwidth management
Issue Date: 2018
Publisher: Institute of Electrical and Electronics Engineers
Source: IEEE transactions on parallel and distributed systems, 1 May 2018, v. 29, no. 5, p. 1013-1029 How to cite?
Journal: IEEE transactions on parallel and distributed systems 
Abstract: Inefficient bandwidth sharing in a datacenter network, between different application frameworks, e.g., MapReduce and Spark, can lead to inelastic and skewed usage of link bandwidth and increased completion times for the applications. Existing work, however, either solely focuses on managing computation and storage resources or controlling only sending/receiving rate at hosts. In this paper, we present CoMan, a solution that provides global in-network bandwidth management in multiplexed data centers, with two goals: improving bandwidth utilization and reducing application completion time. CoMan first designs a novel abstraction of virtual link groups (VLGs) to establish a shared bandwidth resource pool. Based on this pool, CoMan implements a three-level bandwidth allocation model, which enables elastic bandwidth sharing among computing frameworks as well as guarantees network performance for the applications. CoMan further improves the bandwidth utilization by devising a VLG dependency graph and solves an optimization problem to guide the path selection using 3/2-approximation algorithm. We conduct comprehensive trace-driven simulations as well as small-scale testbed experiments to evaluate the performance of CoMan. Extensive simulation results show that CoMan improves the bandwidth utilization and speeds up the application completion time by up to 2.83 x and 6.68 x, respectively, compared to the ECMP + ElasticSwitch solution. Our implementation also verifies that CoMan can realistically speed up the application completion times by 2.32 x on average.
ISSN: 1045-9219
EISSN: 1558-2183
DOI: 10.1109/TPDS.2017.2788003
Appears in Collections:Journal/Magazine Article

View full-text via PolyU eLinks SFX Query
Show full item record

Page view(s)

Citations as of Nov 12, 2018

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.