Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/30970
Title: Joint scheduling of mapReduce jobs with servers : performance bounds and experiments
Authors: Yuan, Y
Wang, D 
Liu, J
Keywords: Approximation theory
Computational complexity
Parallel programming
Scheduling
Issue Date: 2014
Publisher: IEEE
Source: IEEE INFOCOM 2014 : IEEE Conference on Computer Communications, April 27-May 2, 2014, Toronto, ON, Canada, p. 2175-2183 How to cite?
Abstract: MapReduce has achieved tremendous success for large-scale data processing in data centers. A key feature distinguishing MapReduce from previous parallel models is that it interleaves parallel and sequential computation. Past schemes, and especially their theoretical bounds, on general parallel models are therefore, unlikely to be applied to MapReduce directly. There are many recent studies on MapReduce job and task scheduling. These studies assume that the servers are assigned in advance. In current data centers, multiple MapReduce jobs of different importance levels run together. In this paper, we investigate a schedule problem for MapReduce taking server assignment into consideration as well. We formulate a MapReduce server-job organizer problem (MSJO) and show that it is NP-complete. We develop a 3-approximation algorithm and a fast heuristic. We evaluate our algorithms through both simulations and experiments on Amazon EC2 with an implementation in Hadoop. The results confirm the advantage of our algorithms.
URI: http://hdl.handle.net/10397/30970
ISBN: 
DOI: 10.1109/INFOCOM.2014.6848160
Appears in Collections:Conference Paper

Access
View full-text via PolyU eLinks SFX Query
Show full item record

SCOPUSTM   
Citations

7
Last Week
3
Last month
Citations as of Sep 17, 2017

Page view(s)

33
Last Week
0
Last month
Checked on Sep 18, 2017

Google ScholarTM

Check

Altmetric



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.