Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/43582
Title: Optimizing big data processing performance in the public cloud : opportunities and approaches
Authors: Wang, D 
Liu, J
Issue Date: 2015
Publisher: Institute of Electrical and Electronics Engineers
Source: IEEE network, 2015, v. 29, no. 5, 7293302, p. 31-35 How to cite?
Journal: IEEE network 
Abstract: Today's lightning fast data generation from massive sources is calling for efficient big data processing, which imposes unprecedented demands on the computing and networking infrastructures. State-of-the-art tools, most notably MapReduce, are generally performed on dedicated server clusters to explore data parallelism. For grass roots users or non-computing professionals, the cost of deploying and maintaining a large-scale dedicated server clusters can be prohibitively high, not to mention the technical skills involved. On the other hand, public clouds allow general users to rent virtual machines and run their applications in a pay-as-you-go manner with ultra-high scalability with minimal upfront costs. This new computing paradigm has gained tremendous success in recent years, becoming a highly attractive alternative to dedicated server clusters. This article discusses the critical challenges and opportunities when big data meet the public cloud. We identify the key differences between running big data processing in a public cloud and in dedicated server clusters. We then present two important problems for efficient big data processing in the public cloud, resource provisioning (i.e., how to rent VMs) and VM-MapReduce job/task scheduling (i.e., how to run MapReduce after the VMs are constructed). Each of these two questions have a set of problems to solve. We present solution approaches for certain problems, and offer optimized design guidelines for others. Finally, we discuss our implementation experiences.
URI: http://hdl.handle.net/10397/43582
ISSN: 0890-8044
DOI: 10.1109/MNET.2015.7293302
Appears in Collections:Journal/Magazine Article

Access
View full-text via PolyU eLinks SFX Query
Show full item record

SCOPUSTM   
Citations

2
Last Week
0
Last month
Citations as of Oct 8, 2018

Page view(s)

24
Last Week
0
Last month
Citations as of Oct 14, 2018

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.