Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/80445
DC FieldValueLanguage
dc.contributor.authorYang, Len_US
dc.contributor.authorLiu, XXen_US
dc.contributor.authorCao, JNen_US
dc.contributor.authorWang, ZYen_US
dc.date.accessioned2019-03-26T09:17:13Z-
dc.date.available2019-03-26T09:17:13Z-
dc.date.issued2018-
dc.identifier.citationIEEE access, 2018, v. 6, p. 66600-66611en_US
dc.identifier.issn2169-3536-
dc.identifier.urihttp://hdl.handle.net/10397/80445-
dc.description.abstractAs an increasing number of big data processing platforms like Hadoop, Spark, and Storm appear and normally share the resources in the data center, it has been important and challenging to schedule various jobs from these platforms onto the underlying data center resources such that the overall job completion time is minimized. To solve the problem, the existing work either focus on the task-level scheduling techniques, such as Quincy and delay scheduling, or focus on the network flow scheduling techniques, such as D3 and preemptive distributed quick. These works deal with the scheduling of tasks and network flows separately and cannot achieve optimal performance. The reason is that the task scheduling without regard of the available network bandwidths may generate the task placement that causes serious network congestions and thus leads to long data transmission time. In this paper, we propose the joint scheduling technique by coordinating the task placement and the scheduling of network flows arising from these tasks. We develop a software-defined network (SDN)-based online scheduling framework which selects the task placement based on the available bandwidth on the SDN switches and at meanwhile optimally allocates the bandwidth to each data flow. Comprehensive trace-driven simulations show that the joint scheduling technique can take full use of the network bandwidth and thus reduce the job completion time by 55% on average compared with the benchmark methods.en_US
dc.description.sponsorshipDepartment of Computingen_US
dc.language.isoenen_US
dc.publisherInstitute of Electrical and Electronics Engineersen_US
dc.relation.ispartofIEEE accessen_US
dc.rights© 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.en_US
dc.rightsPost with permission of the publisher.en_US
dc.rightsThe following publication Yang, L., Liu, X. X., Cao, J. N., & Wang, Z. Y. (2018). Joint scheduling of tasks and network flows in big data clusters. IEEE Access, 6, 66600-66611 is available at https://dx.doi.org/10.1109/ACCESS.2018.2878864en_US
dc.subjectTask schedulingen_US
dc.subjectFlow schedulingen_US
dc.subjectData centersen_US
dc.subjectSoftware defined networksen_US
dc.titleJoint scheduling of tasks and network flows in big data clustersen_US
dc.typeJournal/Magazine Articleen_US
dc.identifier.spage66600-
dc.identifier.epage66611-
dc.identifier.volume6-
dc.identifier.doi10.1109/ACCESS.2018.2878864-
dc.identifier.isiWOS:000452355000001-
dc.description.validate201903 bcrc-
dc.description.oapublished_final-
Appears in Collections:Journal/Magazine Article
Files in This Item:
File Description SizeFormat 
Yang_Network_Flows_Clusters.pdf7.2 MBAdobe PDFView/Open
Access
View full-text via PolyU eLinks SFX Query
Show simple item record
PIRA download icon_1.1View/Download Contents

Page view(s)

41
Citations as of Dec 4, 2019

Download(s)

40
Citations as of Dec 4, 2019

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.