Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/79355
Title: Online internet traffic measurement and monitoring using spark streaming
Authors: Zhou, B
Li, J
Guo, S 
Wu, J
Hu, Y
Zhu, L
Issue Date: 2018
Publisher: Institute of Electrical and Electronics Engineers Inc.
Source: 2017 IEEE Global Communications Conference, GLOBECOM 2017 - Proceedings, 2018, v. 2018-January, p. 1-6 How to cite?
Abstract: Due to the explosive growth of Internet traffic, network operators must be able to monitor the whole network situations and manage their network resources in an efficient way. Traditional network analysis method that works on a single machine are no longer suitable for this huge traffic data due to its poor processing ability. Some big data frameworks, such as Hadoop and Spark, can handle such analysis job even for large network traffic, but they are inherently designed for offline data analysis. In this paper, we treat the online network analysis as a stream analysis problem and use Spark Streaming to cope with the high-speed Internet traffic data in real time. The system consists of two parts, collector and stream processor. Firstly, several collectors capture network traffic data from switches through mirrored ports and send the packet information to a central stream processor which is a cluster running Spark Streaming. Then, the stream processor analyzes the input data streams and calculates Internet performance metrics. We take TCP performance monitoring as an example to show how network measurement can be done using the stream processing platform. Finally, we conducted typical experiments in a cluster of 3 computers with the standalone mode, showing that our system performs well in huge Internet traffic measurement and monitoring.
Description: 2017 IEEE Global Communications Conference, GLOBECOM 2017, Singapore, 4-8 December 2017
URI: http://hdl.handle.net/10397/79355
ISBN: 9781509050192
DOI: 10.1109/GLOCOM.2017.8255000
Appears in Collections:Conference Paper

Access
View full-text via PolyU eLinks SFX Query
Show full item record

SCOPUSTM   
Citations

3
Last Week
0
Last month
Citations as of Mar 29, 2019

Page view(s)

13
Citations as of May 21, 2019

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.