Please use this identifier to cite or link to this item:
Title: MatrixMap : programming abstraction and implementation of matrix computation for big data applications
Authors: Huangfu, Y
Cao, J 
Lu, H
Liang, G
Keywords: Graph processing
Big data
Parallel programming
Matrix computation
Machine learning
Issue Date: 2015
Publisher: Institute of Electrical and Electronics Engineers
Source: 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS), Melbourne, VIC, December 14-17, 2015, p.19-28 How to cite?
Abstract: The computation core of many big data applications can be expressed as general matrix computations, including linear algebra operations and irregular matrix operations. However, existing parallel programming systems such as Spark do not have programming abstraction and efficient implementation for general matrix computations. In this paper, we present MatrixMap, a unified and efficient data-parallel system for general matrix computations. MatrixMap provides powerful yet simple abstraction, consisting of a distributed data structure called bulk key matrix and a computation interface defined by matrix patterns. Users can easily load data into bulk key matrices and program algorithms into parallel matrix patterns. MatrixMap outperforms current state-of-the-art systems by employing three key techniques: matrix patterns with lambda functions for irregular and linear algebra matrix operations, asynchronous computation pipeline with optimized data shuffling strategies for specific matrix patterns and in-memory data structure reusing data in iterations. Moreover, it can automatically handle the parallelization and distribute execution of programs on a large cluster. The experiment results show that MatrixMap is 12 times faster than Spark.
ISBN: 978-0-7695-5785-4 (electronic)
978-1-4673-8670-8 (USB)
978-1-4673-8669-2 (print on demand(PoD))
EISSN: 1521-9097
DOI: 10.1109/ICPADS.2015.11
Appears in Collections:Conference Paper

View full-text via PolyU eLinks SFX Query
Show full item record


Last Week
Last month
Citations as of May 9, 2020


Last Week
Last month
Citations as of May 21, 2020

Page view(s)

Last Week
Last month
Citations as of May 6, 2020

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.