Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/85628
Title: Towards self-tuning parameter servers
Authors: Liu, Chun Yin
Degree: M.Phil.
Issue Date: 2018
Abstract: Machine Learning (ML) has driven advances in many applications in recent years. Nowadays, it is common to see industrial-strength machine learning jobs that involve billions of model parameters, petabytes of training data, and weeks of training. Good efficiency, i.e., fast completion time of running a specific ML job, therefore, is a key feature of a successful ML system. While the completion time of a long-running ML job is determined by the time required to reach model convergence, practically that is largely influenced by the values of various system settings. In this thesis, we present techniques towards building self-tuning parameter servers. Parameter Server (PS) is a de-facto system architecture for large-scale machine learning; and by self-tuning we mean while a long-running ML job is iteratively training the expert-suggested model, the system is also iteratively learning which setting is more efficient for that job and applies it online. We have implemented our three techniques, namely, (1) online ML job optimization framework, (2) online ML job progress estimation, and (3) online ML system recon.guration, on top of TensorFlow. Experiments show that our techniques can reduce the completion times of long-running TensorFlow jobs from 1.7X to 5.1X.
Subjects: Hong Kong Polytechnic University -- Dissertations
Machine learning
Pages: xiv, 62 pages : color illustrations
Appears in Collections:Thesis

Show full item record

Page views

3
Citations as of May 22, 2022

Google ScholarTM

Check


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.