Towards self-tuning parameter servers

Abstract

Machine Learning (ML) has driven advances in many applications in recent years. Nowadays, it is common to see industrial-strength machine learning jobs that involve billions of model parameters, petabytes of training data, and weeks of training. Good efficiency, i.e., fast completion time of running a specific ML job, therefore, is a key feature of a successful ML system. While the completion time of a long-running ML job is determined by the time required to reach model convergence, practically that is largely influenced by the values of various system settings. In this thesis, we present techniques towards building self-tuning parameter servers. Parameter Server (PS) is a de-facto system architecture for large-scale machine learning; and by self-tuning we mean while a long-running ML job is iteratively training the expert-suggested model, the system is also iteratively learning which setting is more efficient for that job and applies it online. We have implemented our three techniques, namely, (1) online ML job optimization framework, (2) online ML job progress estimation, and (3) online ML system recon.guration, on top of TensorFlow. Experiments show that our techniques can reduce the completion times of long-running TensorFlow jobs from 1.7X to 5.1X.