Statistics and Its Interface

Volume 9 (2016)

Number 4

Special Issue on Statistical and Computational Theory and Methodology for Big Data

Guest Editors: Ming-Hui Chen (University of Connecticut); Radu V. Craiu (University of Toronto); Faming Liang (University of Florida); and Chuanhai Liu (Purdue University)

A split-and-merge approach for singular value decomposition of large-scale matrices

Pages: 453 – 459

DOI: https://dx.doi.org/10.4310/SII.2016.v9.n4.a5

Authors

Faming Liang (Department of Biostatistics, University of Florida, Gainesville, Fl., U.S.A.)

Runmin Shi (Department of Statistics, University of Florida, Gainesville, Fl., U.S.A.)

Qianxing Mo (Department of Medicine and Dan L. Duncan Cancer Center, Baylor College of Medicine, Houston, Texas, U.S.A.)

Abstract

We propose a new SVD algorithm based on the split-and-merge strategy, which possesses an embarrassingly parallel structure and thus can be efficiently implemented on a distributed or multicore machine. The new algorithm can also be implemented in serial for online eigen-analysis. The new algorithm is particularly suitable for big data problems: Its embarrassingly parallel structure renders it usable for feature screening, while this has been beyond the ability of the existing parallel SVD algorithms.

Keywords

feature screening, parallel computation, online eigen-learning, singular value decomposition

Full Text (PDF format)

Published 14 September 2016