Distributed iteratively reweighted least squares and applications

The iteratively reweighted least squares (IRLS) method has been one of the most used methods in statistics estimation. From maximum likelihood estimation for various models, to general estimating equations with longitudinal data, to robust regression, and to general nonlinear parameter estimation, IRLS has been popularly used to find solutions. However, for very large data, the iterative style could be computationally expensive. We propose a distributed version of IRLS, which can be used on a cluster of computers/threads networked with high speed communications. We explore applications of the distributed IRLS on various estimation problems. Using message passing interface (MPI), we implemented the distributed IRLS for robust regression on a cluster with 41 threads. Experiments from the implementation show that the distributed IRLS can solve very large problems more efficiently and economically compared to the classical non-distributed IRLS.

Keywords

iteratively reweighted least squares, maximum likelihood estimation, message passing interface, quasi-likelihood estimation, robust regression

2010 Mathematics Subject Classification

60K35

Full Text (PDF format)

Published 10 January 2014