Statistics and Its Interface

Volume 9 (2016)

Number 4

Special Issue on Statistical and Computational Theory and Methodology for Big Data

Guest Editors: Ming-Hui Chen (University of Connecticut); Radu V. Craiu (University of Toronto); Faming Liang (University of Florida); and Chuanhai Liu (Purdue University)

Direct regression modelling of high-order moments in big data

Pages: 445 – 452

DOI: https://dx.doi.org/10.4310/SII.2016.v9.n4.a4

Authors

Ruibin Xi (School of Mathematical Sciences and Center of Statistical Science, Peking University, Beijing, China)

Nan Lin (Department of Mathematics, Washington University in St. Louis, Missouri, U.S.A.)

Abstract

Big data problems present great challenges to statistical analyses, especially from the computational side. In this paper, we consider regression estimation of high-order moments in big data problems based on the U-statistic-based Functional Regression Model (U-FRM) model. The U-FRM model is a nonparametric method that allows direct estimation of higher-order moments without imposing parametric assumptions on the high order-moments. Despite this modeling advantage, its estimation relies on a U-statistics-based estimating equation whose computational complexity is generally too high for big data. In this paper, we propose using the “divide-and-conquer” strategy to construct a computationally more succinct surrogate estimating equation. Through both theoretical proof and simulations, we show that our method significantly reduces the computational time and meanwhile enjoys the same asymptotic behavior as the original estimation method.We then apply our method to a genomic problem to illustrate its performance on real data.

Keywords

big data, higher-order moment, U-statistics, estimating equation, divide-and-conquer, aggregation, consistency, asymptotic normality, data cube

Full Text (PDF format)

Published 14 September 2016