Brittleness of Bayesian inference and new Selberg formulas

The incorporation of priors [H. Owhadi, C. Scovel, and T.J. Sullivan, Electronic H. Stat., 2013] in the Optimal Uncertainty Quantification (OUQ) framework [H. Owhadi, C. Scovel, T.J. Sullivan, M. McKerns, and M. Ortiz, SIAM Rev., 2013] reveals brittleness in Bayesian inference; a model may share an arbitrarily large number of finite-dimensional marginals with, or be arbitrarily close (in Prokhorov or total variation metrics) to, the data-generating distribution and still make the largest possible prediction error after conditioning on an arbitrarily large number of samples. The initial purpose of this paper is to unwrap this brittleness mechanism by providing (i) a quantitative version of the Brittleness Theorem of [H. Owhadi, C. Scovel, and T.J. Sullivan, Electronic H. Stat., 2013] and (ii) a detailed and comprehensive analysis of its application to the revealing example of estimating the mean of a random variable on the unit interval [0,1] using priors that exactly capture the distribution of an arbitrarily large number of Hausdorff moments.

However, in doing so, we discovered that the free parameter associated with Markov and Kreĭn’s canonical representations of truncated Hausdorff moments generates reproducing kernel identities corresponding to reproducing kernel Hilbert spaces of polynomials. Furthermore, these reproducing identities lead to biorthogonal systems of Selberg integral formulas.

This process of discovery appears to be generic: whereas Karlin and Shapley used Selberg’s integral formula to first compute the volume of the Hausdorff moment space (the polytope defined by the first n moments of a probability measure on the interval [0,1]), we observe that the computation of that volume along with higher order moments of the uniform measure on the moment space, using different finite-dimensional representations of subsets of the infinite-dimensional set of probability measures on [0,1] representing the first $n$ moments, leads to families of equalities corresponding to classical and new Selberg identities.

Keywords

Bayesian inference, misspecification, robustness, uncertainty quantification, optimal uncertainty quantification, reproducing kernel Hilbert spaces (RKHS), Selberg integral formulas

2010 Mathematics Subject Classification

11M35, 46E22, 62A01, 62F12, 62F15, 62G20, 62G35

Full Text (PDF format)

Published 16 September 2015