Demystify Lindley’s paradox by connecting $p$-value and posterior probability

In the hypothesis testing framework, $p$‑value is often computed to determine whether to reject the null hypothesis or not. On the other hand, Bayesian approaches typically compute the posterior probability of the null hypothesis to evaluate its plausibility. We revisit Lindley’s paradox and demystify the conflicting results between Bayesian and frequentist hypothesis testing procedures by casting a two-sided hypothesis as a combination of two one-sided hypotheses along the opposite directions. This formulation can naturally circumvent the ambiguities of assigning a point mass to the null and choices of using local or non-local prior distributions. As $p$‑value solely depends on the observed data without incorporating any prior information, we consider non-informative prior distributions for fair comparisons with $p$‑value. The equivalence of $p$‑value and the Bayesian posterior probability of the null hypothesis can be established to reconcile Lindley’s paradox. More complicated settings, such as multivariate cases, random effects models and non-normal data, are also explored for generalization of our results to various hypothesis tests.

Keywords

Bayesian posterior probability, hypothesis testing, Interpretation of $p$-value, point null hypothesis, two-sided test

2010 Mathematics Subject Classification

Primary 62A01, 62F15. Secondary 62F03.

Full Text (PDF format)

The authors’ research was supported by a grant (106200216) from the Research Grants Council of Hong Kong.

Received 11 December 2020

Accepted 9 March 2021

Published 8 July 2021