Abstract
In one-sided testing, Bayesians and frequentists differ on whether or not there is discrepancy between the inference based on the posterior model probability and that based on the p value. We add some arguments to this debate analyzing the discrepancy for moderate and large sample sizes. For small and moderate samples sizes, the discrepancy is measured by the probability of disagreement. Examples of the discrepancy on some basic sampling models indicate the somewhat unexpected result that the probability of disagreement is larger when sampling from models in the alternative hypothesis that are not located at the boundary of the hypotheses. For large sample sizes, we prove that the Bayesian one-sided testing is, under mild conditions, consistent, a property that is not shared by the frequentist procedure. Further, the rate of convergence is \(O(e^{nA})\), where A is a constant that depends on the model from which we are sampling. Consistency is also proved for an extension to multiple hypotheses.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Berger JO (2003) Could Fisher, Jeffreys and Neyman have agreed on testing? Stat Sci 18:1–32
Berger JO, Mortera J (1999) Default Bayes factors for one-sided hypothesis testing. J Am Stat Assoc 94:542–554
Berger JO, Pericchi LR (1996) The intrinsic Bayes factor for linear models (with discussion). In: Bernardo JM et al (eds) Bayesian statistics, vol 5. Oxford University Press, New York, pp 25–44
Berger JO, Sellke T (1987) Testing a point null hypothesis: the irreconcilability of p values and evidence. J Am Stat Assoc 82:112–122
Casella G, Berger R (1987) Reconciling Bayesian and frequentist evidence in the one-sided testing problem. J Am Stat Assoc 82:106–111
Casella G, Girón FJ, Martínez ML, Moreno E (2009) Consistency of Bayesian procedures for variable selection. Ann Stat 37:1207–1228
Casella G, Moreno E, Girón FJ (2014) Cluster analysis, model selection, and prior distributions on models, Bayesian. Analysis 9:613–658
Dickey JM (1977) Is the tail area useful as an approximate Bayes factor? J Am Stat Assoc 72:138–142
Dudley RM, Haughton D (1997) Information criteria for multiple data sets and restricted parameters. Stat Sin 7:265–284
Edwards W, Lindman H, Savage L (1963) Bayesian statistical inference for psychological research. Psychol Rev 70:193–242
Efron B, Gous A (2001) Scales of evidence for model selection: Fisher versus Jeffreys. In: Lahiri P (ed) Lecture notes-monograph series, vol 38. Institute of Mathematical Statistics, Hayward, pp 208–246
Girón FJ, Martínez ML, Moreno E, Torres F (2006) Objective testing procedures in linear models: calibration of the p-values. Scand J Stat 33:765–784
Kass RE, Vaidyanathan SK (1992) Approximate Bayes factors and orthogonal parameters. With application to testing equality of two binomial proportions. J R Stat Soc Ser B 54:129–144
Micheas AC, Dey DK (2003) Prior and posterior predictive p-values in the one-side location parameter testing problem. Sankhya Ser A 65:158–178
Moreno E (1997) Bayes factors for intrinsic and fractional priors in nested models, Bayesian robustness. In: Dodge Y (ed) L\(_{1}\) Statistical procedures and related topics, vol 31 of lecture notes-monograph series. Institute of Mathematical Statistics, Hayward, CA, pp 257–270
Moreno E (2005) Objective Bayesian analysis for one-sided testing. Test 14:181–198
Moreno E, Girón FJ (2005) Consistency of Bayes factors for linear models. C R Acad Sci Paris Ser I 340:911–914
Moreno E, Bertolino F, Racugno W (1998) An intrinsic limiting procedure for model selection and hypothesis testing. J Am Stat Assoc 93:1451–1460
Moreno E, Girón FJ, Casella G (2010) Consistency of objective Bayes factors as the model dimension grows. Ann Stat 38:1937–1952
Moreno E, Girón FJ, Casella G (2015) Posterior model consistency in variable selection as the model dimension grows. Stat Sci 30:228–241
Morris C (1987) Comment to Casella and Berger (1987). J Am Stat Assoc 82:112–139
Mulder J (2014) Prior adjusted default Bayes factors for testing (in) equality constrained hypotheses. Comput Stat Data Anal 71:448–463
Mulder J, Raftery AE (2019) BIC extensions for order-constrained model selection. Sociol Methods Res. https://doi.org/10.1177/0049124119882459
Wang M, Maruyama Y (2016) Consistency of Bayes factor for nonnested model selection when the model dimension grows. Bernoulli 22:2080–2100
Wilks SS (1963) Mathematical statistics. Wiley, New York
Acknowledgements
The first author of this research was supported by the Junta de Andalucía Grant A-FQM-456-UGR18. We are grateful to two anonymous Referees for their comments that lead to improve the presentation of the article.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Moreno, E., Martínez, C. Bayesian and frequentist evidence in one-sided hypothesis testing. TEST 31, 278–297 (2022). https://doi.org/10.1007/s11749-021-00778-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11749-021-00778-8