Abstract
This paper proposes an econometric method to construct confidence sets for rankings by considering the situations in which data on a comparison between each pair of alternatives are available. The key idea is that if an alternative, say A, is ranked higher than some other alternative, say B, then it would be natural to assume that drawing an observation in which A is compared favorably with B is more likely than the opposite. This provides an insight that we can use to test for a particular ranking by translating it into a problem of testing moment inequalities and that we can obtain confidence sets for rankings by inverting these tests of moment inequalities. We apply the proposed method to statistical inference concerning the rankings of teams in the Nippon Professional Baseball league in Japan.
Similar content being viewed by others
Notes
See, for the example, “University Rankings Go Global” New York Times, April 14, 2014, https://www.nytimes.com/2013/04/14/education/edlife/university-rankings-go-global.html.
If a sample of individual rankings is available, \(D_{ij}\) can be constructed in the following way. For each individual ranking, let \(D_{ij}=1\) if i is ranked higher than j in that ranking. For example, university rankings are typically constructed by aggregating various indexes which may be considered as individual rankings. We can thus convert the data on indexes into data on pairwise comparisons between two universities. Namely, if university i ranks higher than j in the nth index, we regard that the nth observation of \(D_{ij}\) is one.
Strictly speaking, a confidence set can be empty. However, the probability of the null set, \(\emptyset\), including the true ranking, is obviously zero. The proposed procedure never yields \(\emptyset\) as a confidence set in cases with two alternatives. If there are more than three alternatives, and transitivity is imposed, it is possible to obtain an empty confidence set (in such a case, transitivity is rejected). If willing to drop transitivity as an assumption, the proposed method never yields an empty confidence set.
If correlation exists among observations, the t test statistic should be modified such that the denominator is a correlation-robust standard error. The remainder of the procedure does not need alternation.
As explained in footnote 3, if a sample of individual rankings is available, it can be converted into a sample of pairwise comparisons. The number of indexes corresponds to \(N=N_{ij}\) in our setting. If index n ranks alternative i over j, then we set \(D_{nij}=1\). We then compute \({\hat{p}}_{ij} = \sum _{n=1}^N D_{nij} /N\). \({\hat{p}}_{ij}\) is the fraction of indexes that rank i higher than j.
See, e.g., “The 27th survey of popular sports” conducted by Chuo Chousa-sha in April 2017 available at https://www.crs.or.jp/data/pdf/sports19.pdf.
See, e.g., “Softbank nihon-ichi nara, keizaikouka 313 oku yen (Softbank’s winning the Japan series would yield an economic impact of 31,300,000,000 JPY),” Nishinippon Shimbun, October 28, 2018. https://www.nishinippon.co.jp/item/n/460934/. Headline translation by the author. The Japan Series is the most prestigious league in the NPB, where after a regular season each league holds a final series, known in Japan as the Climax Series, played by the three highest-ranked teams. The winners of the Climax Series from each league then play the Japan Series.
References
Alvo, M., & Yu, P. L. H. (2014). Statistical methods for ranking data. Berlin: Springer.
Bontemps, C., & Magnac, T. (2017). Set identification, moment restrictions, and inference. Annual Review of Economics, 9, 103–129.
Canay, Ivan A., & Shaikh, Azeem M. (2017). Practical and theoretical advances in inference for partially identified models. In B. Honoré, A. Pakes, M. Piazzesi, & L. Samuelson (Eds.), Advances in economics and econometrics: Eleventh world congress. Econometric society monographs (Vol. 2, pp. 271–306). Cambridge: Cambridge University Press. https://doi.org/10.1017/9781108227223.009.
Chernozhukov, V., Chetverikov, D., & Kato, K. (2019). Inference on causal and structural parameters using many moment inequalities. The Review of Economic Studies, 86(5), 1867–1900.
Ciliberto, F., & Tamer, E. (2009). Market structure and multiple equilibria in airline markets. Econometrica, 77(6), 1791–1828.
Dzemski, A., & Okui, R. (2020). Confidence set for group membership. mimeo. Available at https://arxiv.org/abs/1801.00332
Hall, P., & Miller, H. (2010). Modeling the variability of rankings. The Annals of Statistics, 38, 2652–2677.
Hall, P., & Schimek, M. G. (2012). Moderate-deviation-based inference for random degeneration in paired rank lists. Journal of the American Statistical Association, 107(498), 661–672.
Jin, Y., & Okui, R. (2020). Testing for overconfidence statistically: A moment inequality approach. Journal of Applied Econometrics (forthcoming). https://doi.org/10.1002/jae.2780
Kemeny, J. G. (1959). Mathematics without numbers. Daedalus, 88(4), 577–591.
Molinari, F. (2020). Microeconometrics with partial identification. In: Hand-book of econometrics, vol. 7A. Elsevier (forthcoming). Available at https://arxiv.org/abs/2004.11751
Romano, J. P., Shaikh, A. M., & Wolf, M. (2014). A practical two-step method for testing moment inequalities. Econometrica, 82(5), 1979–2002.
Sakai, T. (2013). Syakai Sentaku Riron heno Syoutai (Invitation to social choice theory). in Japanese. Nihon Hyohron Sha.
Stock, J. H., & Watson, M. W. (2001). Vector autoregressions. The Journal of Economic Perspectives, 15(4), 101–115.
Young, H. P. (1988). Condorcet’s theory of voting. American Political science review, 82(4), 1231–1244.
Acknowledgements
The author would like to thank Hiroshi Teruyama, an anonymous referee, Mototsugu Shintani, Chiaki Hara, Yukihiko Funaki, Naoya Sueishi, Maki Michinaka and participants at the 2020 Spring Meeting of the Japanese Economic Association for their helpful comments. Thanks also to KIM Hyeongyu for excellent research assistance with data collection and cleaning. The author gratefully acknowledges the financial support of the School of Social Sciences and a New Faculty Startup Grant at Seoul National University and from the Housing and Commercial Bank Economic Research Fund in the Institute of Economic Research at Seoul National University. The usual disclaimer applies.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A: Proof of Theorem 2
Appendix A: Proof of Theorem 2
The proof closely follows Chernozhukov et al. (2019, Proof of Theorem 4.2). Let \(c (x) = \Phi ^{-1} (1-x)\). Note that \(c_{\beta } = c(\beta / S_N)\). Define \({\hat{J}} = \{(i,j) \in \succ _T : t_{ij} \ge - 2c (\beta /S_N ) \}\) and
Recall that \(E (A_{nij}) = 0.5-p_{ij}\) and the variance of \(A_{nij}\) is \(p_{ij}(1-p_{ij})\). Note that \(\succ _T\) is included in the confidence set with moment selection if \(\max _{(i,j) \in \succ _T} t_{ij} < c( (\alpha -2\beta )/ |\hat{J} | )\). The proof proceeds with three steps.
First, we show that
The key observation is that \(\sum _{n=1}^{N_{ij}} A_{nij} > 0\) for some \((i,j) \in \succ _T \backslash J\) implies that
It holds by the Bonferroni inequality and the central limit theorem that
This completes the first step.
Second, we show that \(\lim _{N_{ij} \rightarrow \infty \text { for all } (i,j)} \Pr \left( {\hat{J}} \supset J \right) \ge 1- \beta\). Note that
where \({\hat{\sigma }}_{ij}^2 = \sum _{n=1}^{N_{ij}} (A_{nij} - {\bar{A}}_{ij})^2 / N_{ij}\) and \(\sigma _{ij} = p_{ij} (1- p_{ij})\). If \((1 - \sigma _{ij}/ {\hat{\sigma }}_{ij} ) \ge -r\) for some \(0< r < 1\), it holds
Thus, it follows that
for all \(0< r <1\). We take \(r = \min _{(i,j) \in \succ _T} N_{ij}^{-1/3}\) so that the second term on the right hand side of the above inequality vanishes. By the Bonferroni inequality and the standard asymptotic argument for t statistic, we have
We thus complete the second step.
Lastly, we show the result of the theorem. Note that when \(|J|=0\), the limit of \(\Pr ( \succ _T \in C_{\alpha }^M)\) (\(\ge \Pr ( \max _{(i,j) \in \succ _T} t_{ij} <0)\)) is more than \(1-\alpha\) as shown in the first step and \(\beta < \alpha\) by definition. We consider cases with \(|J| \ge 1\). We observe that
It also holds that
because \(|J| < S_R\) if \({\hat{J}} \supset J\). Using the results obtained in the first and second steps yields
Rights and permissions
About this article
Cite this article
Okui, R. A moment inequality approach to statistical inference for rankings. JER 72, 169–184 (2021). https://doi.org/10.1007/s42973-020-00068-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42973-020-00068-2