Forecast evaluations aim to choose an accurate forecast for making decisions by using loss functions. However, different loss functions often generate different ranking results for forecasts, which complicates the task of comparisons. In this paper, we develop statistical tests for comparing performances of forecasting expectiles and quantiles of a random variable under consistent loss functions. The test statistics are constructed with the extremal consistent loss functions of Ehm et al. (2016). The null hypothesis of the tests is that a benchmark forecast at least performs equally well as a competing one under all extremal consistent loss functions. It can be shown that if such a null holds, the benchmark will also perform at least equally well as the competitor under all consistent loss functions. Thus under the null, when different consistent loss functions are used, the result that the competitor does not outperform the benchmark will not be altered. We establish asymptotic properties of the proposed test statistics and propose to use the re-centered bootstrap to construct their empirical distributions. Through simulations, we show that the proposed test statistics perform reasonably well. We then apply the proposed method to evaluations of several different forecast methods.

预测评估旨在通过使用损失函数来选择准确的预测以做出决策。但是，不同的损失函数通常会为预测生成不同的排名结果，这使比较的任务变得复杂。在本文中，我们开发了统计测试，用于比较一致损失函数下随机变量的预测预期指标和分位数的性能。利用Ehm等人的极值一致性损失函数构造检验统计量。（2016）。检验的零假设是，在所有极端一致的损失函数下，基准预测至少具有与竞争性预测相同的性能。可以证明，如果保持这样的零值，则在所有一致的损失函数下，基准测试的表现至少也将与竞争对手相同。因此，在null下 当使用不同的一致损失函数时，竞争对手未超过基准的结果将不会改变。我们建立拟议的测试统计量的渐近性质，并建议使用重新定中心的引导程序来构造其经验分布。通过仿真，我们表明所提出的测试统计数据表现合理。然后，我们将提出的方法应用于几种不同预测方法的评估。