A new model validation and performance assessment tool is introduced, the sliding threshold of observation for numeric evaluation (STONE) curve. It is based on the relative operating characteristic (ROC) curve technique, but instead of sorting all observations in a categorical classification, the STONE tool uses the continuous nature of the observations. Rather than defining events in the observations and then sliding the threshold only in the classifier/model data set, the threshold is changed simultaneously for both the observational and model values, with the same threshold value for both data and model. This is only possible if the observations are continuous and the model output is in the same units and scale as the observations, that is, the model is trying to exactly reproduce the data. The STONE curve has several similarities with the ROC curve—plotting probability of detection against probability of false detection, ranging from the (1,1) corner for low thresholds to the (0,0) corner for high thresholds, and values above the zero‐intercept unity‐slope line indicating better than random predictive ability. The main difference is that the STONE curve can be nonmonotonic, doubling back in both the x and y directions. These ripples reveal asymmetries in the data‐model value pairs. This new technique is applied to modeling output of a common geomagnetic activity index as well as energetic electron fluxes in the Earth's inner magnetosphere. It is not limited to space physics applications but can be used for any scientific or engineering field where numerical models are used to reproduce observations.

中文翻译：

---

STONE 曲线：ROC 衍生模型性能评估工具


引入了一种新的模型验证和性能评估工具，即数值评估观察滑动阈值（STONE）曲线。它基于相对操作特征 (ROC) 曲线技术，但 STONE 工具使用观测值的连续性质，而不是按类别分类对所有观测值进行排序。不是在观测值中定义事件，然后仅在分类器/模型数据集中滑动阈值，而是同时更改观测值和模型值的阈值，数据和模型具有相同的阈值。只有当观测值是连续的并且模型输出与观测值具有相同的单位和比例时，这才有可能，也就是说，模型试图精确地再现数据。 STONE 曲线与 ROC 曲线有一些相似之处 - 绘制检测概率与错误检测概率的关系，范围从低阈值的 (1,1) 角到高阈值的 (0,0) 角，以及高于零的值‐截距统一斜率线表明优于随机预测能力。主要区别在于 STONE 曲线可以是非单调的，在x和y方向上加倍。这些涟漪揭示了数据模型值对中的不对称性。这项新技术适用于对常见地磁活动指数的输出以及地球内部磁层中的高能电子通量进行建模。它不仅限于空间物理应用，还可以用于使用数值模型重现观测结果的任何科学或工程领域。