ABSTRACT

Automated Valuation Models (AVMs) based on Machine Learning (ML) algorithms are widely used for predicting house prices. While there is consensus in the literature that cross-validation (CV) should be used for model selection in this context, the interdisciplinary nature of the subject has made it hard to reach consensus over which metrics to use at each stage of the CV exercise. We collect 48 metrics (from the AVM literature and elsewhere) and classify them into seven groups according to their structure. Each of these groups focuses on a particular aspect of the error distribution. Depending on the type of data and the purpose of the AVM, the needs of users may be met by some classes, but not by others. In addition, we show in an empirical application how the choice of metric can influence the choice of model, by applying each metric to evaluate five commonly used AVM models. Finally – since it is not always practicable to produce 48 different performance metrics – we provide a short list of 7 metrics that are well suited to evaluate AVMs. These metrics satisfy a symmetry condition that we find is important for AVM performance, and can provide a good overall model performance ranking.

摘要

基于机器学习 (ML) 算法的自动估值模型 (AVM) 被广泛用于预测房价。虽然文献中一致认为在这种情况下应该使用交叉验证 (CV) 进行模型选择，但该主题的跨学科性质使得很难就在 CV 练习的每个阶段使用哪些指标达成共识。我们收集了 48 个指标（来自 AVM 文献和其他地方），并根据它们的结构将它们分为七组。这些组中的每一个都专注于错误分布的特定方面。根据数据的类型和 AVM 的用途，用户的需求可能会被某些类满足，而另一些则不能。此外，我们在实证应用中展示了度量的选择如何影响模型的选择，通过应用每个指标来评估五个常用的 AVM 模型。最后——因为生成 48 个不同的性能指标并不总是可行的——我们提供了一个非常适合评估 AVM 的 7 个指标的简短列表。这些指标满足我们发现对 AVM 性能很重要的对称条件，并且可以提供良好的整体模型性能排名。