Abstract Advances in machine learning technologies have led to increasingly powerful models in particular in the context of big data. Yet, many application scenarios demand for robustly interpretable models rather than optimum model accuracy; as an example, this is the case if potential biomarkers or causal factors should be discovered based on a set of given measurements. In this contribution, we focus on feature selection paradigms, which enable us to uncover relevant factors of a given regularity based on a sparse model. We focus on the important specific setting of linear ordinal regression, i.e. data have to be ranked into one of a finite number of ordered categories by a linear projection. Unlike previous work, we consider the case that features are potentially redundant, such that no unique minimum set of relevant features exists. We aim for an identification of all strongly and all weakly relevant features as well as their type of relevance (strong or weak); we achieve this goal by determining feature relevance bounds, which correspond to the minimum and maximum feature relevance, respectively, if searched over all equivalent models. In addition, we discuss how this setting enables us to substitute some of the features, e.g. due to their semantics, and how to extend the framework of feature relevance intervals to the setting of privileged information, i.e. potentially relevant information is available for training purposes only, but cannot be used for the prediction itself.

中文翻译：

---

特征冗余和特权信息背景下有序回归的特征相关性确定

摘要 机器学习技术的进步导致了越来越强大的模型，特别是在大数据的背景下。然而，许多应用场景需要稳健可解释的模型，而不是最佳模型精度；例如，如果应根据一组给定的测量结果发现潜在的生物标志物或因果因素，就会出现这种情况。在这个贡献中，我们专注于特征选择范式，这使我们能够基于稀疏模型发现给定规律的相关因素。我们专注于线性有序回归的重要特定设置，即数据必须通过线性投影被归入有限数量的有序类别之一。与以前的工作不同，我们考虑特征可能是冗余的情况，因此不存在唯一的最小相关特征集。我们的目标是识别所有强相关和所有弱相关特征以及它们的相关类型（强或弱）；我们通过确定特征相关性边界来实现这一目标，如果在所有等效模型上搜索，这些边界分别对应于最小和最大特征相关性。此外，我们讨论了此设置如何使我们能够替换某些特征，例如由于它们的语义，以及如何将特征相关间隔的框架扩展到特权信息的设置，即潜在相关信息仅可用于训练目的，但不能用于预测本身。如果在所有等效模型上搜索，它们分别对应于最小和最大特征相关性。此外，我们讨论了此设置如何使我们能够替换某些特征，例如由于它们的语义，以及如何将特征相关间隔的框架扩展到特权信息的设置，即潜在相关信息仅可用于训练目的，但不能用于预测本身。如果在所有等效模型上搜索，它们分别对应于最小和最大特征相关性。此外，我们讨论了此设置如何使我们能够替换某些特征，例如由于它们的语义，以及如何将特征相关间隔的框架扩展到特权信息的设置，即潜在相关信息仅可用于训练目的，但不能用于预测本身。