This paper investigates the problem of extracting actionable patterns/models from unlabeled and potentially erroneous datasets in an unsupervised way. To address the need for both model extraction and data reliability evaluation, we propose a novel iterative multilayer micro-macro (IM3) method that defines data reliability, learns micro-macro models, and iteratively refines learned models. The IM3 method includes a general data reliability definition to evaluate the reliability level of each sample, a micro-macro model complexity determination, and an iterative data reliability and model complexity update mechanism to overcome the underfitting and overfitting issue. In particular, we propose a consistency-index-based approach to address underfitting and overfitting in an unsupervised way. The refinement of the learned models is enabled via dropping the most unreliable data until the data reliability is above a given threshold. The sensitivity of the proposed IM3 method with respect to the reliability threshold selection is further quantified via false alarm and missdetection to facilitate the selection of an appropriate reliability threshold. Evaluation of the proposed method and quantitative analysis of its sensitivity are provided on a polynomial regression problem via Monte Carlo simulations.

中文翻译：

---

用于传感数据可靠性评估的迭代多层无监督学习方法


本文研究了以无监督的方式从未标记的和可能错误的数据集中提取可操作的模式/模型的问题。为了满足模型提取和数据可靠性评估的需求，我们提出了一种新颖的迭代多层微观宏观（IM3）方法，该方法定义数据可靠性、学习微观宏观模型并迭代细化学习模型。 IM3方法包括用于评估每个样本的可靠性水平的通用数据可靠性定义、微观-宏观模型复杂性确定以及用于克服欠拟合和过拟合问题的迭代数据可靠性和模型复杂性更新机制。特别是，我们提出了一种基于一致性指数的方法，以无监督的方式解决欠拟合和过拟合问题。通过丢弃最不可靠的数据，直到数据可靠性高于给定阈值，可以对学习模型进行细化。所提出的 IM3 方法相对于可靠性阈值选择的敏感性通过误报和漏检进一步量化，以方便选择适当的可靠性阈值。通过蒙特卡罗模拟对多项式回归问题进行了对所提出方法的评估及其敏感性的定量分析。