To illustrate an unprejudiced comparison among machine learning classifiers established on proprietary databases, and to guarantee the validity and robustness of these classifiers, a Performance Evaluation Indicator (PEI) and the corresponding failure criterion are proposed in this study. Three types of machine learning classifiers, including the strictly binary classifier, the normal multiclass classifier and the misclassification cost-sensitive classifier, are trained on four datasets recorded from a water drainage TBM project. The results indicate that: (1) the PEI successfully compares the competence of classifiers under different scenarios by isolating the effects of different overlapping-degree of rockmass classes, and (2) the cost-sensitive algorithm is warranted to classify rockmasses when the ratio of inter-class classes is more than 8:1. The contributions of this research are to fill the gap in performance evaluations of a classifier for imbalanced training data, and to identify the best situation to apply this classifier.

为了说明在专有数据库上建立的机器学习分类器之间的无偏见比较，并确保这些分类器的有效性和鲁棒性，本研究提出了一种性能评估指标（PEI）和相应的故障准则。在从排水TBM项目记录的四个数据集上训练了三种类型的机器学习分类器，包括严格的二进制分类器，常规的多分类器分类和对成本敏感的分类错误的分类器。结果表明：（1）PEI通过隔离岩体类别的不同重叠度的影响，成功地比较了不同场景下分类器的能力；（2）当比率为班级间的班级比例大于8：1。