Human-in-the-loop machine learning is widely used in artificial intelligence (AI) to elicit labels for data points from experts or to provide feedback on how close the predicted results are to the target. This simplifies away all the details of the decision-making process of the expert. In this work, we allow the experts to additionally produce decision rules describing their decision-making; the rules are expected to be imperfect but to give additional information. In particular, the rules can extend to new distributions, and hence enable significantly improving performance for cases where the training and testing distributions differ, such as in domain adaptation. We apply the proposed method to lifelong learning and domain adaptation problems and discuss applications in other branches of AI, such as knowledge acquisition problems in expert systems. In simulated and real-user studies, we show that decision rule elicitation improves domain adaptation of the algorithm and helps to propagate expert's knowledge to the AI model.

中文翻译：

---

领域适应的决策规则启发

环人机器学习已在人工智能（AI）中广泛使用，以从专家那里获取数据点的标签，或者提供有关预测结果与目标的接近程度的反馈。这简化了专家决策过程的所有细节。在这项工作中，我们允许专家额外制定描述其决策的决策规则；该规则可能不完善，但会提供更多信息。尤其是，规则可以扩展到新的分布，因此可以显着提高训练和测试分布不同的情况下的性能，例如域适应。我们将提出的方法应用于终身学习和领域适应问题，并讨论了在AI其他分支中的应用，例如专家系统中的知识获取问题。在模拟和实际用户研究中，我们表明决策规则启发可改善算法的领域适应性，并有助于将专家的知识传播到AI模型。