Numerous Bayesian Network (BN) structure learning algorithms have been proposed in the literature over the past few decades. Each publication makes an empirical or theoretical case for the algorithm proposed in that publication and results across studies are often inconsistent in their claims about which algorithm is ‘best’. This is partly because there is no agreed evaluation approach to determine their effectiveness. Moreover, each algorithm is based on a set of assumptions, such as complete data and causal sufficiency, and tend to be evaluated with data that conforms to these assumptions, however unrealistic these assumptions may be in the real world. As a result, it is widely accepted that synthetic performance overestimates real performance, although to what degree this may happen remains unknown. This paper investigates the performance of 15 state-of-the-art, well-established, or recent promising structure learning algorithms. We propose a methodology that applies the algorithms to data that incorporates synthetic noise, in an effort to better understand the performance of structure learning algorithms when applied to real data. Each algorithm is tested over multiple case studies, sample sizes, types of noise, and assessed with multiple evaluation criteria. This work involved learning approximately 10,000 graphs with a total structure learning runtime of seven months. In investigating the impact of data noise, we provide the first large scale empirical comparison of BN structure learning algorithms under different assumptions of data noise. The results suggest that traditional synthetic performance may overestimate real-world performance by anywhere between 10% and more than 50%. They also show that while score-based learning is generally superior to constraint-based learning, a higher fitting score does not necessarily imply a more accurate causal graph. The comparisons extend to other outcomes of interest, such as runtime, reliability, and resilience to noise, assessed over both small and large networks, and with both limited and big data. To facilitate comparisons with future studies, we have made all data, raw results, graphs and BN models freely available online.

在过去的几十年中，已经提出了许多贝叶斯网络（BN）结构学习算法。每份出版物都对该出版物中提出的算法提出了经验或理论依据，并且跨研究的结果在关于哪种算法“最佳”的主张中往往不一致。部分原因是因为没有商定的评估方法来确定其有效性。此外，每种算法都基于一组假设，例如完整的数据和因果关系，并且倾向于使用符合这些假设的数据进行评估，但是这些假设在现实世界中可能是不现实的。结果，尽管性能尚不清楚，但综合性能高估了实际性能已被广泛接受。本文研究了15种最先进的，成熟的或最近很有希望的结构学习算法的性能。我们提出了一种将算法应用于包含合成噪声的数据的方法，以更好地理解应用于实际数据的结构学习算法的性能。每种算法均经过多个案例研究，样本量，噪声类型测试，并使用多种评估标准进行评估。这项工作涉及学习大约10,000个图，整个结构学习时间为七个月。在研究数据噪声的影响时，我们提供了在不同数据噪声假设下BN结构学习算法的首次大规模经验比较。结果表明，传统的合成性能可能会高估真实世界的性能10％到50％以上。他们还表明，虽然基于分数的学习通常要优于基于约束的学习，但是较高的拟合分数并不一定意味着更准确的因果图。这些比较可扩展到其他有意义的结果，例如运行时，可靠性和对噪声的适应性，可在小型和大型网络上进行评估，并在有限和大型数据下进行评估。为了便于与将来的研究进行比较，我们已在线免费提供了所有数据，原始结果，图表和BN模型。这些比较可扩展到其他有意义的结果，例如运行时，可靠性和对噪声的适应性，可在小型和大型网络上进行评估，并在有限和大型数据下进行评估。为了便于与将来的研究进行比较，我们已在线免费提供了所有数据，原始结果，图表和BN模型。这些比较可扩展到其他有意义的结果，例如运行时，可靠性和对噪声的适应性，可在小型和大型网络上进行评估，并在有限和大型数据下进行评估。为了便于与将来的研究进行比较，我们已在线免费提供了所有数据，原始结果，图表和BN模型。