BACKGROUND Several prediction problems in computational biology and genomic medicine are characterized by both big data as well as a high imbalance between examples to be learned, whereby positive examples can represent a tiny minority with respect to negative examples. For instance, deleterious or pathogenic variants are overwhelmed by the sea of neutral variants in the non-coding regions of the genome: thus, the prediction of deleterious variants is a challenging, highly imbalanced classification problem, and classical prediction tools fail to detect the rare pathogenic examples among the huge amount of neutral variants or undergo severe restrictions in managing big genomic data. RESULTS To overcome these limitations we propose parSMURF, a method that adopts a hyper-ensemble approach and oversampling and undersampling techniques to deal with imbalanced data, and parallel computational techniques to both manage big genomic data and substantially speed up the computation. The synergy between Bayesian optimization techniques and the parallel nature of parSMURF enables efficient and user-friendly automatic tuning of the hyper-parameters of the algorithm, and allows specific learning problems in genomic medicine to be easily fit. Moreover, by using MPI parallel and machine learning ensemble techniques, parSMURF can manage big data by partitioning them across the nodes of a high-performance computing cluster. Results with synthetic data and with single-nucleotide variants associated with Mendelian diseases and with genome-wide association study hits in the non-coding regions of the human genome, involhing millions of examples, show that parSMURF achieves state-of-the-art results and an 80-fold speed-up with respect to the sequential version. CONCLUSIONS parSMURF is a parallel machine learning tool that can be trained to learn different genomic problems, and its multiple levels of parallelization and high scalability allow us to efficiently fit problems characterized by big and imbalanced genomic data. The C++ OpenMP multi-core version tailored to a single workstation and the C++ MPI/OpenMP hybrid multi-core and multi-node parSMURF version tailored to a High Performance Computing cluster are both available at https://github.com/AnacletoLAB/parSMURF.

中文翻译：

---

parSMURF，一种用于全基因组检测致病变异的高性能计算工具。

背景技术计算生物学和基因组医学中的几个预测问题的特点是大数据以及要学习的示例之间的高度不平衡，从而正例相对于负例只能代表极少数。例如，有害或致病变异被基因组非编码区域中的大量中性变异所淹没：因此，有害变异的预测是一个具有挑战性的、高度不平衡的分类问题，而经典的预测工具无法检测到罕见的变异。大量中性变异中的致病实例或在管理大基因组数据方面受到严格限制。结果为了克服这些限制，我们提出了 parSMURF，一种采用超集成方法和过采样和欠采样技术来处理不平衡数据的方法，以及采用并行计算技术来管理大基因组数据并显着加快计算速度的方法。贝叶斯优化技术与 parSMURF 的并行特性之间的协同作用能够高效且用户友好地自动调整算法的超参数，并允许轻松解决基因组医学中的特定学习问题。此外，通过使用 MPI 并行和机器学习集成技术，parSMURF 可以通过跨高性能计算集群的节点对大数据进行分区来管理大数据。合成数据和与孟德尔疾病相关的单核苷酸变异以及人类基因组非编码区域中全基因组关联研究的结果，涉及数百万个例子，表明 parSMURF 取得了最先进的结果并且相对于顺序版本的速度提高了 80 倍。结论 parSMURF 是一种并行机器学习工具，可以通过训练来学习不同的基因组问题，它的多级并行化和高可扩展性使我们能够有效地拟合以庞大且不平衡的基因组数据为特征的问题。为单个工作站量身定制的 C++ OpenMP 多核版本和为高性能计算集群量身定制的 C++ MPI/OpenMP 混合多核和多节点 parSMURF 版本均可在 https://github.com/AnacletoLAB/parSMURF 上获得.