With the development of peta- and exascale size computational systems there is growing interest in running Big Data and Artificial Intelligence (AI) applications on them. Big Data and AI applications are implemented in Java, Scala, Python and other languages that are not widely used in High-Performance Computing (HPC) which is still dominated by C and Fortran. Moreover, they are based on dedicated environments such as Hadoop or Spark which are difficult to integrate with the traditional HPC management systems. We have developed the Parallel Computing in Java (PCJ) library, a tool for scalable high-performance computing and Big Data processing in Java. In this paper, we present the basic functionality of the PCJ library with examples of highly scalable applications running on the large resources. The performance results are presented for different classes of applications including traditional computational intensive (HPC) workloads (e.g. stencil), as well as communication-intensive algorithms such as Fast Fourier Transform (FFT). We present implementation details and performance results for Big Data type processing running on petascale size systems. The examples of large scale AI workloads parallelized using PCJ are presented.

随着PB级和百亿级规模计算系统的发展，人们对在其上运行大数据和人工智能（AI）应用程序的兴趣日益浓厚。大数据和AI应用程序以Java，Scala，Python和其他尚未在C和Fortran主导的高性能计算（HPC）中广泛使用的语言实现。此外，它们基于诸如Hadoop或Spark之类的专用环境，这些环境难以与传统的HPC管理系统集成。我们已经开发了Java并行计算（PCJ）库，该工具可用于以Java扩展可扩展的高性能计算和大数据处理。在本文中，我们以在大型资源上运行的高度可扩展应用程序为例，介绍了PCJ库的基本功能。针对不同类型的应用程序提供了性能结果，包括传统的计算密集型（HPC）工作负载（例如模板）以及通信密集型算法（例如快速傅立叶变换（FFT））。我们提供了在petascale大小的系统上运行的大数据类型处理的实现细节和性能结果。给出了使用PCJ并行化的大规模AI工作负载的示例。