当前位置: X-MOL 学术Res. Astron. Astrophys. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Fine-grained distributed averaging for large-scale radio interferometric measurement sets
Research in Astronomy and Astrophysics ( IF 1.8 ) Pub Date : 2021-05-20 , DOI: 10.1088/1674-4527/21/4/80
Shou-Lin Wei 1 , Kai-Da Luo 1 , Feng Wang 1, 2 , Hui Deng 2 , Ying Mei 2
Affiliation  

The Square Kilometre Array (SKA) would be the world’s largest radio telescope with eventually over a square kilometre of collecting area. However, there are enormous challenges in its data processing. The use of modern distributed computing techniques to solve the problem of massive data processing in the SKA is one of the most important challenges. In this study, basing on the Dask distribution computational framework, and taking the visibility function integral processing as an example, we adopt a multi-level parallelism method to implement distributed averaging over time and channel. Dask Array was used to implement super large matrix or arrays with supported parallelism. To maximize the usage of memory, we further exploit the data parallelism provided by Dask that intelligently distributes the computational load across a network of computer agents and has a built-in fault tolerance mechanism. The validity of the proposed pattern was also verified by using the Common Astronomy Software Application (CASA), wherein we analyze the smearing effects on images reconstructed from different resolution visibilities.



中文翻译:

大规模无线电干涉测量集的细粒度分布平均

平方公里阵列(SKA)将成为世界上最大的射电望远镜,最终收集面积超过一平方公里。然而,其数据处理存在巨大挑战。使用现代分布式计算技术解决 SKA 中的海量数据处理问题是最重要的挑战之一。本研究基于Dask分布计算框架,以可见度函数积分处理为例,采用多级并行方法实现对时间和通道的分布式平均。Dask Array用于实现超大矩阵或支持并行度的数组。为了最大限度地利用内存,我们进一步利用Dask提供的数据并行性它通过计算机代理网络智能地分配计算负载,并具有内置的容错机制。所提出的模式的有效性也通过使用通用天文学软件应用程序 (CASA) 进行了验证,其中我们分析了从不同分辨率可见度重建的图像上的拖尾效应。

更新日期:2021-05-20
down
wechat
bug