当前位置: X-MOL 学术Astron. Comput. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A Real Time Processing system for big data in astronomy: Applications to HERA
Astronomy and Computing ( IF 2.5 ) Pub Date : 2021-08-02 , DOI: 10.1016/j.ascom.2021.100489
P. La Plante 1, 2, 3 , P.K.G. Williams 4, 5 , M. Kolopanis 6 , J.S. Dillon 1 , A.P. Beardsley 6, 7 , N.S. Kern 1, 8 , M. Wilensky 9 , Z.S. Ali 1 , Z. Abdurashidova 1 , J.E. Aguirre 3 , P. Alexander 10 , Y. Balfour 11 , G. Bernardi 11, 12, 13 , T.S. Billings 3 , J.D. Bowman 6 , R.F. Bradley 14 , P. Bull 15 , J. Burba 16 , S. Carey 10 , C.L. Carilli 17
Affiliation  

As current- and next-generation astronomical instruments come online, they will generate an unprecedented deluge of data. Analyzing these data in real time presents unique conceptual and computational challenges, and their long-term storage and archiving is scientifically essential for generating reliable, reproducible results. We present here the real-time processing (RTP) system for the Hydrogen Epoch of Reionization Array (HERA), a radio interferometer endeavoring to provide the first detection of the highly redshifted 21 cm signal from Cosmic Dawn and the Epoch of Reionization by an interferometer. The RTP system consists of analysis routines run on raw data shortly after they are acquired, such as calibration and detection of radio-frequency interference (RFI) events. RTP works closely with the Librarian, the HERA data storage and transfer manager which automatically ingests data and transfers copies to other clusters for post-processing analysis. Both the RTP system and the Librarian are public and open source software, which allows for them to be modified for use in other scientific collaborations. When fully constructed, HERA is projected to generate over 50 terabytes (TB) of data each night, and the RTP system enables the successful scientific analysis of these data.



中文翻译:

天文学大数据实时处理系统:HERA的应用

随着当前和下一代天文仪器的上线,它们将产生前所未有的海量数据。实时分析这些数据带来了独特的概念和计算挑战,它们的长期存储和存档对于生成可靠、可重复的结果在科学上是必不可少的。我们在此展示了用于再电离阵列氢时代 (HERA) 的实时处理 (RTP) 系统,这是一种射电干涉仪,旨在通过干涉仪首次探测到来自宇宙黎明和再电离时代的高度红移的 21 厘米信号. RTP 系统由在获取原始数据后不久运行的分析程序组成,例如校准和检测射频干扰 (RFI) 事件。RTP 与图书馆员密切合作,HERA 数据存储和传输管理器,可自动摄取数据并将副本传输到其他集群以进行后处理分析。RTP 系统和图书馆员都是公共和开源软件,允许对其进行修改以用于其他科学合作。完全构建后,HERA 预计每晚生成超过 50 TB 的数据,而 RTP 系统能够成功地对这些数据进行科学分析。

更新日期:2021-08-15
down
wechat
bug