当前位置: X-MOL 学术Nat. Mach. Intell. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Neutralizing the impact of atmospheric turbulence on complex scene imaging via deep learning
Nature Machine Intelligence ( IF 18.8 ) Pub Date : 2021-10-14 , DOI: 10.1038/s42256-021-00392-1
Darui Jin 1, 2 , Ying Chen 1 , Yi Lu 1 , Junzhang Chen 1 , Peng Wang 1 , Zichao Liu 1 , Sheng Guo 1 , Xiangzhi Bai 1, 3, 4
Affiliation  

A turbulent medium with eddies of different scales gives rise to fluctuations in the index of refraction during the process of wave propagation, which interferes with the original spatial relationship, phase relationship and optical path. The outputs of two-dimensional imaging systems suffer from anamorphosis brought about by this effect. Randomness, along with multiple types of degradation, make it a challenging task to analyse the reciprocal physical process. Here, we present a generative adversarial network (TSR-WGAN), which integrates temporal and spatial information embedded in the three-dimensional input to learn the representation of the residual between the observed and latent ideal data. Vision-friendly and credible sequences are produced without extra assumptions on the scale and strength of turbulence. The capability of TSR-WGAN is demonstrated through tests on our dataset, which contains 27,458 sequences with 411,870 frames of algorithm simulated data, physical simulated data and real data. TSR-WGAN exhibits promising visual quality and a deep understanding of the disparity between random perturbations and object movements. These preliminary results also shed light on the potential of deep learning to parse stochastic physical processes from particular perspectives and to solve complicated image reconstruction problems given limited data.



中文翻译:

通过深度学习中和大气湍流对复杂场景成像的影响

具有不同尺度涡流的湍流介质在波传播过程中会引起折射率的波动,干扰原有的空间关系、相位关系和光路。二维成像系统的输出会受到这种效应带来的变形。随机性以及多种类型的退化使得分析互易物理过程成为一项具有挑战性的任务。在这里,我们提出了一个生成对抗网络(TSR-WGAN),它整合了嵌入在三维输入中的时间和空间信息,以学习观察到的和潜在的理想数据之间的残差表示。无需对湍流的规模和强度进行额外假设即可生成视觉友好且可信的序列。TSR-WGAN 的能力通过对我们的数据集的测试得到证明,该数据集包含 27,458 个序列,411,870 帧算法模拟数据、物理模拟数据和真实数据。TSR-WGAN 展示了令人鼓舞的视觉质量以及对随机扰动和物体运动之间差异的深刻理解。这些初步结果还揭示了深度学习从特定角度解析随机物理过程以及在数据有限的情况下解决复杂图像重建问题的潜力。

更新日期:2021-10-14
down
wechat
bug