Multi-exposure image fusion inevitably causes ghost artifacts owing to inaccurate image registration. In this study, we propose a deep learning technique for the seamless fusion of multi-exposed low dynamic range (LDR) images using a focus-pixel sensor. For auto-focusing in mobile cameras, a focus-pixel sensor originally provides left (L) and right (R) luminance images simultaneously with a full-resolution RGB image. These L/R images are less saturated than the RGB images because they are summed up to be a normal pixel value in the RGB image of the focus pixel sensor. These two features of the focus pixel image, namely, relatively short exposure and perfect alignment are utilized in this study to provide fusion cues for high dynamic range (HDR) imaging. To minimize fusion artifacts, luminance and chrominance fusions are performed separately in two sub-nets. In a luminance recovery network, two heterogeneous images, the focus pixel image and the corresponding overexposed LDR image, are first fused by joint learning to produce an HDR luminance image. Subsequently, a chrominance network fuses the color components of the misaligned underexposed LDR input to obtain a 3-channel HDR image. Existing deep-neural-network-based HDR fusion methods fuse misaligned multi-exposed inputs directly. They suffer from visual artifacts that are observed mostly in saturated regions because pixel values are clipped out. Meanwhile, the proposed method reconstructs missing luminance with aligned unsaturated focus pixel image first, and thus, the luma-recovered image provides the cues for accurate color fusion. The experimental results show that the proposed method not only accurately restores fine details in saturated areas, but also produce ghost-free high-quality HDR images without pre-alignment.

中文翻译：

---

使用聚焦像素处理复杂运动场景的无鬼影深空动态范围成像

多次曝光图像融合会由于图像配准不准确而不可避免地导致重影伪影。在这项研究中，我们提出了一种深度学习技术，用于使用聚焦像素传感器对多曝光低动态范围（LDR）图像进行无缝融合。为了在移动相机中进行自动对焦，焦点像素传感器最初会同时提供左（L）和右（R）亮度图像以及全分辨率RGB图像。这些L / R图像比RGB图像饱和度低，因为它们被加在一起成为焦点像素传感器的RGB图像中的正常像素值。这项研究利用了聚焦像素图像的这两个特征，即相对较短的曝光和完美的对准，为高动态范围（HDR）成像提供融合线索。为了最大程度地减少融合伪影，亮度和色度融合在两个子网中分别执行。在亮度恢复网络中，首先通过联合学习对两个异质图像（聚焦像素图像和相应的曝光过度的LDR图像）进行融合，以生成HDR亮度图像。随后，色度网络融合未对准的曝光不足的LDR输入的颜色分量，以获得3通道HDR图像。现有的基于深度神经网络的HDR融合方法直接融合未对齐的多重曝光输入。它们会遭受视觉伪影的影响，因为像素值被裁剪掉了，因此通常会在饱和区域中观察到。同时，提出的方法首先利用对准的不饱和焦点像素图像重建了缺失的亮度，因此，亮度恢复的图像为准确的色彩融合提供了线索。