We present a bi-branch network for efficient dynamic scene deblurring. The challenge is to simultaneously reduce the computational cost and enhance the restoration accuracy. The proposed network conduct heterogeneous transformations on motion and RGB content in an encoder–decoder structure with skip connections. The computational efficiency is achieved by explicitly decomposing the intertwined mapping of spatiotemporal and cross-channel correlations into the motion branch that processes grayscale frames with our proposed pseudo depth-wise separable 3D convolution and the color branch that conducts depth-wise separable 2D convolution on RGB content. We refine features captured by the motion branch and the color branch by incorporating a lightweight nonlocal fusion layer that adapts the double attention operation to aggregate heterogeneous transformations and generate for each location in the feature space an output based on its correlation with the entire video clip. Our nonlocal fusion maintains low computational cost in processing high-resolution frames and operates in a patch-based manner during inference. The proposed architecture strikes the right balance between complexity and accuracy for dynamic scene deblurring. In comparison with state-of-the-art methods, the proposed network is compact and shows competitive restoration accuracy with a significant reduction in computational cost.

我们提出了一种用于高效动态场景去模糊的双分支网络。挑战在于同时降低计算成本并提高恢复精度。所提出的网络在具有跳过连接的编码器-解码器结构中对运动和RGB内容进行异构转换。通过将时空和跨通道相关性的交织映射显式分解为运动分支（通过我们提出的伪深度方向可分离3D卷积处理灰度帧和在RGB上进行深度方向可分离2D卷积的颜色分支）来实现计算效率内容。我们通过合并轻量级的非局部融合层来优化由运动分支和颜色分支捕获的特征，该层适用于双重注意操作以聚合异构变换，并基于特征空间中与整个视频剪辑的相关性为特征空间中的每个位置生成输出。我们的非局部融合在处理高分辨率帧时保持较低的计算成本，并且在推理过程中以基于补丁的方式进行操作。所提出的架构在动态场景去模糊的复杂性和准确性之间取得了正确的平衡。与最先进的方法相比，该网络结构紧凑，显示出具有竞争力的恢复精度，并显着降低了计算成本。