Multi-task learning aims to tackle various tasks with branched feature sharing architectures. Considering its diversity and complexity, discriminative feature representations need to be extracted for each individual task. Fixed geometric structures as a limitation of convolutional neural networks (CNNs) in building models, is also exists and poses a severe challenge in multi-task learning since the geometric variations will augment when we deal with multiple tasks. In this paper, we go beyond these limitations and propose a novel multi-task network by introducing the deformable convolution. Our design, the Deformable Multi-Task Network (DMTN), starts with a single shared network for constructing a shared feature pool. Then, we present task-specific deformable modules to extract discriminative features to be tailored for each task from the shared feature pool. The task-specific deformable modules utilize two new parts, deformable part and alignment part, to extract more discriminative task-specific features while greatly enhancing the transformation modeling capability. Experiments conducted on various multi-task learning types demonstrate the effectiveness of the proposed method. On multiple classification tasks, semantic segmentation and depth estimation tasks, our DMTN exceeds state-of-the-art approaches against strong baselines.

多任务学习旨在通过分支功能共享体系结构来解决各种任务。考虑到它的多样性和复杂性，需要为每个单独的任务提取具有区别性的特征表示。作为构建模型中卷积神经网络（CNN）的局限性，固定几何结构也存在，并且在多任务学习中提出了严峻的挑战，因为当我们处理多个任务时，几何变化会增加。在本文中，我们超越了这些限制，并通过引入可变形卷积提出了一种新颖的多任务网络。我们的设计是“可变形多任务网络（DMTN）”，它从用于构建共享功能库的单个共享网络开始。然后，我们介绍特定于任务的可变形模块从共享功能库中提取针对每个任务量身定制的区别性功能。在特定的任务模块变形利用两个新的部件，可变形部分和校准部件，提取更有辨别特定任务的功能，同时极大地提高了变换模型的能力。在各种多任务学习类型上进行的实验证明了该方法的有效性。在多个分类任务，语义分割和深度估计任务上，我们的DMTN超越了针对强大基准的最新方法。