Rational use of multilevel structures of deep networks to extract multiscale features is crucial for instance segmentation. The Feature Pyramid Network (FPN) is a classical architecture that enriches the semantic information of multiscale objects. However, inherent defects in FPN structure are bound to cause loss of information during feature extraction and feature fusion. In this paper, we propose a feature pyramid structure (called Refine-FPN) based on a non-local multi-feature aggregation operation, a module that integrates multi-scale feature to rely on attention mechanisms to improve pyramid feature representation. The algorithm enriches the feature details of feature layers by aggregating multiple features to form a contextual global feature representation. By replacing FPN with Refine-FPN in the Mask R-CNN, our model improved the performance of the mask AP by 0.6% and 0.5% on the COCO dataset, when using ResNet-50 and ResNet-101 as the backbone, respectively. Moreover, it is friendly to integrate the proposed method into other popular architectures. For example, equipping the Cascade Mask R-CNN with Refine-FPN achieves an improvement of 0.5% and 0.4% mask AP under ResNet-50 and ResNet-101, respectively.

合理使用深度网络的多级结构来提取多尺度特征对于实例分割至关重要。特征金字塔网络（FPN）是一种丰富多尺度对象语义信息的经典架构。然而，FPN结构的固有缺陷必然会导致特征提取和特征融合过程中的信息丢失。在本文中，我们提出了一种基于非局部多特征聚合操作的特征金字塔结构（称为Refine-FPN），该模块集成了多尺度特征以依靠注意力机制来改进金字塔特征表示。该算法通过聚合多个特征形成上下文全局特征表示，丰富了特征层的特征细节。通过在 Mask R-CNN 中将 FPN 替换为 Refine-FPN，当使用 ResNet-50 和 ResNet-101 作为主干时，我们的模型分别在 COCO 数据集上将 mask AP 的性能提高了 0.6% 和 0.5%。此外，将所提出的方法集成到其他流行的架构中是友好的。例如，为 Cascade Mask R-CNN 配备 Refine-FPN，在 ResNet-50 和 ResNet-101 下分别实现了 0.5% 和 0.4% 的 mask AP 改进。