Diverse receptive field network with context aggregation for fast object detection

Highlights

• The context introduced by DRF modules improves the detection performance by assisting the detector to better distinguish the background and objects.

• The design principles for DRF modules can better encode contextual information.

• Almost all the categories are benefitted from context aggregation, and serval monotonous-background categories and small-scale objects get large gains.

• The proposed DRFNet achieves a good trade-off between speed and accuracy on PASCAL VOC and MS COCO datasets.

Abstract

Current context-utilizing detectors are all based on two-stage approaches. However, their computational efficiency and context quality extremely depend on the accuracy of proposal-generating methods, which limits their performance and makes them hardly perform real-time detection. In this work, we present a context-exploited method that integrates features in different receptive fields to obtain contextual representation. Based on this idea, we put forward the multi-branch diverse receptive field modules (DRF modules) and their design principles to encode context. To further utilize contextual information for fast object detection, we propose a one-stage diverse receptive field network (DRFNet). In DRFNet, the DRF modules are first applied to capture rich context as the basis, then a parallel structure is constructed to exploit the context at different scales along with DRF modules. Comprehensive experiments indicate that the context introduced by our methods improves the detection performance and DRFNet achieves a good trade-off between speed and accuracy.

Introduction

Object detection has enjoyed significant advances benefiting from the rapid process of deep convolutional neural networks [1], [2], [3], [4], [5], [6]. The state-of-the-art object detectors [7], [8], [9], [10], [11] have achieved impressive performance in detection benchmarks. However, most of them only exploit the features inside the proposals or anchors and ignore the contextual information surrounding them. It has been recognized that contextual information plays an important role in object detection [12], [13].

To utilize contextual information, an intuitive approach is to employ multi-region for feature aggregation [14], [15], [16], [17], [18], [19], [20], [21], as shown in Fig. 1(a). Such methods extract features in multi-region associated with the original proposal, then fuse them to obtain contextual representations.

Another idea is committed to creating contextual clues to guide the detection [22], [23], [24], [25], [26], [27], which is popular in both traditional methods and CNN-based detectors, as shown in Fig. 1(b). Traditional methods [12], [13], [25] usually utilize local or global appearance statistics to assist detection, while in the deep learning area context clues are more diverse, such as segmentation information [23], previous detection results [28], instance-level relationships [29], [30] and so on. Compared with the former, this kind of approach utilizes context more explicitly.

Nevertheless, both multi-region methods and contextual-cue methods are based on proposals to further exploit context information or contextual clues. This means that they can only be embedded in two-stage detectors. In addition, their computation efficiency and context quality depend on the accuracy of proposal-generating methods, which limits their performance and makes them hardly perform real-time detection. For example, if the pre-generating proposals only contain the background or some parts of the object, it is meaningless to exploit context based on this kind of proposals. However, this situation is inevitable in two-stage methods currently.

So why do we not exploit context in the feature extraction stage regardless of proposals? In spired by multi-region methods [14], [15], [16], [17], [18], [19], [20], [21], we believe that the aggregated features from multi-scale regions contain contextual information, so do the features in different receptive fields (RFs). Therefore, we fuse features with different RFs in the same regions during feature extraction, so that the obtained features encode context information and do not rely on proposals, as shown in Fig. 1(c). Since contextual representations are obtained in the feature-extracting stage, this contextual module can be embedded in one-stage approaches.

Based on this idea, we propose a one-stage diverse receptive field network (DRFNet) aiming to utilize context information for fast object detection. Specifically, we put forward the multi-branch fully-convolutional DRF modules and the parallel structure to achieve this. The multi-branch DRF modules are designed according to three design principles and each branch stacks conv layers and dilated conv layers [31] to exploit detail information and the context. Additionally, the parallel structure is constructed via the spatial pyramid pooling to capture the context at different scales along with DRF modules. The general framework of DRFNet is shown in Fig. 2.

We show that the context introduced by our methods can improve the detection performance of most categories from 0.5% mAP to 3% mAP on PASCAL VOC. In addition, serval monotonous-background categories and small-scale objects get large gains such as bottle (7.8% mAP), chair (7.6% mAP), boat (6.5% mAP), airplane (4.9% mAP), and TV (4.5% mAP). These validate that our method is effective and the introduced local context assists the detector in better distinguishing the background and foreground, which improves detection performance.

Contribution: To summarize, our contributions are as follows:

• We propose a new idea of exploiting context information for fast object detection, which aggregates features in different receptive fields in the feature extraction stage. Therefore, it no longer relies on proposals and can be embedded in one-stage approaches with high computational efficiency.

• We put forward DRF modules and their design principles, according to which we can make better use of DRF modules to exploit context information. Additionally, we can design DRF modules in any detection framework, which facilitates the follow-up research.

• We design a one-stage DRFNet, which employs parallel branching structure and DRF modules to integrate context information at different scales for fast object detection. DRFNet300 and DRFNet512 achieve 80.2% mAP (38.2FPS) and 82.3% mAP (18.4FPS) on PASCAL VOC 2007, and 78.3% mAP and 80.4% mAP on PASCAL VOC 2012 respectively. On COCO, DRFNet300 and DRFNet 512 produce 30% AP and 33.5% AP.

The remainder of this paper is organized as follows. Section 2 reviews the related work. In Section 3 we investigate the DRF modules, and Section 4 details the proposed DRFNet. Experiments are given in Section 5, followed by the conclusion in Section 6.