Environmental Perception with Spatial Regularization Correlation Filter for Visual Tracking

Highlights

• We propose a new framework for visual target tracking that perceives changes in the environment and adaptively adjusts the learning rate to avoid template pollution. Specifically, the APEC indicator and the response error between two frames are used to jointly perceive changes in the environment and adaptively adjust the learning rate.

• We uses realistic background information to train filters, which not only suppresses the interference of background information, but also enhances the discrimination ability of the filter.

• We solve the energy function of the filter as a non-closed solution. To reduce computational complexity, we use ADMM algorithm to transform the energy function into two subproblems with globally optimal solutions, because both subproblems are convex. Therefore, the two sub-problems have their own closed solution and globally optimal solutions.

• Experimental results on three benchmarks, i.e., OTB-2015, TC-128 and VOT-2018 validate that the proposed approach obviously improves the tracking accuracy of CF based trackers.

Abstract

With the introduction of correlation filtering (CF), the performance of visual object tracking is significantly improved. Circular shifts collecting samples is a key component of the CF tracker, and it also causes negative boundary effects. Most trackers add spatial regularization to alleviate boundary effects well. However, these trackers ignore the effect of environmental changes on tracking performance, and the filter discriminates poorly in the background interference. Here, to break these limitations, we propose a new correlation filter model, namely Environmental Perception with Spatial Regularization Correlation Filter for Visual Tracking. Specifically, we use the Average Peak to Correlation Energy (APCE) and the response value error between the two frames together to perceive environmental changes, which adjusts the learning rate to make the template more adaptable to environmental changes. To enhance the discriminatory capability of the filter, we use real background information as negative samples to train the filter model. In addition, the introduction of the regular term destroys the closed solution of CF, and this problem can be effectively solved by the use of the alternating direction method of multipliers (ADMM). Extensive experimental evaluations on three large tracking benchmarks are performed, which demonstrate the good performance of the proposed method over some of the state-of-the-art trackers.

Introduction

Visual object tracking is a popular research direction in the field of computer vision. Its purpose is to be able to accurately determine the location of a target in an image sequence without the need to identify what the target is. It has been widely used in areas such as intelligent visual surveillance [1], [2], unmanned vehicles [3], [4] , human–computer interaction [5] and biology [6], etc. Although in the past decades, visual object tracking algorithms have developed rapidly and in a wide variety of ways, there are still many challenges. For example, the tracking process is often disturbed by scale variation deformation, fast movements or target occlusion. At the same time, achieving high-speed and high-precision target localization has always been a difficult task.

In CF trackers, the use of circular shifts and Fast Fourier Transform (FFT) for effective training and detection can greatly improve computational efficiency, but it also causes boundary effects. This problem leads to degraded tracking performance, especially in the case of fast moving targets. Recently, there are two schemes which can effectively reduce the influence of boundary effects. The first scheme is the use of cropping operations to collect real negative samples in order to improve samples quality, such as CFLB [7], BACF [8], etc. The second scheme adds spatial regularization to constrain the weight of the filter coefficients, such as SRDCF [9], CSR-DCF [10], etc. However, SRDCF uses circular shifts to obtain negative samples instead of using real negative samples, so it is prone to drift when the target is similar to the background information. SRDCF also failed to consider the effect of environmental changes on the template, which can lead to pollution of the template when the environment changes (e.g., occlusion, background clutters). In addition, SRDCF uses Gauss–Seidel solver filters, which are computationally expensive.

In this paper, for these problems mentioned above, we propose a novel visual object tracking algorithm, namely Environmental Perception with Spatial Regularization Correlation Filter for Visual Tracking. The main contributions of this paper are summarized as follows:

$•$ We propose a new framework for visual target tracking that perceives changes in the environment and adaptively adjusts the learning rate to avoid template pollution. Specifically, the APEC indicator [11] and the response error between two frames are used to jointly perceive changes in the environment and adaptively adjust the learning rate.

$•$ We uses realistic background information to train filters, which not only suppresses the interference of background information, but also enhances the discrimination ability of the filter.

$•$ We solve the energy function of the filter as a non-closed solution. To reduce computational complexity, we use ADMM algorithm [12] to transform the energy function into two sub-problems with globally optimal solutions, because both sub-problems are convex. Therefore, the two sub-problems have their own closed solution and globally optimal solutions.

$•$ Experimental results on three benchmarks, i.e., OTB-2015 [13], TC-128 [14] and VOT-2018 [15] validate that the proposed approach obviously improves the tracking accuracy of CF based trackers.