Abstract

This paper proposes a cooperative search algorithm to enable swarms of unmanned aerial vehicles (UAVs) to capture moving targets. It is based on prior information and target probability constrained by inter-UAV distance for safety and communication. First, a rasterized environmental cognitive map is created to characterize the task area. Second, based on Bayesian theory, the posterior probability of a target’s existence is updated using UAV detection information. Third, the predicted probability distribution of the dynamic time-sensitive target is obtained by calculating the target transition probability. Fourth, a customized information interaction mechanism switches the interaction strategy and content according to the communication distance to produce cooperative decision-making in the UAV swarm. Finally, rolling-time domain optimization generates interactive information, so interactive behavior and autonomous decision-making among the swarm members are realized. Simulation results showed that the proposed algorithm can effectively complete a cooperative moving-target search when constrained by communication distance yet still cooperate effectively in unexpected situations such as a fire.

1. Introduction

In combat, search and reconnaissance are important for providing effective information to accelerate the observe-orient-decide-act (OODA) cycle [14]. Consequently, the US military has identified wide-area target search capability as one of the medium and long-term development goals of unmanned aerial vehicles (UAVs) [3, 4]. UAV swarms have excellent wide-area search capabilities affected by cooperation among swarm members; that is, the collective capability is far greater than the sum of all single UAVs [5, 6]. Cooperative search planning is integral for guiding swarms to achieve wide-area search and target acquisition and has been widely studied [7]. To ensure cooperative search efficiency, a reasonable search planning area and an efficient cooperative strategy are needed.

To determine the area, grid [8, 9], landmark [10, 11], and potential field [12, 13] methods are the main ones proposed. In [8], based on rasterizing the task area, real-time path planning was realized through an improved ant colony algorithm. In [11], the task area was divided by a Voronoi diagram, and waypoint allocation and track smoothing were used to realize the fast planning of a search track in a static environment. In [13], based on describing the task area using an artificial potential field, an improved logarithmic linear learning algorithm was proposed to reduce the risk that a UAV may wander into a zero-potential field area.

Valente et al. [14] proposed a cooperative search method based on a diffusion-weighted uncertainty model. Each UAV is assigned a search area, and then a potential field algorithm based on a rolling-time domain program solves each search track, but this method can only search for a single moving target. Zhang et al. [15] initialized the target probability distribution map using prior target information and then introduced the environmental uncertainty map to guide the UAV to return to a grid that had not been searched for a long time. The result was a feasible scheme for long-time swarm searches and surveillance track planning. However, only one kind of prior information, including the initial position of the target, was considered, so the speed and direction of motion were not considered. Dong et al. [16] defined a digital pheromone map and a corresponding updating strategy to realize UAV cooperation in a moving-target search. In [17], a Markov chain was used to describe the target, but it could not consider communication distance and other constraints.

To sum up, the current research enables UAV swarms to have certain cooperative search capabilities, but there are still problems:(1)The influence of communication distance on a UAV swarm’s cooperation is not considered(2)The use of a variety of prior information in a moving-target search is not considered(3)The risk of collision in a UAV swarm is simplified as a function of height layers, or it is not considered

In view of the preceding, this paper has done the following work:(1)It designed a cooperative search method suitable for dynamic communication distance(2)It analyzed the prior information of four typical moving targets to generate a mathematical model that defines them in a cooperative search(3)It provided an interface to apply to current UAV conflict resolution results [1821] in swarm cooperative search missions

2. Description of the Cooperative Moving-Target Search Problem

2.1. Task Description

UAV swarm cooperation is usually divided into area-coverage and target search tasks [22, 23]. The former is to make the UAV swarm complete a flight over a maximum coverage area as soon as possible when prior information about the mission area is difficult to obtain. The latter is to obtain all target information when some prior information, such as location and quantity of targets, is known [24]. This paper is concerned with the second kind of task. In practice, some prior information of enemy target distribution can be obtained from satellite remote sensing imagery and radar detection, which also provides advantages for target acquisition.

Figure 1 is a typical task scenario for a cooperative UAV target search. There are potential moving targets in the mission area, and UAVs are used to search the mission area. Moving targets, such as enemy missile launch vehicles and radar vehicles, are deployed at corresponding positions to protect key enemy targets. Our four UAVs set out from different positions to inspect and defeat moving enemy targets.

2.2. UAV Motion Model

If the UAV swarm that performs the task is , then its motion is given bywhere is the number of the swarm members and is the scale of the swarm.

To simplify the search decision, a UAV is regarded as a particle in space. The task area is divided into a grid map where the two-dimensional coordinates (w, h) are used to discretize the UAV motion range and decision set [25]. Assuming that the UAV moves in the grid every time, the constraint of normal overload of UAV movement is satisfied by limiting the grid size and the maximum turning angle, thus ensuring flyability along the planned track. Therefore, the UAV can fly in eight directions at any time, as shown in Figure 2.

The state vector of at moment iswhere is the position of at moment in the environmental awareness map, and is the flight course of at moment, and

Then the flight direction of at moment can be given bywhere values are the maximum turning angle constraints of the UAV. The state transition function of UAV is thenwhere is the UAV state transition function determined by Figure 2 and equation (4).

2.3. Autonomous Decision Function

When performing a cooperative search task amid strong electromagnetic interference, centralized decision-making can realize collaboration among swarm members, but it depends on strict communication, which is often difficult to apply [24]. In this paper, the distributed decision-making method is adopted to make full use of the limited UAV communication distance so that swarm members can make interactive decisions to improve the search and avoid collisions.

Rolling-time domain optimization decision-making allows a UAV to predict a subsequent multistep decision according to its current state vector and environmental cognitive map at a given moment [26, 27]. The first step prediction is the actual decision quantity, which avoids a decline in overall search efficiency caused by a focus on only short-term search benefits.

If cannot communicate effectively with other members of the swarm at moment, it uses the differential evolution algorithm to solve the cumulative autonomous decision function according to its current state vector and environmental cognitive map, thereby obtaining the current decision value. The autonomous decision function of at moment iswhere is composed of target search revenue , environmental search income , and expect probe revenue ; indicates the weight of each income in the process of generating interactive information to satisfy and , and values according to the specific task requirements and engineering experience. The benefits of autonomous decision function are described as follows.

2.3.1. Probability Return of Target Existence

Target existence probability income represents the value of possible targets in the corresponding environmental cognitive map, which guides the UAV to search areas that have a high possibility of targets. It is defined aswhere indicates the probability of having a goal in grid at moment that satisfies .

2.3.2. Income from Environmental Uncertainty

Environmental uncertainty revenue represents the reduction of uncertainty in the grid of the corresponding environmental cognition map after the UAV has searched it. It guides the UAV to search the task area with high uncertainty and reduces the possibility of missing targets:where represents the uncertainty of grid in the environment cognition map, and (see Section 3.2 for details).

2.3.3. Comprehensive Income

Comprehensive income is obtained by multiplying environmental uncertainty and target existence probability, which is used to guide the UAV to detect areas with high uncertainty and target existence probability. The detailed description is [15]

Then, the autonomous decision value of at moment can be given by the following formula:where is the state vector of at moment, is the environmental cognitive map of at moment, and is the step size of the rolling-time domain.

2.4. Interactive Decision Function

When is in the communication range of other swarm members, the efficiency of cooperative search can be improved by sharing information, but the problem of collision prevention should be considered. The flight conflict resolution method and information interaction method are described below.

There has been much research on collisions in UAV swarms in the distributed decision framework. The literature [12] guides UAVs to avoid conflict by establishing an artificial potential field, which has the characteristic of a short response time and requires only a small amount of calculation. It can realize real-time obstacle avoidance but cannot resolve complex conflict problems. In [1820], the speed obstacle method broadcasts automatic correlation monitoring to give each UAV the position and speed of the others; thus it solved the potential problem by detecting flight conflict and determining a relief flight path; however, the relief path can easily deviate the UAV from the search target point, thus compromising mission efficiency. In [21], the distributed model predictive control method is adopted, in which the collision avoidance management unit and the interactive graph updating mechanism address conflict resolution in multi-UAV route planning, but it requires a large amount of computation.

To study cooperative strategy in the search for moving targets, this paper introduced the artificial potential field term to meet the basic requirements of collision prevention. In practice, the minimum safe distance can be defined by this method. To achieve a better anticollision effect, the previous conflict resolution decision method replaced the interactive decision function within the minimum safe distance.

The anticollision constraint is defined aswhere represents the number of members that can communicate with (see Section 3.2) and indicates the distance between and the other swarm members within the communication distance of , which is given bywhere means that is not within the direct communication distance of at moment; means that can communicate directly with ; is the second norm, which is used to calculate the distance between two UAVs that can communicate directly. Furthermore, it is possible to obtain the interactive decision function of aswhere is composed of target search revenue , environmental search income , expected detection income , and anticollision constraint ; indicates the weight of each income in the process of generating interactive information that satisfies and .

Then the interactive decision value of can be given by the following formula:where is the state vector of at moment, is a decision-making environment cognitive map fused to the environment map of other members acquired by at moment (see Section 4.1), and is the optimized step size of the rolling-time domain.

3. Construction and Update of Environmental Cognition Map

In a UAV swarm search, environmental cognitive maps (target probability distribution and environmental uncertainty maps) are used to describe the environmental state, and swarm members interact with each other through their own environmental cognitive maps.

3.1. Target Probability Distribution Map Initialization and Update

In a cooperative search, the existence probability of the target in grid at moment can be expressed as . Among them, expresses no target in grid at moment, whereas expresses targets in grid at moment. Now, the target probability distribution diagram of at moment can be expressed as

In order to make full use of the prior information of the moving target, we divide the target in the cooperative search task into four types, as shown in Table 1.

3.1.1. Unknown Target Position and Speed Information

At this time, the probability distribution of targets in the task area is uniform, and the probability density function of any target in the task area can be expressed as

3.1.2. The Initial Position of the Target Is Known, but the Velocity Information Is Unknown

Assume that the task area has targets that have type 2 prior information, and is used to represent its initial position. Each such target can be considered to obey a two-dimensional normal distribution , because the distribution of , is independent of each other and  = 0. Without losing generality, assuming that the initial position distribution of each target is independent, the total probability distribution density of the second target can be expressed as

After time, the target moves from the initial position in an independent, incremental process [16] as described by the Wiener stochastic process: , . The distribution density probability of the second kind of target at time is

3.1.3. The Initial Position and Velocity of the Target Are Known, but the Moving Direction Is Unknown

Assume that the task area has targets having type 3 prior information, and each initial position is expressed by and speed size is represented by . Since the speed of the target is fixed after the probability density of the grid is transferred from the probability distribution of the arc region of the radius by , namely,where is an arc region with center and the radius . By transforming using the first curve integration, the probability distribution of the third-class target area can be obtained as

3.1.4. The Initial Position, Velocity Direction, and Size of the Target Are Known

Assume that the task area has targets with type 4 prior information. The initial position is , speed size is , and represents the speed direction. After , the target position offset is , and the probability distribution density of the fourth target can be expressed as

3.2. Initialization and Update of Environmental Uncertainty Map

The environmental uncertainty in the grid at moment can be expressed as , where represents the information of the grid that is completely uncertain at moment and represents the information of the grid that is completely obtained by a UAV at moment. The environmental uncertainty diagram of at moment can be expressed as

The initial environmental uncertainty map is defined as an all-1 matrix. With the increasing number of UAV searches, the grid uncertainty continues to decline. The specific update method iswhere indicates the attenuation factor of environmental uncertainty [28]. It is used to characterize the amount of information obtained by the UAV after searching the corresponding grid once; is the number of the grid that is searched at moment.

4. Swarm Cooperation Strategy

Under the distributed cooperative search architecture, when reaches the communication range of other members of the swarm, it is possible to avoid searching the searched grids repeatedly through information interaction, thus improving the efficiency of the swarm cooperative search. When making interactive decisions under distributed architecture, the decision of a UAV does not depend on the state information of other UAVs or the operation of central nodes. Therefore, this interactive decision-making method can be applied to a strong confrontation environment that has dynamic changes of effective communication distance.

4.1. Interactive Information Fusion Method

When performing the cooperative search, the target probability maps of all members can be considered to be updated synchronously because the prior information of the target is completely shared. However, the state of the environmental uncertainty map changes in real time with the search, which requires interaction in the cooperative search. The environmental cognitive map of after realizing information interaction is then defined aswhere the target probability distribution map is updated according to the task execution time and target prior information formulas (16)–(21); indicates the interactive environment uncertainty map. If the information interaction between and is taken as an example, the interaction mode can be given bywhere is an upward rounding function; is the environment uncertain map carried by UAV that can communicate with at moment.

4.2. Swarm Communication Topology

When UAVs send messages to each other in the form of a swarm broadcast, the distance between and at moment can be expressed as

The finite set of UAVs that can communicate with at moment can be expressed aswhere represents the communication state between and and is the effective communication distance among swarm members at moment.

4.3. Algorithm Flow

The pseudocode of a UAV swarm moving-target search algorithm with communication distance constraint is shown in Algorithm 1, and the specific steps are described as follows:Step 1: environmental awareness map and parameter initialization. The environmental cognitive map is initialized according to the prior information of four kinds of moving objects. The rolling-time domain optimization step of the UAV is ; the initial effective communication distance is ; the scale of the UAV swarm is ; and the initial state, course, and weight coefficient of the autonomous decision and interactive decision functions of the UAV are set.Step 2: autonomous decision. According to the UAV’s own independent decision function and environmental cognitive map, the differential evolution algorithm is used to solve the problem and make real-time route planning.Step 3: information interaction. When the UAV reaches the communication range of the other members of the swarm, the interactive information of the other members is fused by formula (25), and the cognitive map of its own decision environment is updated.Step 4: interactive decision. After updating the self-knowledge map of the environment according to the interactive decision function in formula (11), the differential evolution algorithm is used to track in real time.Step 5: update the target probability map. In the cooperative search process of a UAV swarm, according to UAV decision information and the target probability map updating method, the target probability distribution map is updated by formula (16) to formula (21), and the environment uncertain map is updated by formula (23).Step 6: repeat step 2 to make the next decision based on the updated environmental cognitive map.

main program
(1)Initialize algorithm parameters, environment map, UAV position, and heading
(2)for
(3)if
(4)Initialize the environmental cognitive map according to formula (16);
(5)
(6)if
(7)Initialize the environmental cognitive map according to formula (21);
(8)end if
(9)end for
(10)for
(11)Update the target probability distribution map according to equations (16)–(21);
(12)for
(13)  According to formula (26), judge whether to make an independent decision;
(14)  According to formula (11), a differential evolution algorithm is adopted to make a real-time decision;
(15)   for
(16)    Determine the interactive member set according to formula (27);
(17)    Complete information interaction and fusion according to formula (25);
(18)   end for
(19)   According to formula (14), the differential evolution algorithm is adopted to make real-time decisions;
(20)   Update the UAV position according to formula (5);
(21)   Update your own environmental cognition map according to formula (23);
(22)  end for
(23)end for

5. Comparative Analysis of Simulation

In this section, concerning the moving-target search scene with four types of prior information, the UAV swarm cooperative search was simulated numerically, with the impact of introducing prior target information. In the cooperative search, the task was simulated and analyzed, and the effectiveness of the algorithm in the strong confrontation environment was verified, such as the dynamic change of the communication distance, the damage of some members of the swarm, and other emergencies.

5.1. Task Assumption and Parameter Setting

The reconnaissance mission area is a 30 × 40 km rectangle divided into 1 × 1 km grids. The initial distribution, speed direction, and performance constraints of a UAV swarm are shown in Table 2, and the parameters of autonomous decision-making and interactive decision-making are shown in Table 3 and Table 4, respectively. Set the simulation time to 6000 s and the rolling-time domain optimization step to 30 s. The simulation time is divided into 600 planning steps with an interval of 10 s. The initial effective UAV communication distance is set at 3 km. According to the predetermined prior information, the initial target location distribution is shown in Figure 3, the speed of the target is 10 km/h, and the target probability distribution generated by the UAV swarm according to the prior information is shown in Figure 4.

5.2. Planning Results of Moving-Target Cooperative Search

Given a limited communication distance, the numerical simulation of a cooperative moving-target search is carried out using the number of captured targets as an evaluation index. The simulation results are shown in Figure 5.

Figure 5(a) shows that the swarm captured two type 4 targets after 1000 s based on prior information and another target had moved out of the task area, so the target probability distribution is concentrated. It attracted the attention of the UAV swarm and was then captured. In Figure 5(b), after the swarm captured the category 4 targets, it quickly carried out a cooperative search in the center of the task area where other target categories were concentrated. According to Figures 5(c) and 5(d), when the task was executed at 6000 s, the swarm completed coverage of the task area and captured 1 type-1 target, 1 type-2 target, 2 type-3 targets, and 3 type-4 targets. Because prior information of the type-1 moving target was unknown, it was difficult to capture, but the swarm captured other moving targets by making full use of prior information: the richer the prior information, the higher the capture probability.

5.3. Search Path Planning under Dynamic Communication Conditions

Electromagnetic interference (EMI) is a regular means of attack in a strong confrontation environment, and it has a severe influence on battlefield communication. To verify the applicability of the algorithm in this paper to a complex communication environment, this section simulated and analyzed the cooperative search of a UAV swarm under a dynamically changing effective communication distance. The total simulation time (6000 s) was divided into 600 planning steps at an interval of 10 s. The effective initial intermachine communication distance was 20 km. After 1500 s, the distance was reduced to 10 km and restored to 2000 s. The effective communication distance was reduced to 0 km when the task was executed at 3500 s and was restored at 4500 s.

To further demonstrate the applicability of the algorithm in a dynamic communication environment, this section took environmental uncertainty as the main goal guiding the swarm and used the search coverage rate of the task area as the evaluation index to carry out a numerical simulation. At this time, the algorithm simulation parameters were updated (Tables 5 and 6).

The left half of Figure 6 shows that when the search went to 1500 s because of decreased effective communication distance, the frequency of information interaction among the UAVs also decreased, but after communication was restored at 2000 s, the number of information interactions was quickly restored. From the right half of Figure 6, when the search went for 3500 s, the distance between computers decreased, which led to slow growth in the area-coverage rate. This was caused by the repeated searching of some grids after interference led to the loss of information interaction. At 4500 s, communication and interactive decision-making were restored, and search coverage gradually improved, which shows that interactive decision-making can improve cooperative efficiency but that a UAV can still perform scheduled tasks autonomously when communication conditions are not guaranteed.

5.4. Search Route Planning When Some Members of the Swarm Are Damaged

Compared with the preplanning method (referring to the related documents of preplanning), the dynamic planning method can effectively reduce the risk that the enemy can predict and attack the track and adapt to unexpected situations such as the failure of some members. In this section, the parameters of target probability distribution (Tables 3 and 4) and environmental uncertainty (Tables 5 and 6) guide the swarm and take the number of captured targets and the coverage rate as evaluation indexes for the numerical simulation. In this scenario, UAV1 and UAV3 failed at 1500 s and 3500 s, respectively, and stopped executing tasks.

From Figures 7(a)7(c), when UAV1 and UAV3, respectively, stop performing tasks due to faults, UAV2 and UAV4 still performed effectively and achieved higher regional coverage because in the distributed decision-making architecture each UAV does not depend on other members to make decisions. When the swarm is large, the efficiency of a cooperative search can be improved through interactive decision-making. It can be seen from Figure 7(d) that when some members are damaged, which leads to a decline in swarm size, a UAV can still carry out search tasks through autonomous decision-making, which has good robustness.

Figure 8 shows that, driven by prior information, the UAV swarm searches for and quickly captures type-4 targets according to prior information. It can be seen from the whole search process from Figures 8(a)8(d), even after UAV1 and UAV3 quit at 1500 s and 3000 s, UAV2 and UAV4 still searched effectively.

5.5. Influence of Communication Distance on Search Efficiency

On the basis of completing the path planning of the cooperative moving-target search using the coverage rate as the main evaluation index, the efficiency under different communication distances was analyzed by using the control variable method. Five groups of simulations having a 10 km communication interval over 6000 s were carried out and the results are shown in Figure 9.

According to the simulation results, information interaction avoided the repeated search of the same grid, and the cooperative efficiency of the UAV swarm improved. With the increase in communication distance, the cooperative search efficiency gradually increases, but when the communication distance was greater than 30 km the efficiency no longer increased. The results showed a positive correlation in the nonlinear relationship of communication distance to cooperative search efficiency. In practice, the minimum effective communication distance can be preliminarily determined by the simulation to achieve better task cooperation.

6. Conclusion

(1)This algorithm simulated the cooperative search decision-making of a UAV swarm at a limited communication distance and improved its robustness in a strong countermeasure environment by establishing autonomous and interactive decision-making(2)The mathematical models and updating methods of prior information of the four types of moving targets were established so that a UAV swarm could make full use of prior information to carry out a cooperative search(3)Using a distributed control architecture in an experimental simulation, the algorithm in this paper proved that a cooperative search task can still be completed effectively when some members fail

The algorithm did not consider the influence of communication delay and packet loss on the cooperative search efficiency, but it could in subsequent research to improve the cooperative search algorithm of the UAV swarm.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare that there are no conflicts of interest.

Authors’ Contributions

Ning Wang, Zhe Li, and Feihu Zhao were responsible for conceptualization, methodology, software, and validation; Ying Li and Feihu Zhao performed data curation. Ning Wang, Xiaolong Liang, and Ying Li prepared the original draft and reviewed and edited the manuscript. All authors have read and agreed to the published version of the manuscript.

Acknowledgments

This work was funded by the National Natural Science Foundation of China, Grant no. 61703427.