Dynamic multi-objective evolutionary algorithm with objective space prediction strategy

Highlights

• A new algorithm to deal with dynamic multi-objective optimization problems.

• Probabilistic models used both in the evolutionary process and environment changes.

• Statistical information extracted from both the decision and objective space.

• Both the population distribution and local information of each individual considered.

• A dynamic real-world multi-objective optimization problem solved.

Abstract

To solve dynamic multi-objective optimization problems, a dynamic multi-objective evolutionary algorithm (DMOEA) must be able to deal with the dynamics of the environment, and such modifications can lead to new optimal solutions over time. Various algorithms have been proposed that modify the way a change is handled. Among them, prediction-based methods are promising for solving this kind of problem. They provide guided direction for population evolution through a prediction mechanism that assists the DMOEA to respond quickly to new changes. Based on these strategies, we propose a dynamic non-dominated sorting differential evolution improvement with prediction in the objective space (DOSP-NSDE). The proposal uses the objective space prediction (OSP) strategy for both the static evolutionary process (between changes) and the change reaction mechanism to predict the new optimal front location. Experiments were performed on a real-world problem and four sets of test problems: FDA, dMOP, UDF, and DF. Comparison of DOSP-NSDE with several algorithms in the literature, considering three metrics, is presented, showing that the proposal is competitive with most problems.

Introduction

Several contemporary problems, such as scheduling, vehicle routing, and control, may involve two or more goals that must be simultaneously achieved. These objective functions can be conflicting in nature, i.e., there are no solutions that simultaneously achieve the best of each objective. Typically, optimal solutions entail trade-offs between the goals, respecting the constraints, if any, of a problem. If the fitness landscape, parameters, or constraints of these problems change over time, the problem is known as a dynamic multi-objective optimization problem (DMOP). A continuous DMOP is mathematically defined as:

$$\begin{array}{c}\hfill \underset{x}{min}\mathbf{f}(\mathbf{x},t)={[f_1(\mathbf{x},t)f_2(\mathbf{x},t)\dots f_m(\mathbf{x},t)]}^T\hfill \\ \hfill subjectto:\mathbf{x}\in {[x_{min}^i,x_{max}^i]}^D\hfill \\ \hfill g_j(\mathbf{x},t)\le 0,j=1,\dots ,q\hfill \\ \hfill h_j(\mathbf{x},t)=0,j=q+1,\dots ,p\hfill \end{array}$$

where $\mathbf{x}\in {\Re }^D$ is the decision vector, $D$ is the total number of decision variables, and the values ${[x_{min}^i,x_{max}^i]}^D$ represent the lower and upper limits, respectively, for the decision variables. These limits, also called variable constraints, define a D-dimensional hypercube space $S\in {\Re }^D$. The feasible decision space $S_{dec}\subseteq S$ is defined by the set of $p$ additional linear or non-linear constraints, divided into $q$ inequality constraints and $p-q$ equality constraints. $f_i(\mathbf{x},t)\in S_{obj}$ is the $i$th objective function, and $S_{obj}$ is the objective space. $m$ is the total number of objectives, and $\mathbf{f}(\mathbf{x},t)\in S_{obj}\subseteq {\Re }^m$ is the vector of objective functions. For each solution $\mathbf{x}$ in the decision space, there is a $\mathbf{f}\left(\mathbf{x}\right)$ in $S_{obj}$.

Research on evolutionary algorithms (EAs) has shown that they can be applied to dynamic problems [1], [2], [3], [4], [5]. A significant number of dynamic multi-objective evolutionary algorithms (DMOEAs) have been developed to solve DMOPs [6]. A DMOEA must be able to continuously find the Pareto Optimal Front (POF) in a changing environment [6], that is, to make the population converge toward a changing Pareto optimal set (POS), determining a well-distributed set of solutions. Therefore, a successful DMOEA must achieve its goals while dealing with issues such as a discontinuous landscape, different shapes of POFs, and POS in each environment.

The severity and frequency of changes may also influence the convergence of a DMOEA. The severity of the change (rate) defines its degree, from low to high. A DMOEA is often more effective in the first case because the algorithm can more easily converge to the POF because the information obtained from the previous environment can be exploited and reused to guide the convergence of the POS [6]. If the severity of the change is considerable, each instance of the problem may be uncorrelated with the next one. In this case, therefore, it may be useful to restart the algorithm completely [6]. The frequency of change determines how often environments change. As the frequency increases, the time necessary for adaptation becomes shorter, impairing the search [6].

When a landscape changes, a new instance of the problem is established; consequently, diversity is essential for finding new solutions. However, the convergence may provoke loss of diversity over the course of the optimization process, making it more difficult to track a dynamic POF. To deal with this issue, a DMOEA can introduce a degree of diversity through the change reaction mechanism (CRM). Thus, DMOEAs can be classified according to the CRM used as [7] memory-based approaches, population-based approaches, immigrant-based approaches, and prediction-based approaches. The memory approach can guide the DMOEA to the wrong places in the search space, hampering convergence. [8] concluded that memory could not act alone. Population-based approaches divide a given population into several subpopulations that explore multiple regions of the fitness landscape to locate and track the optimal POF. However, finding suitable data decomposition is the main difficulty with these strategies [7]. Immigration strategies increase population diversity without the right guidance, and the algorithm might not converge in more complex DMOPs [9]. Prediction-based methods have been shown to be promising because they also accelerate the convergence of the algorithm when the forecasting model is accurate [10], i.e., they are expected to insert new individuals (increasing diversity) in promising regions (facilitating convergence).

Prediction-based methods [10], [11], [12], [13], [14], [15], [16], [17], [18] provide guided direction for population evolution through a forecasting model. This forecasting model can predict the next location of the POS/POF based on the patterns exhibited from the best solutions previously discovered. Currently, predictive EAs usually perform the regression¹ procedure in the decision space, often a higher-dimensional space (determined by the number of genes in an individual) than the objective space, after which the prediction can become harder and less accurate. In spite of this prevalent choice, information extracted from the objective space appears to be more reliable and consistent for establishing POF movement trends [19]. The front movements in the objective space tend to occur toward the Pareto front through the insertion of new non-dominated solutions. Hence, promising population movements can be expected if we can determine individuals placed on a next-step predicted front.

The discussion above has directed our attention to prediction-based methods; thus, we propose the dynamic non-dominated sorting differential evolution with prediction in the objective space (DOSP-NSDE). This dynamic algorithm provides guidance for population evolution using the objective space prediction (OSP) strategy [19]. OSP estimates the progress of the front in the objective space and then finds individuals in the decision space that fit in the predicted front. A step of improvement of the front is determined, and the population is evolved to reach it. The OSP strategy is used to track and predict the POF (applied as a CRM) using the past sequence of locations of the Pareto front as determined by the algorithm during a number of previous time-steps. Moreover, OSP is used to accelerate the convergence of the algorithm before a new environment modification. When OSP is employed within the evolutionary process, it considers useful information from the past consecutive generations. DOSP-NSDE employs a criterion based on the approximated hypervolume (HV) metric to trigger OSP in suitable generations. If the HV-based condition is not satisfied, the algorithm carries on the differential evolution and polynomial mutation operators, and a k-nearest neighbor-based diversity mechanism is used to balance convergence and diversity.

The main contributions of the DOSP-NSDE are as follows:

• Unlike most existing DMOEAs, DOSP-NSDE takes advantage of probabilistic methods to forecast both the evolutionary process and the POF location in a new environment.

• DOSP-NSDE uses a criterion based on the approximated HV. The criterion triggers probabilistic modeling in the most suitable generations of the evolutionary process to accelerate the algorithm convergence between changes. The HV metric was chosen because most real problems have an unknown POF, and this metric correlates with diversity [20].

• The OSP strategy extracts information from both the decision and the objective spaces. The statistical information from the objective space can increase the convergence speed, with regard to generations, because such data effectively captures the probabilistic behavior of the front movement. The information yielded from the decision space creates a population with more scattered solutions in the neighborhood of the forecasted front.

• DOSP-NSDE considers population distribution information and local information for each individual. Population distribution information is considered using probabilistic modeling. Each individual’s local information is considered using the variation operators.

• DOSP-NSDE is used to solve a real-world DMOP.

• Our study also provides a framework to improve the performance of dynamic algorithms through inclusion of the OSP strategy into DMOEAs as a CRM or as part of the static optimization process between changes.

Several experiments were carried out to compare DOSP-NSDE with other state-of-the-art DMOEAs for 28 unconstrained test functions (DF1–14, FDA1–5, dMOP1, 3, and UDF1–7). The results show that DOSP-NSDE is competitive, according to three performance indicators used in the comparisons.

The rest of this paper is organized as follows. Section 2 presents the background of this study, and Section 3 describes the proposed algorithm: DOSP-NSDE. Section 4 introduces the test instances, performance metrics, and experimental results. Finally, Section 5 outlines some conclusions.