Abstract
Homotopy methods are powerful tools for solving nonlinear programming. Their global convergence can be generally established under conditions of the nonemptiness and boundness of the interior of the feasible set, the Positive Linear Independent Constraint Qualification (PLICQ), which is equivalent to the Mangasarian-Fromovitz Constraint Qualification (MFCQ), and the normal cone condition. This paper provides a comparison of the existing normal cone conditions used in homotopy methods for solving inequality constrained nonlinear programming.
1. Introduction
In this paper, we consider the following inequality constrained nonlinear programming (ICNLP) problem: where is the variable, and , , are three times continuously differentiable.
Over the past few decades, the theory, algorithms, and applications of nonlinear programming have been rapidly developed, and many numerical methods have been proposed, such as augmented Lagrangian methods, sequential quadratic programming methods, reduced gradient methods, interior point methods, and homotopy methods. Homotopy methods are powerful numerical methods for solving many nonlinear problems. The primary advantage of homotopy methods is that their global convergence can be established under fairly weak assumptions, and the starting points can be chosen rather freely. A comprehensive introduction of the homotopy methods can be found in, e.g., the books [1, 2]. The first homotopy method for solving nonlinear programming was proposed for general convex programming in [3]. Among these available homotopy methods for solving nonlinear programming, most of them are designed for solving nonconvex ICNLP problems.
In 1995, the combined homotopy interior point (CHIP) method was proposed for solving nonconvex ICNLP problems in [4, 5]. Hereafter, many modified CHIP methods have been proposed for nonconvex ICNLP problems. The global convergence of these homotopy methods can be generally established under three conditions on the original problem: the nonemptiness and boundedness of the interior of the feasible set; the Positive Linear Independent Constraint Qualification (PLICQ), which is equivalent to the Mangasarian-Fromovitz Constraint Qualification (MFCQ) (see [6]); and one type of normal cone conditions, which guarantees the boundedness of the homotopy path near the starting hyperplane. It is well known that the first two conditions are generally used in numerical methods for solving nonlinear programming. The normal cone conditions are generalization of the convexity of the feasible set and extend these homotopy methods from convex programming to nonconvex programming. In addition, a probability-one homotopy method was also proposed for solving nonconvex ICNLP problems in [7]; its global convergence was established under the nonemptiness and boundedness of a parametrized feasible set, Arrow-Hurwicz-Uzawa constraint qualification, and that the homotopy path does not go to infinity near the starting hyperplane. In recent years, these homotopy methods have been extended to fixed point problems, variational inequalities, semidefinite programming, multiobjective programming, constrained sequential minimax problems, and so on.
In this paper, we present the typical normal cone conditions for homotopy methods for solving ICNLP problems, along with the corresponding homotopy maps and global convergence. We give a comparison of four normal cone conditions, including the normal cone condition, the quasinormal cone condition, the pseudocone condition, and the weak normal cone condition. Their relations are discussed in detail for the first time. Some typical nonconvex sets are presented. The comparison can help us to identify features of these normal cone conditions and the corresponding homotopy methods and may motivate us to give some improved homotopy methods for specialized nonconvex programming.
To obtain our results, we conclude this section with some notations. Throughout this paper, represents the feasible set of the ICNLP problem (1); denotes the interior of ; means the boundary of . represents the active index set of inequality constraints at . and denote the nonnegative and positive quadrant of , respectively. indicates the identity matrix. denotes the Euclidean norm. For a function , is the inverse of the set ; the matrix , whose th element is , is the transpose of the Jacobian of .
Assumption 1. is nonempty and bounded.
Assumption 2. The Positive Linear Independent Constraint Qualification holds: for any , , ,
2. Normal Cone Conditions and Homotopy Methods for ICNLP Problems
There exist four typical normal cone conditions in homotopy methods for solving ICNLP problems in literatures; related results will be introduced in this section.
For the nonconvex ICNLP problem (1), the first homotopy method, called the combined homotopy interior point (CHIP) method, was proposed in [4, 5]. The combined homotopy map is constructed as where , , and are diagonal matrices with the th diagonal elements and for , respectively. Under Assumption 1, the assumption that has a full column rank for any , which can be replaced by Assumption 2, and the assumption that satisfies the normal cone condition (see Definition 3), for almost all , the zero point set of (3) defines a smooth homotopy path , which starts from and approaches to the hyperplane . For any limit point of , is a KKT point of the problem (1), is the corresponding Lagrange multiplier.
Definition 3. (normal cone condition (NCC), see [4, 5]). For any , if then, is said to satisfy the NCC.
According to Definition 3, the NCC means that, for any , the set does not intersect with the cone . Moreover, if , , are convex, then satisfies the normal cone condition; however, the reciprocal implication is not true, a typical counterexample is (see Figure 1). In [8], a modified CHIP method was presented. The homotopy map is defined as where , , and is a positive independent map with respect to (see Definition 4). Under Assumption 1, the assumption that are linear independent for any , which can be also replaced by Assumption 2, and the assumption that satisfies the quasinormal cone condition related to the positive independent map (see Definition 5), for almost all , the global convergence of a smooth homotopy path starting from and approaching to the hyperplane can be established; then, a KKT point with the corresponding Lagrange multiplier can be obtained.
Definition 4 (positive independent map, see [8]). If there exist smooth maps for, such that, for any , then, is said to be a positive independent map with respect to .
Definition 5 (quasinormal cone condition (QNCC) [8]). If there exists a smooth positive independent map with respect to such that, for any ,
then, is said to satisfy the QNCC related to .
According to Definition 5, the QNCC means that, for any , the set does not intersect with the cone .
In [9], another modified CHIP method was proposed with the homotopy map
where , , and is a consistent hair map (see Definition 7). Similarly to the homotopy methods in [1, 6], under Assumption 1, the assumption that has a full column rank for any , and the assumption that satisfies the pseudocone condition with respect to the consistent hair map (see Definition 8), the global convergence can be established.
Definition 6 (hair map, see [9]). For , the map is said to be a hair map of , if (1) and for any (2)For any ,
Definition 7 (consistent hair map, see [9]). The map is said to be a consistent hair map of , if for any , (1) is a hair map of for (2) imply ;(3) implies
Definition 8 (pseudocone condition (PCC), see [9]). If there exists a consistent hair map of , such that for any ,
then, is said to satisfy the PCC with respect to .
According to Definition 8, the PCC means that, for any , the set does not intersect with the set .
In [10], using the following aggregate function introduced in [11, 12],
where is the smoothing parameter, the parameter , an aggregate constraint homotopy interior point method was proposed. The aggregate constraint homotopy map is defined as
where , , and is an opened subset of . Under Assumption 1, the assumption that has a full column rank for any , and the assumption that satisfies the weak normal cone condition with respect to (see Definition 9), then, there exists a such that for any , for , for almost all , the zero point set of (12) defines a smooth homotopy path , which starts from and approaches to the hyperplane . Moreover, let () be any limit point of , and (where is a limit point of ) for , then is finite, is a KKT point of (1), and is the corresponding Lagrangian multiplier.
Definition 9 (weak normal cone condition (WNCC) [10]). If there exists an open subset such that, for any ,
then, is said to satisfy the WNCC with respect to .
According to Definition 9, the WNCC means that there exists an open subset such that for any , the set does not intersect with the cone .
3. A Comparison of the Four Typical Normal Cone Conditions
In this section, for the first time, we study the relations of the four typical normal cone conditions introduced in Section 2, and some typical nonconvex sets are introduced.
Proposition 10. If the PLICQ holds, the NCC implies the QNCC and PCC.
Proof. Suppose that the PLICQ holds, then is a positive independent map with respect to by Definition 4, and is a consistent hair map of by Definition 7. Then, satisfies the QNCC with respect to by Definition 5, and the PCC with respect to by Definition 8.
Counterexample 3.2. The nonconvex set (see Figure 2) .
The set in Counterexample 3.2 satisfies the NCC. However, for , we have , and , which means that the PLICQ does not hold at . Hence, there does not exist a positive independent map with respect to by Definition 4, and the consistent hair map of by the second item of Definition 7. Therefore, the PLICQ is necessary in Proposition 10.
Proposition 11. The NCC implies the WNCC.
Proof. By Definition 3 and Definition 9, if satisfies the NCC, satisfies the WNCC with respect to any open subset .
Counterexample 3.4. The nonconvex set (see Figure 3) , where The set in Counterexample 3.4 satisfies the QNCC with respect to the positive independent map , the PCC with respect to the consistent hair map , and the WNCC with respect to any open subset of . For with and , we have with , which contradicts (4). Then, we know that the NCC does not hold. Therefore, we have the following result.
Remark 12. Any of the QNCC, PCC, and WNCC does not imply the NCC.
Proposition 13. The QNCC implies the PCC.
Proof. If satisfies the QNCC with respect to the positive independent map , we know that is a consistent hair map of by Definition 7. Then, satisfies the PCC with respect to the consistent hair map by Definition 8.
Counterexample 3.7. The nonconvex set (see Figure 4) , where
For with , and any positive independent map , there exists a such that . Hence, the QNCC does not hold. Define the consistent hair map as where satisfying which means that is continuously differentiable in . For any , we have , and , and hence which means that is monotone decreasing for . By using , for any with and , we define then, we know that is continuously differentiable in , and for any given , is monotone decreasing for . For ,
For with , we have that (10) holds by the definition of . For any , we know that (4) holds. Hence, by Proposition 10 and the definitions of , , we obtain that (10) also holds.
For with , and any , , we know
Then, we have and hence, . For with , and any , , we know
Then, we have and hence, . For with , and any , , we know
Then, we have and hence, . For with , and any , , we know
Then, we have and hence, . For with , and any , , we know
Then, we have and hence, . For with , and any , , we know
Then, we have and hence, . Therefore, the PCC holds with respect to the consistent hair map , and hence we have the following result.
Remark 14. The PCC does not imply the QNCC.
Counterexample 3.9. The nonconvex set (see Figure 5) , where
The set in Counterexample 3.9 satisfies the QNCC with respect to the positive independent map , and the PCC with respect to the hair map . For any , let for , for . For convenience, we rewrite as with , , , , , .
For any , , and with , we have
For any , let , with , we have
For any and , and reduce to and , respectively. By the continuity of and on , , coming from and , there exists an such that , which means
Then, for with , , we have
Analogously, for any , and , there exist an with and a such that
Therefore, the WNCC does not hold, and hence we have the following result.
Remark 15. The QNCC or PCC does not imply the WNCC.
Counterexample 3.11. The nonconvex set (see Figure 6) , where
The set in Counterexample 3.11 satisfies the WNCC with respect to any open subset of . For with and , and any hair map satisfying (10), we have and by Definition 6. Hence, there exists a such that . Then, we know and with , which contradicts the second item of Definition 7. Therefore, there does not exist a consistent hair map with respect to to satisfy the PCC, and the QNCC also does not hold by Proposition 13. Then, we obtain the following result.
Remark 16. The WNCC does not imply the QNCC or PCC.
In conclusion, we show the relations among the four typical normal cone conditions in Figure 7. It is noted that there still exist many nonconvex sets satisfying none of existing normal cone conditions, such as the nonconvex set in Counterexample 3.13.
Counterexample 3.13. The nonconvex set (see Figure 8) .
4. The Global Convergence of the Homotopy Methods for Solving ICNLP
As shown in Section 2, the global convergence of these homotopy methods can be established under three conditions. In this section, we present some comments for these conditions.
The boundedness of ensures that the variable in the homotopy path keeps bounded. For Example 4.1, is unbounded, we have with and for the starting point with in the homotopy path defined by the CHIP method. Since the real-world ICNLP problems generally have the optimal solutions at finity, the variable in the homotopy path always keeps bounded even if is unbounded. For Example 4.2, is unbounded, but we have with and for the starting point with in the homotopy path defined by the CHIP method.
Example 4.1.
Example 4.2. The PLICQ (MFCQ) is the most widely used constraint qualification for the ICNLP problems, it ensures that the variable in the homotopy path keeps bounded. For Example 4.3, the PLICQ does not hold at , we have that is not a KKT point, and for the starting point with , and , and for the starting point with in the homotopy path defined by the CHIP method.
Example 4.3. The normal cone conditions and the PLICQ ensure that the variable in the homotopy path keeps bounded when the homotopy path tends to the hyperplane . Although our numerical experiences show that these homotopy methods always globally converge even if the normal cone conditions do not hold, the normal cone conditions are important to establish the global convergence of these homotopy methods in theory. For Example 4.4, the normal cone conditions do not hold for any , but we have , and for all tested starting points with in the homotopy path defined by the CHIP method.
Example 4.4.
5. Conclusion
In this paper, we provide a comparison of four typical normal cone conditions used in homotopy methods for solving inequality constrained nonlinear programming. The NCC holds for convex sets and a class of nonconvex sets. The WNCC, QNCC, and PCC are more weak than the NCC. However, the NCC is more convenient to use than the WNCC, QNCC and PCC. For the feasible set satisfying the QNCC or PCC, some auxiliary maps should be constructed in the homotopy maps. For the feasible set satisfying the WNCC, the starting point for the homotopy methods should be chosen from a special open subset of the feasible set. On the other hand, except for few special nonconvex sets, there does not exist a general way of checking a set to satisfy the NCC or construct the open subset, the positive independent map and the consistent hair map for the WNCC, QNCC, and PCC, respectively.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Acknowledgments
This study was supported by the National Natural Science Foundation of China (No. 11701350).