Second order optimality on orthogonal Stiefel manifolds

Abstract

The main tool to study a second order optimality problem is the Hessian operator associated to the cost function that defines the optimization problem. By regarding an orthogonal Stiefel manifold as a constraint manifold embedded in an Euclidean space we obtain a concise matrix formula for the Hessian of a cost function defined on such a manifold. We introduce an explicit local frame on an orthogonal Stiefel manifold in order to compute the components of the Hessian matrix of a cost function. We present some important properties of this frame. As applications we rediscover second order conditions of optimality for the Procrustes and the Penrose regression problems (previously found in the literature). For the Brockett problem we find necessary and sufficient conditions for a critical point to be a local minimum. Since many optimization problems are approached using numerical algorithms, we give an explicit description of the Newton algorithm on orthogonal Stiefel manifolds.

Introduction

Optimization problems on Stiefel manifolds appear in important applications such as statistical analysis of data [16], blind signal separation [12], distance metric learning [15], among many other problems.

In Section 2 we regard an orthogonal Stiefel manifold as a preimage of a regular value for a set of constraint functions. We adapt the method presented in the papers [4] and [5] (the so called embedded gradient vector field method) to the particular case of Stiefel manifolds. We embed the Stiefel manifold $S{t}_p^n$ in the larger Euclidean space ${\mathcal{M}}_{n\times p}(\mathbb{R})$, in order to take advantage of the simpler geometry of this Euclidean space. This setting allows us to present necessary and sufficient conditions for critical points of a cost function defined on a Stiefel manifold and a formula for the Hessian of this cost function in a concise matrix form. This formula is important in the study of the second order optimality.

In order to explicitly compute the components of the Hessian matrix of a cost function, in Section 3 we introduce an explicit local frame for an orthogonal Stiefel manifold and we present some important properties of this local frame. We determine the components of the Hessian matrices of the constraint functions in this frame.

In the last section we apply the results of the previous sections to some important problems arising from practical applications. For the Procrustes and the Penrose regression problems we obtain second order conditions, which have been previously presented in [7] using a different approach, which involves the projected Hessian. For the Brockett problem, see [1], we find necessary and sufficient conditions for a critical point of the cost function to be a local minimum. As an example, for a particular Brockett cost function defined on the orthogonal Stiefel manifold $S{t}_2^4$ we give a list of the critical points and we completely characterize them.

In many cases optimization problems are approached using numerical methods on manifolds. The Newton algorithm is a type of algorithm that uses the second order information about a cost function. We give an explicit description of the Newton algorithm in the case of an orthogonal Stiefel manifold. There exists a rich literature that deals with the construction of Newton algorithm on manifolds, see [1], [11], [13], and [17].