Deep lifted decision rules for two-stage adaptive optimization problems

https://doi.org/10.1016/j.compchemeng.2022.107661Get rights and content

Highlights

  • Flexible piecewise linear lifting of uncertainty through neural network activation.

  • Improved decision rule approximation of adaptive decisions in adaptive optimization.

  • Simultaneous optimization of the network parameters and decision rule coefficients.

Abstract

This paper presents a novel method to generate flexible piecewise linear decision rules for two-stage adaptive optimization problems. Borrowing the idea of a neural network, the lifting network consists of multiple processing layers that enable the construction of more flexible piecewise linear functions used in decision rules whose quality and flexibility is superior to linear decision rules and axially-lifted ones. Two solution methods are proposed to optimize the weights and the decision rule approximation quality: a derivative-free method via an evolutionary algorithm and a derivative-based method using approximate derivative information. For the latter method, we suggest local-search heuristics that scale well and reduce the computational time by several folds while offering similar solution quality. We illustrate the flexibility of the proposed method in comparison to linear and axial piecewise linear decision rules via a transportation and an airlift operations scheduling problem.

Introduction

Multi-stage adaptive optimization is a framework to handle sequential decision-making problems that arise in numerous business and engineering disciplines. It has elicited growing attention due to its ability to manage uncertainty in the decision making process. Arguably the three most popular modeling paradigms for multi-stage adaptive optimization are stochastic programming (SP) (Shapiro et al., 2009), robust optimization (RO) (Ben-Tal et al., 2009), and stochastic dynamic programming (SDP) (Ross, 2014). SP is widely used in operations research applications, SDP has roots in control theory and RO is popular in engineering and operations research problems where the failure to manage all uncertain events can lead to severe consequences.

As described by Georghiou et al. (2019), the decision rule approach to decision making under uncertainty is quite expressive and flexible, encompassing multi-stage stochastic problems, chance-constrained problems, and adaptive robust optimization problems. When coupled with closed polyhedral or conic uncertainty sets, the approach inherits the tractability of robust optimization, without sacrificing the essential dynamics of a multi-stage optimization problem under uncertainty. It can therefore be seen as a hybrid approach, although RO is its nearest neighbour methodologically.

A decision rule is nothing more than a function mapping realized uncertain parameters to implementable decisions. In the reinforcement learning community, a decision rule is known more generally as a policy, “any function that returns an action given a state” (Powell, 2011, p.233). Multi-stage adaptive optimization categorizes decisions into non-adaptive and adaptive groups. The former, also known as “here-and-now” or “first-stage” decisions, are independent of any future uncertain information and are made at the outset of the problem. In contrast, the latter, also known as “wait-and-see” or “second-stage” decisions, are implemented gradually in response to the revealed uncertain information. Since, at the outset of a multi-stage adaptive optimization problem, we do not know the realizations of future uncertain parameters, a decision rule prescribes a recourse decision to take once the history of the uncertain parameters has been realized up to and including that decision stage.

One of the main limitations of decision rule-based methods, and ultimately a key motivation for this work, is that they require the user to specify the structural form of the approximate policy before optimizing. To date, practitioners must propose a structural form - with linear, quadratic, and piecewise linear decision rules being by far the most popular - and then empirically test its suitability. Unfortunately, it is not possible, in general, to deduce an optimal policy’s structure a priori. This conundrum brings us to the driving question for our research: Can we relax the requirement of having the user specify a fixed policy structure and instead allow this structure to be as flexible as possible via a sequence of simple operations?

To answer this question, we turn to basic concepts used to train deep neural networks (DNNs), which have ultimately helped popularize deep learning. At its core, a DNN is a functional approximator. Given a set of inputs, the user would like to determine outputs that best match the existing sample output. To determine what could be a complex approximation, a DNN performs a succession of simple operations, namely by passing input through a feedforward network with simple activation functions and optimizing activation function weights in the process. Most importantly, the user does not need to assume the approximate function’s structure a priori; it is learned during the training process. In this work, we attempt to demonstrate that cross-pollinating deep learning concepts (which are used for learning) with decision rule concepts (which are used for stochastic optimization) can lead to improved solution methods for stochastic adaptive optimization problems.

Initially proposed by Garstka and Wets (1974), if not early, decision rule-based methods came to prominence as a viable approach for solving stochastic optimization problems only after the seminal paper of Ben-Tal et al. (2004). They assumed adaptive policies are linearly dependent on the uncertain parameters to derive a linear deterministic counterpart, under a polytopic uncertainty set, of a multi-stage linear adaptive optimization problem. Due to their favorable modeling characteristics, linear decision rules have been implemented in several fields such as supply chain (Ben-Tal et al., 2005), power systems management (Zugno et al., 2016) and scheduling (Zhang, Morari, Grossmann, Sundaramoorthy, Pinto, 2016, Rahal, Li, Papageorgiou, 2020), to name few.

Ben-Tal and Den Hertog (2011) introduced a more complex approximation which defines the adaptive policies as quadratic functions of the uncertain parameters. The derived deterministic counterpart, under an ellipsoidal uncertainty set, is a second-order cone programming problem. Bertsimas et al. (2011) proposed polynomial decision rules in multi-stage robust dynamic problems. The increase in the complexity/flexibility of the adaptive policies comes at the expense of a more complex deterministic counterpart which, under an intersection of convex uncertainty sets, is a semidefinite programming problem. Bampou and Kuhn (2011) similarly proposed polynomial decision rules in the context of stochastic programming.

The well-known trade-off between solution quality and computational cost is evident here. Increasing an adaptive policy flexibility by simply increasing its complexity is not an attractive direction. An alternative paradigm is to refine or transform the uncertainty set to improve the quality of less complex policies. In this regard, Chen and Zhang (2009) proposed an extended linear decision rule using an extended uncertainty set defined via the positive and negative perturbations of the original uncertain parameters. The linear approximation over the extended uncertainty space projects into a piecewise linear approximation over the original uncertainty set with two linear segments. Georghiou et al. (2015) introduced lifted decision rules which are based on a one-to-one correspondence between a set of linear functions in the lifted uncertainty space and a family of piecewise linear functions in the original uncertainty space. Rahal et al. (2020b) devised hybrid lifted decision rules which emphasize higher refinement of the lifted uncertainty space in the short- rather than the long-term time horizon.

Hanasusanto et al. (2015) partitioned the uncertainty space to compute k-adaptive linear policies. The optimal implemented policy depends on the value of the observed uncertainty. Ben-Tal et al. (2020) proposed an approximate piecewise-linear policy for two-stage robust linear problems by approximating the uncertainty set via a dominated simplex. The proposed approximation method scales well and is computationally efficient, however the constructed policy is not guaranteed to outperform a linear decision rule approximation for budget uncertainty sets or an intersection of thereof. Yanıkoğlu et al. (2019) surveyed decision rule theory, applications and methodologies in the context of robust optimization. Computational tools such as ROME (Goh and Sim, 2011), JuMPeR (Dunning, 2016) and RSOME (Chen et al., 2020) have been developed in response to the wide implementation of decision rule-based methods for optimization under uncertainty.

Reinforcement learning, also known as approximate dynamic programming, is a common solution method that utilizes, in one of its forms, the prowess of an artificial neural network in approximating multi-stage stochastic decision problems (Lee et al., 2018). The problem is typically modeled as a Markov decision process after which a designed network approximates complex functions to output predictions, after being trained, based on a given input (i.e., observed uncertainty) (Lee and Lee, 2006). Similarly, Han (2016) introduced deep learning approximations for multi-stage stochastic control problems. The network architecture is composed of interconnected sub-networks; each sub-network has multiple hidden layers. The authors claimed that their approach overcomes the “curse of dimensionality” in solving high-dimensional problems. Huré et al. (2018) developed convergence analysis for a proposed deep neural network algorithm for stochastic control problems followed by a set of numerical applications in Bachouch et al. (2018).

Deep learning via a deep neural network (DNN) has been increasingly incorporated in speech recognition, recommender systems, objection detection and other applications (LeCun et al., 2015). Though the focus of this paper is on solving two-stage adaptive stochastic optimization problems, we emphasize the capability of DNNs, through successive applications of simple operations, to extract complex features from big data sets. Each layer consists of an affine transformation followed by a non-linear operator, known as “activation function, at each node. Among others, the rectified linear activation unit (ReLU) is a piecewise linear function with two linear segments and a breakpoint at 0. The slope of the positive and negative ranges are 1 and 0, respectively. The idea of using a sequence of simple operations to construct complex features/structures is a catalyst for this work. Instead of using a ReLU as the non-linear operator, we use the lifting operator introduced by Georghiou et al. (2015). Further, we introduce different solution methodologies to optimize/train our proposed network to devise the optimal decision rule structure.

The contributions of this paper are:

  • 1.

    We propose a “deep lifting” network to generate complex functions to approximate the adaptive policies in two-stage stochastic adaptive optimization problems. Taking the original uncertain parameters as inputs, the network consists of multiple processing layers that enable the construction of complex lifted parameters. Each layer consists of two operations: an affine transformation followed by a lifting operator at each node. The coupling was introduced by Georghiou et al. (2015) and coined as generalized lifting, which is equivalent to a deep lifting network with a single layer. The functions used to define the adaptive policies are linear functions of the uncertain primitive vector, the lifted uncertain parameters in all layers and an intercept. We call the resulting policies “deep lifted decision rules” to reflect the two critical components upon which the approach is founded: (a) the lifting operations applied to the uncertainty set in each layer and (b) the connections to deep neural networks through the succession of simple operations.

  • 2.

    We present two solution methods for two-stage stochastic adaptive optimization problems with fixed cost coefficients, recourse coefficients and right-hand-side uncertainty. In the first method, we optimize the deep lifting network within a black-box optimizer using a differential evolutionary algorithm. In the second method, the solution is obtained using a derivative-based approach via first- and second-order approximate derivatives. We further show the benefits of piecewise linear decision rules (i.e., PLDRs) using a deep lifting network in comparison to linear and axial piecewise linear decision rules.

  • 3.

    We introduce local-search heuristics to optimize deep lifting networks. We empirically verify that the proposed heuristics generate flexible deep lifted decision rules while reducing the computational cost by several folds. We also illustrate that the proposed heuristics scale well with large dimensional instances.

Notation. Throughout the paper, we denote a column (row) vector by using a semicolon (comma) to separate each element. Similarly, a matrix is specified explicitly by listing its elements and using a semicolon to separate each row.

Section snippets

Problem statement

In this work, we address the solution of a two-stage stochastic adaptive optimization problem with fixed cost coefficients and a recourse matrix given in Model (1).minx,y(ξ)cx+E[qy(ξ)]s.t.AxbTx+Dy(ξ)h(ξ)ξΞ

We assume that the uncertainty set Ξ is a well-defined polytope. We also assume a linear uncertainty dependence for the right-hand-side uncertain vector h(ξ)=Hξ. Model (1) is intractable in its general form due to the presence of the semi-infinite constraints required to be satisfied

Limitation of LDRs and axial lifting based PLDRs

In this section, we introduce linear decision rules and illustrate the optimal distribution policies of the transportation problem given by formulation (2). We then restate concepts of lifted decision rules originally introduced by Georghiou et al. (2015) and highlight a possible limitation of this type of decision rule.

Deep lifting

A deep lifting network comprises a sequence of two simple operations: an affine transformation followed by a lifting operator at each node. Combining ideas from deep neural networks and lifted decision rules, we believe that a deep lifting network is capable of generating complex and flexible policies at an attractive computational cost. To be clear, we highlight that the term “deep lifting” is not meant for machine learning. Rather, it describes a lifting strategy used in constructing decision

Solution methods

We propose two solution methods to address the two-layer optimization problem: a derivative-free and a derivative-based method.

Computational experiments

In this section, we illustrate the training of a deep lifting network using a black-box, a traditional gradient descent method and local-search heuristics for a two-stage airlift operations scheduling problem from Ariyawansa and Felt (2004). We investigate the impact of increasing decision variables and uncertainty dimensions on the performance of the solution methods. The lower and upper weight bounds in all experiments are -2 and 2, respectively. In all computational experiments, we assume no

Conclusions and future research directions

The quest to construct flexible decision rules for adaptive policies has attracted growing interests in the stochastic and robust optimization communities since the introduction of affine decision rules. As a part of the ongoing efforts, this work introduced an instrument to devise complex piecewise linear decision rules using a deep lifting network to solve two-stage stochastic optimization problems. The solution involves simultaneous training of the network. In this regard, we proposed

CRediT authorship contribution statement

Said Rahal: Methodology, Software, Writing – original draft. Zukui Li: Conceptualization, Supervision, Writing – review & editing, Funding acquisition. Dimitri J. Papageorgiou: Supervision, Writing – original draft, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors gratefully acknowledge the financial support from the Natural Sciences and Engineering Research Council of Canada (NSERC).

References (38)

  • A. Ben-Tal et al.

    Immunizing Conic Quadratic Optimization Problems Against Implementation Errors

    Technical Report

    (2011)
  • A. Ben-Tal et al.

    Robust optimization

    (2009)
  • A. Ben-Tal et al.

    A tractable approach for designing piecewise affine policies in two-stage adjustable robust optimization

    Math Program

    (2020)
  • A. Ben-Tal et al.

    Retailer-supplier flexible commitments contracts: arobust optimization approach

    Manufacturing & Service Operations Management

    (2005)
  • A. Ben-Tal et al.

    Adjustable robust solutions of uncertain linear programs

    Math Program

    (2004)
  • D. Bertsimas et al.

    A hierarchy of near-optimal policies for multistage adaptive optimization

    IEEE Trans Automat Contr

    (2011)
  • X. Chen et al.

    Uncertain linear programs: extended affinely adjustable robust counterparts

    Oper Res

    (2009)
  • Z. Chen et al.

    Robust stochastic optimization made easy with rsome

    Manage Sci

    (2020)
  • I. Dunning et al.

    Jump: a modeling language for mathematical optimization

    SIAM Rev.

    (2017)
  • Cited by (0)

    View full text