An axiomatic framework for influence diagram computation with partially ordered preferences

https://doi.org/10.1016/j.ijar.2020.06.011Get rights and content

Abstract

This paper presents an axiomatic framework for influence diagram computation, which allows reasoning with partially ordered values of utility. We show how an algorithm based on sequential variable elimination can be used to compute the set of maximal values of expected utility (up to an equivalence relation). Formalisms subsumed by the framework include decision making under uncertainty based on multi-objective utility, or on interval-valued utilities, as well as a more qualitative decision theory based on order of magnitude probabilities and utilities. Consequently, we also introduce the order of magnitude influence diagram to model and solve partially specified sequential decision problems when only qualitative (or imprecise) information is available.

Introduction

Influence diagrams have been widely used for the past three decades as a graphical model to formulate and solve decision problems under uncertainty. The standard formulation of an influence diagram consists of two types of information: qualitative information that defines the structure of the problem and quantitative information (also known as parametric structure) that, together with the former, defines the model. The qualitative information includes the set of (discrete) chance variables, where the outcome is determined randomly based on the values assigned to other variables, describing the set of possible world configurations, the set of decision variables, which the decision maker can choose the value of, based on observations on some other variables, as well as the dependencies between the two sets of variables. The parametric structure is composed of the conditional probability distributions associated with each of the chance variables (thus representing uncertainty like in Bayesian networks), as well as a collection of utility functions, whose sum describes the overall value of an outcome, and thus is used to represent the preferences of the decision maker. A policy defines which actions to take for each decision variable, given the available information, and has a corresponding expected utility. The solution to an influence diagram is an optimal policy that maximizes the expected utility and therefore depends on both types of qualitative and quantitative information.

In general, actions can lead to many different kinds of consequences, for example, financial gain/loss, risk to health, effect on the environment or gain/loss to reputation. It may not be possible to map the various potential consequences of a set of actions to the same scale of utility in a way that avoids making essentially arbitrary choices. It is thus natural to consider in this case notions of multi-attribute/objective utility (including imprecise tradeoffs between utility objectives), where utility values are only partially ordered. Quite often, we may have precise knowledge of the qualitative information but only very rough (or imprecise) estimates of the quantitative parameters. In such cases, the standard solution techniques cannot be applied directly, unless the missing or imprecise information is accounted for.

In this paper, we consider decision making under uncertainty using influence diagrams, but where we allow more general notions of uncertainty than probability, and more general notions of utility functions, which, in particular, allow utility values to be only partially ordered. We next highlight the major contributions of the paper, as follows:

  • •

    We construct an axiomatic framework, listing properties of a formalism that allow maximal (generalized) expected utility to be computed by sequential elimination of all the variables.

  • •

    We prove formally that the set of utility values computed by a sequential variable elimination algorithm within this framework is equivalent to the set of maximal values of expected utility.

  • •

    We discuss in detail and provide numerical experiments for order of magnitude influence diagrams, a formalism that can be used to model and solve partially specified sequential decision making problems when only qualitative (or imprecise) information is available.

In general terms, variable elimination algorithms can be viewed as follows. We have a collection Θ of functions, where each function in Θ only involves a small number of variables. In the case of influence diagram computation, Θ contains both probability functions and utility functions. Θ is used as a compact representation (or decomposition) of a function ⨂Θ on all the variables, equaling a combination of all the functions in Θ. For example, in a Bayesian network, Θ consists of a collection of conditional probability functions and ⨂Θ is the joint probability distribution. Since ⨂Θ involves all the variables, it will be a huge object to represent explicitly.

What we want to compute is the result of marginalizing out (eliminating) all the variables from ⨂Θ. For a standard influence diagram we have both chance and decision variables, and we eliminate chance variables with a sum operator, and decision variables with a max operator. We can compute the maximum expected utility by performing a sequence of sum and max eliminations to ⨂Θ. Performing combinations leads to functions involving larger sets of variables, which is expensive in terms of both computational cost and time. One therefore would like to delay performing computations where possible. Thus, when eliminating a variable X, with, for example, the ∑ operator, one transforms Θ to a collection Θ′, including only functions that don't involve X, such that ∑X(⨂Θ)=⨂Θ′. Crucially, the functions in Θ that don't involve X are left unchanged, so still appear in Θ′. However, these variable elimination computations are not trivial when generalized forms of probability and utility are considered in our proposed framework.

We begin with a standard example of influence diagrams, and show in detail how a variable elimination algorithm computes expected utility (Section 2). We will extend this algorithm for a wide range of formalisms. More precisely, we consider generalized forms of probability values and utility values, and associated generalized uncertainty and utility functions, and consider what properties are needed for the algorithm to be correct (Section 3). We describe some formalisms that satisfy the axioms, including interval-valued utility, multi-objective utility and order of magnitude probability and utility (Section 3.2).

We then show how both chance variables and decision variables are eliminated (Section 4). To eliminate a variable involves replacing the current collection of generalized probability and utility functions with a new set whose combination is equivalent to the marginal of the initial set. We continue by defining generalized influence diagram systems (Section 5) and proving that one can iteratively eliminate all the variables to obtain the maximum value of expected utility for the case where utility values are totally ordered (Section 6).

We go on to consider the case where values of utility are only partially ordered (Section 7); then there will typically not be a unique maximal value of utility, but a set of them. To compute this set we need to perform operations on sets of utility values. We therefore show how to compute, by sequential variable elimination, a set of utility values that is equivalent, in a natural sense, to the set of maximal values of expected utility (Section 8). This therefore allows influence diagram computation for any formalisms satisfying the axioms.

Finally, we take a closer look at the case when only rough (or imprecise) estimates of the decision model's parameters are available. We describe in detail the order of magnitude influence diagram which can be viewed as a qualitative theory for influence diagrams in which such partially specified sequential decision problems can be modeled and solved (Section 9). More specifically, the model involves an order of magnitude representation of the probabilities and utilities, and thus allows the decision maker to specify partially ordered preferences via finite sets of utility values. To compute the set of maximal expected utility values we show how to use a variable elimination algorithm that performs efficient operations on sets of utility values involving at most two elements.

Following preliminaries and notations (Section 2), Section 3 introduces the notions of generalized uncertainty and utility values while Section 4 shows how to eliminate chance and decision variables from these structures. In Section 5 we define generalized influence diagram systems and show in Section 6 how to compute the maximum value of expected utility. Section 7 and Section 8 present the technical results for the case of partially ordered utility values. Section 9 is dedicated to the order of magnitude influence diagram formalism, Section 10 overviews related work, while Section 11 provides concluding remarks and directions of future work.

The Appendix contains the proofs and auxiliary material required for the main results. The paper extends earlier work of the authors which was published in [1], [2].

Section snippets

Preliminaries

We first define some notation that we will use throughout the paper, and then go through a standard influence diagram example to illustrate the computational of expected utility using iterative variable elimination. In later sections we extend this solution method for more general forms of influence diagram.

Generalized uncertainty and utility values

In standard influence diagrams probability potentials take non-negative real values, and utility functions take real values. In this section, we introduce the notions of generalized probability and utility values along with combination and marginalization operators. Most of the properties of positive reals are still assumed for generalized probability values, the most important exception being that we do not assume a cancellation property for addition (i.e., we do not assume that a+b=a+c only

Elimination of variables

In this section we will consider an u.u.v. structure U, and a pair (Φ,Ψ), where Φ (respectively, Ψ) is a collection of U-uncertainty (respectively, U-utility) functions over X∪D. We assume that each variable in X∪D is involved in some element of Φ∪Ψ (if not, then we can delete any non-involved variables).

A pair (Φ,Ψ) will be considered as a compact representation of the (overall) utility function ∏P∈ΦP×∑U∈ΨU. We write ⨂(Φ,Ψ)=∏P∈ΦP×∑U∈ΨU.

We want to compute a generalized expected utility

Influence diagram systems

Theorem 1, Theorem 2 show how to eliminate chance and decision variables, respectively, using marginalization operators ∑X and ⋁D on a pair (Φ,Ψ). We would like to iteratively apply these to eliminate all variables, leading to the maximum expected utility. However, Theorem 2 requires that, before eliminating decision variable D, the current set of uncertainty functions does not depend on D. To ensure that this condition holds for the iterative computation, we require restrictions on the

Elimination of all variables

In Section 4 we showed how to eliminate both chance variables and decision variables. In Section 6.1 we show how this can be iterated in order to eliminate all variables for an influence diagram system, enabling computation of maximum expected utility. In Section 6.2 we demonstrate how this algorithmic approach can be implemented within a bucket elimination structure, and illustrate this in Section 6.3 with our running example. Section 6.4 considers the complexity of the algorithm.

Sets of utility values for partially ordered case

Let U=〈Q,+Q,×Q,0Q,1,U,+U,0U,×〉 be an uncertainty-utility values structure and let I=〈G,(Φ,Ψ)〉 be an ID-system over U. Suppose that the set of utility values U is only partially ordered, by relation ⪰. For finite set A of utility values we can consider the set of maximal ones max⪰⁡(A), consisting of all u∈A such that there does not exist a different element v∈A with v⪰u. We are interested in policies that generate a maximal value of expected utility, i.e., values of utility in max⪰⁡{EUπ:policiesπ

Variable elimination based on sets of utilities

In this section we show how to use variable elimination to compute, up to equivalence, the set of maximal values of expected utility over all policies, for the partially ordered case. Given a u.u.v. structure U, we make use of extended utility functions (that assign a set of utility values rather than a single utility value), i.e., using the induced weak u.u.v. structure U⁎, which satisfies the properties of a u.u.v. structure up to equivalence (see Section 7, especially, Proposition 9).

Order of magnitude influence diagrams

In this section, we describe in detail the Order of Magnitude Influence Diagram (OOM-ID), a more qualitative model that can be used for modeling and solving partially specified sequential decision problems, especially when the quantitative parameters are specified rather poorly and only rough (or imprecise) estimates are available. The model uses an order of magnitude representation of probabilities and utilities, and therefore allows the decision maker to specify partially ordered preferences

Related work

The variable elimination approach we use in this paper is based on that for standard influence diagrams, in particular, in Jensen et al. [11] and Dechter [12], which builds on previous work such as Shachter and Peot [19], and Shenoy [20]. The work that is closest in spirit to the current work is that by Pralet, Schiex and Verfaillie [21], who also consider an axiomatic framework for generalized influence diagrams (and other sequential decision making problems), involving a form of generalized

Conclusion

In this paper we consider decision making under uncertainty using influence diagrams, but where we allow more general notions of uncertainty than probability and more general notions of utility functions, which, in particular, allow utility values to be only partially ordered. We present an axiomatic framework and list the properties of a formalism that allows maximal (generalized) expected utility to be computed by a sequential variable elimination algorithm. Example formalisms that satisfy

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (39)

  • R. Cabanas et al.

    Evaluating interval-valued influence diagrams

    Int. J. Approx. Reason.

    (2017)
  • R. Marinescu et al.

    AND/OR branch-and-bound search for combinatorial optimization in graphical models

    Artif. Intell.

    (2009)
  • R. Marinescu et al.

    Memory intensive AND/OR search for combinatorial optimization in graphical models

    Artif. Intell.

    (2009)
  • R. Marinescu et al.

    Order-of-magnitude influence diagrams

  • N. Wilson et al.

    An axiomatic framework for influence diagram computation with partially ordered utilities

  • H. Raiffa

    Decision Analysis

    (1968)
  • R. Howard et al.

    Influence diagrams

  • T. Nielsen et al.

    Bayesian Networks and Decision Graphs

    (2007)
  • F. Chu et al.

    Great expectations. Part I: On the customizability of generalized expected utility

    • Cited by (0)

    View full text
    © 2020 Elsevier Inc. All rights reserved.