Regression and progression in stochastic domains

https://doi.org/10.1016/j.artint.2020.103247Get rights and content

Abstract

Reasoning about degrees of belief in uncertain dynamic worlds is fundamental to many applications, such as robotics and planning, where actions modify state properties and sensors provide measurements, both of which are prone to noise. With the exception of limited cases such as Gaussian processes over linear phenomena, belief state evolution can be complex and hard to reason with in a general way, especially when the agent has to deal with categorical assertions, incomplete information such as disjunctive knowledge, as well as probabilistic knowledge. Among the many approaches for reasoning about degrees of belief in the presence of noisy sensing and acting, the logical account proposed by Bacchus, Halpern, and Levesque is perhaps the most expressive, allowing for such belief states to be expressed naturally as constraints. While that proposal is powerful, the task of how to plan effectively is not addressed. In fact, at a more fundamental level, the task of projection, that of reasoning about beliefs effectively after acting and sensing, is left entirely open.

To aid planning algorithms, we study the projection problem in this work. In the reasoning about actions literature, there are two main solutions to projection: regression and progression. Both of these have proven enormously useful for the design of logical agents, essentially paving the way for cognitive robotics. Roughly, regression reduces a query about the future to a query about the initial state. Progression, on the other hand, changes the initial state according to the effects of each action and then checks whether the formula holds in the updated state. In this work, we show how both of these generalize in the presence of degrees of belief, noisy acting and sensing. Our results allow for both discrete and continuous probability distributions to be used in the specification of beliefs and dynamics.

Introduction

Reasoning about degrees of belief in uncertain dynamic worlds is fundamental to many applications, such as robotics and planning, where actions modify state properties and sensors provide measurements, both of which are prone to noise. However, there seem to be two disparate paradigms to address this concern, both of which have their limitations. At one extreme, there are logical formalisms, such as the situation calculus [51], [58], which allows us to express strict uncertainty, and exploits regularities in the effects actions have on propositions to describe physical laws compactly. Probabilistic sensor fusion, however, has received less attention here. (Notable exceptions will be discussed in the penultimate section.) At the other extreme, revising beliefs after noisy observations over rich error profiles is effortlessly addressed using probabilistic techniques such as Kalman filtering and Dynamic Bayesian Networks [20], [21]. However, in these frameworks, a complete specification of the dependencies between variables is taken as given, making it difficult to deal with other forms of incomplete knowledge as well as complex actions that shift dependencies between variables in nontrivial ways.

An influential but nevertheless simple proposal by Bacchus, Halpern and Levesque [2], BHL henceforth, was among the first to merge these broad areas in a general way. Their specification is widely applicable because it is not constrained to particular structural assumptions. In a nutshell, they extend the situation calculus language with a provision for specifying the degrees of belief in formulas in the initial state, closely fashioned after intuitions on incorporating probability in modal logics [34], [26]. This then allows incomplete and partial specifications, which might be compatible with one or very many initial distributions and sets of independence assumptions, with beliefs following at a corresponding level of specificity. Moreover, together with a rich action theory, the model not only exhibits Bayesian conditioning [55] (which, then, captures special cases such as Kalman filtering [69]), but also allows flexibility in the ways dependencies and distributions may change over actions.

While that proposal is powerful, the task of how to plan effectively is not addressed. In essence, this would correspond to a flavor of epistemic planning [3] where the state of knowledge, actions and sensing are mixtures of logical and probabilistic assertions. In fact, at a more fundamental level, the task of projection, that of reasoning about beliefs effectively after acting and sensing, is left entirely open. More precisely, while changing degrees of belief do indeed emerge as logical entailments of the given action theory, no procedure is given for computing these entailments. On closer examination, in fact, this is a two-part question:

  • (i)

    How do we effectively reason about beliefs in a particular state?

  • (ii)

    How do we effectively reason about belief state evolution and belief change?

In the simplest case, part (i) puts aside acting and sensing, and considers reasoning about the initial state only, which is then the classical problem of (first-order) probabilistic inference. We do not attempt to do a full survey here, but this has received a lot of attention [56], [32], [18], [11].

This work is about part (ii). Addressing this concern would not only aid planning algorithms, but also has a critical bearing on the assumptions made about the domain for tractability purposes. For example, if the initial state supports a decomposed representation of the distribution, can we expect the same after actions? In the exception of very limited cases such as Kalman filtering that harness the conjugate property of Gaussian processes, the situation is discouraging. In fact, even in the slightly more general case of Dynamic Bayesian Networks, which are in essence atomic propositions, if one were to assume that state variables are independent at time 0, they can become fully correlated after a few steps [19], [17], [33]. Dealing with complex actions, incomplete specifications and mixed representations, therefore, is significantly more involved.

In the reasoning about actions literature, where the focus is on qualitative (non-probabilistic) knowledge, there are two main solutions to projection: regression and progression [58]. Both of these have proven enormously useful for the design of logical agents, essentially paving the way for cognitive robotics [42]. Roughly, regression reduces a query about the future to a query about the initial state. Progression, on the other hand, changes the initial state according to the effects of each action and then checks whether the formula holds in the updated state. In this work, we show how both of these generalize in the presence of degrees of belief, noisy acting and sensing. Our results allow for both discrete and continuous probability distributions to be used in the specification of beliefs and dynamics, that leverage a recent extension of the BHL framework to mixed discrete-continuous domains [10].

To elaborate on the regression result, we show that it is general, not requiring (but allowing) structural constraints about the domain, nor imposing (but allowing) limitations to the family of actions. Regression derives a mathematical formula, using term and formula substitution only, that relates belief after a sequence of actions and observations, even when they are noisy, to beliefs about the initial state. That is, among other things, if the initial state supports efficient factorizations, regression will maintain this advantage no matter how actions affect the dependencies between state variables over time. Going further, the formalism will work seamlessly with discrete probability distributions, probability densities, and perhaps most significantly, with difficult combinations of the two.

To see a simple example of what goal regression does, imagine a robot facing a wall and at a certain distance h to it, as in Fig. 1. The robot might initially believe h to be drawn from a uniform distribution on [2,12]. Assume the robot moves away by 2 units and is now interested in the belief about h5. Regression would tell the robot that this is equivalent to its initial beliefs about h3 which here would lead to a value of .1. To see a nontrivial example, imagine now the robot is also equipped with a sonar unit aimed at the wall, that adds Gaussian noise with mean μ and variance σ2. After moving away by 2 units, if the sonar were now to provide a reading of 8, then regression would derive that belief about h5 is equivalent to1γ23.1×N(6x;μ,σ2)dx, where γ is the normalization factor. Essentially, the posterior belief about h5 is reformulated as the product of the prior belief about h3 and the likelihood of h3 given an observation of 6. (That is, observing 8 after moving away by 2 units is related here to observing 6 initially.)

We believe the broader implications of this result are two-fold. On the one hand, as we show later, simple cases of belief state evolution, as applicable to conjugate distributions for example, are special cases of regression's backward chaining procedure. Thus, regression could serve as a formal basis to study probabilistic belief change wrt limited forms of actions. On the other hand, our contribution can be viewed as a methodology for combining actions with recent advances in probabilistic inference, because reasoning about actions reduces to reasoning about the initial state.

To elaborate on the progression result, it has been argued that for long-lived agents like robots, continually updating the current view of the state of the world, is perhaps better suited. Lin and Reiter [47] show that progression is always second-order definable, and in general, it appears that second-order logic is unavoidable [76]. However, Lin and Reiter also identify some first-order definable cases by syntactically restricting situation calculus basic action theories, and since then, a number of other special cases have been studied [48].

While Lin and Reiter intended their work to be used on robots, one criticism leveled at their work, and indeed at much of the work in cognitive robotics, is that the theory is far removed from the kind of continuous uncertainty and noise seen in typical robotic applications. What exactly filtering mechanisms (such as Kalman filters) have to do with Lin and Reiter's progression has gone unanswered, although it has long been suspected that the two are related.

Our result remedies this situation. However, as we discuss later, progression in stochastic domains is complicated by the fact that actions can transform a continuous distribution to a mixed one. To obtain a closed-form result, we introduce a property of basic action theories called invertibility, closely related to invertible functions in real analysis [70]. We identify syntactic restrictions on basic action theories that guarantee invertibility. For our central result, we show a first-order progression of degrees of belief against noise in effectors and sensors for action theories that are invertible.

We structure this article as follows. In the preliminaries section, we cover the situation calculus, recap BHL and go over the essentials of its continuous extension. (This is taken with slight modifications from [10].) We then present regression for discrete domains, followed by regression for general domains. We then turn to a few special cases, such as conjugate distributions. Next, we turn to invertible theories, and discuss progression. Finally, we conclude after discussing related work.

We note that a preliminary version of this work has appeared in [7], [8]. In particular, the results on regression were first reported in [7], but that account was limited to noise-free actions only. The results on progression were first reported in [8].

Section snippets

Background

We work with the language L of the situation calculus [51], as developed in [58]. It is a special-purpose knowledge representation formalism for reasoning about dynamical systems. The formalism is best understood by arranging the world in terms of three kinds of things: situations, actions and objects. Situations represent “snapshots,” and can be viewed as possible histories. A set of initial situations correspond to the ways the world can be prior to the occurrence of actions. The result of

Regression for discrete domains

We now investigate a computational mechanism for reasoning about beliefs after a trajectory. This is a generalization of the regression operator for knowledge over exact acting and sensing investigated in [62]. In this section, we focus on discrete domains, where a weight-based notion of belief would be appropriate. Domains with both discrete and continuous variables are reserved for the next section.

Formally, given a basic action theory D, a sequence of actions δ, we might want to determine

Regression for general domains

We now generalize regression for domains with discrete and continuous variables, for which a density-based notion of belief is appropriate. (Physical actions are still noise-free for this section.) The main issue is that when formulating posterior beliefs after sensing, something like Definition 2's item 4(b) will not work. This is because over continuous spaces the belief about any individual point is 0. Therefore, we will be unpacking belief in terms of the density function, i.e. in terms of P

Two special cases

Regression is a general property for computing properties about posteriors in terms of priors after actions. It is therefore possible to explore limited cases, which might be appropriate for some applications. We present two such cases.

Regression over noisy actions

The regression operator thus far was limited to noise-free actions and noisy sensing. We first show how the logical account is extended to handle noisy actions (following [10]). We then extend the regression operator.

The idea behind noisy actions is that an agent might attempt a physical move of 3 units, but as a result of the limited accuracy of effectors, actually move (say) 3.094 units. Thus, unlike sensors, where the reading is nondeterministic, observable, but does not affect fluents, the

Progression

In the worst case, regressed formulas are exponentially long in the length of the action sequence [58], and so it has been argued that for long-lived agents like robots, continually updating the current view of the state of the world, is perhaps better suited. Lin and Reiter [47] proposed a theory of progression for the classical situation calculus. What we are after is an account of progression for probabilistic beliefs in the presence of stochastic noise. However, subtleties arise with the p

Computability of progression

In the general case [47], the computability of progression is a major concern, as it requires second-order logic. We are treating a special case here, and because it is defined over simple syntactic transformations, we have the following result immediately:

Theorem 29

Suppose D=D0Σ is any invertible basic action theory. After the iterative progression of D0Σ wrt a sequence δ, the size of the new initial theory is O(|D|×|δ|).

Proof

The result of progression is a theory D0 which is essentially obtained by means

Related work

To the best our knowledge, this work is the first to fully generalize classical first-order regression and progression for degrees of belief, noisy acting and sensing. Below, we discuss related efforts in terms of the language, and the individual techniques. We note that although we restricted L to nullary real-valued fluents, we suspect that both regression and invertibility and its connection to progression may apply more generally. This is left for future investigations.

At the outset, our

Conclusions

Planning and robotic applications have to deal with numerous sources of complexity regarding action and change. Along with efforts in related knowledge representation formalisms such as dynamic logic [73], the action language [30] and the fluent calculus [68], Reiter's [58] reconsideration of the situation calculus has proven enormously useful for the design of logical agents, essentially paving the way for cognitive robotics [42].

In this work, we obtained new results on how to handle

Declaration of Competing Interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

References (77)

  • F. Bacchus

    Representing and Reasoning with Probabilistic Knowledge

    (1990)
  • C. Baral et al.

    Epistemic planning (Dagstuhl Seminar 17231)

    Dagstuhl Rep.

    (2017)
  • V. Belle et al.

    A semantical account of progression in the presence of uncertainty

  • V. Belle et al.

    Reasoning about probabilities in unbounded first-order dynamical domains

  • V. Belle et al.

    Reasoning about continuous uncertainty in the situation calculus

  • V. Belle et al.

    Reasoning about probabilities in dynamic systems using goal regression

  • V. Belle et al.

    How to progress beliefs in continuous domains

  • V. Belle et al.

    PREGO: an action language for belief-based cognitive robotics in continuous domains

  • V. Belle et al.

    Probabilistic inference in hybrid domains by weighted model integration

  • D. Bertsekas et al.

    Introduction to Probability

    (2008)
  • K. Binmore et al.

    Calculus: Concepts and Methods

    (2002)
  • C. Boutilier et al.

    Symbolic dynamic programming for first-order MDPs

  • C. Boutilier et al.

    Decision-theoretic, high-level agent programming in the situation calculus

  • G.E.P. Box et al.

    Bayesian Inference in Statistical Analysis

    (1973)
  • X. Boyen et al.

    Tractable inference for complex stochastic processes

  • L. De Raedt et al.

    Statistical relational learning

  • T. Dean et al.

    Probabilistic temporal reasoning

  • T. Dean et al.

    A model for reasoning about persistence and causation

    Comput. Intell.

    (1989)
  • T. Dean et al.

    Planning and Control

    (1991)
  • L. Demey et al.

    Logic and probabilistic update

  • L. Demey et al.

    Epistemic probabilistic logic

  • P. Domingos et al.

    A tractable first-order probabilistic logic

  • H. Enderton

    A Mathematical Introduction to Logic

    (1972)
  • R. Fagin et al.

    Reasoning about knowledge and probability

    J. ACM

    (1994)
  • R. Fagin et al.

    Reasoning About Knowledge

    (1995)
  • D. Fox et al.

    Bayesian filtering for location estimation

    IEEE Pervasive Comput.

    (2003)
  • C. Fritz et al.

    Monitoring plan optimality during execution

  • L. Getoor et al.

    Learning probabilistic models of relational structure

  • Cited by (9)

    • On the progression of belief

      2023, Artificial Intelligence
    • Concerning Measures in a First-order Logic with Actions and Meta-beliefs

      2023, Proceedings of the International Conference on Knowledge Representation and Reasoning
    • A Logic of Only-Believing over Arbitrary Probability Distributions

      2023, Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
    • From Actions to Programs as Abstract Actual Causes

      2022, Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022
    • The Nondeterministic Situation Calculus

      2021, Proceedings of the 18th International Conference on Principles of Knowledge Representation and Reasoning, KR 2021
    View all citing articles on Scopus
    1

    Vaishak Belle was partly supported by a Royal Society University Research Fellowship.

    View full text