Regression and progression in stochastic domains
Introduction
Reasoning about degrees of belief in uncertain dynamic worlds is fundamental to many applications, such as robotics and planning, where actions modify state properties and sensors provide measurements, both of which are prone to noise. However, there seem to be two disparate paradigms to address this concern, both of which have their limitations. At one extreme, there are logical formalisms, such as the situation calculus [51], [58], which allows us to express strict uncertainty, and exploits regularities in the effects actions have on propositions to describe physical laws compactly. Probabilistic sensor fusion, however, has received less attention here. (Notable exceptions will be discussed in the penultimate section.) At the other extreme, revising beliefs after noisy observations over rich error profiles is effortlessly addressed using probabilistic techniques such as Kalman filtering and Dynamic Bayesian Networks [20], [21]. However, in these frameworks, a complete specification of the dependencies between variables is taken as given, making it difficult to deal with other forms of incomplete knowledge as well as complex actions that shift dependencies between variables in nontrivial ways.
An influential but nevertheless simple proposal by Bacchus, Halpern and Levesque [2], BHL henceforth, was among the first to merge these broad areas in a general way. Their specification is widely applicable because it is not constrained to particular structural assumptions. In a nutshell, they extend the situation calculus language with a provision for specifying the degrees of belief in formulas in the initial state, closely fashioned after intuitions on incorporating probability in modal logics [34], [26]. This then allows incomplete and partial specifications, which might be compatible with one or very many initial distributions and sets of independence assumptions, with beliefs following at a corresponding level of specificity. Moreover, together with a rich action theory, the model not only exhibits Bayesian conditioning [55] (which, then, captures special cases such as Kalman filtering [69]), but also allows flexibility in the ways dependencies and distributions may change over actions.
While that proposal is powerful, the task of how to plan effectively is not addressed. In essence, this would correspond to a flavor of epistemic planning [3] where the state of knowledge, actions and sensing are mixtures of logical and probabilistic assertions. In fact, at a more fundamental level, the task of projection, that of reasoning about beliefs effectively after acting and sensing, is left entirely open. More precisely, while changing degrees of belief do indeed emerge as logical entailments of the given action theory, no procedure is given for computing these entailments. On closer examination, in fact, this is a two-part question:
- (i)
How do we effectively reason about beliefs in a particular state?
- (ii)
How do we effectively reason about belief state evolution and belief change?
In the simplest case, part (i) puts aside acting and sensing, and considers reasoning about the initial state only, which is then the classical problem of (first-order) probabilistic inference. We do not attempt to do a full survey here, but this has received a lot of attention [56], [32], [18], [11].
This work is about part (ii). Addressing this concern would not only aid planning algorithms, but also has a critical bearing on the assumptions made about the domain for tractability purposes. For example, if the initial state supports a decomposed representation of the distribution, can we expect the same after actions? In the exception of very limited cases such as Kalman filtering that harness the conjugate property of Gaussian processes, the situation is discouraging. In fact, even in the slightly more general case of Dynamic Bayesian Networks, which are in essence atomic propositions, if one were to assume that state variables are independent at time 0, they can become fully correlated after a few steps [19], [17], [33]. Dealing with complex actions, incomplete specifications and mixed representations, therefore, is significantly more involved.
In the reasoning about actions literature, where the focus is on qualitative (non-probabilistic) knowledge, there are two main solutions to projection: regression and progression [58]. Both of these have proven enormously useful for the design of logical agents, essentially paving the way for cognitive robotics [42]. Roughly, regression reduces a query about the future to a query about the initial state. Progression, on the other hand, changes the initial state according to the effects of each action and then checks whether the formula holds in the updated state. In this work, we show how both of these generalize in the presence of degrees of belief, noisy acting and sensing. Our results allow for both discrete and continuous probability distributions to be used in the specification of beliefs and dynamics, that leverage a recent extension of the BHL framework to mixed discrete-continuous domains [10].
To elaborate on the regression result, we show that it is general, not requiring (but allowing) structural constraints about the domain, nor imposing (but allowing) limitations to the family of actions. Regression derives a mathematical formula, using term and formula substitution only, that relates belief after a sequence of actions and observations, even when they are noisy, to beliefs about the initial state. That is, among other things, if the initial state supports efficient factorizations, regression will maintain this advantage no matter how actions affect the dependencies between state variables over time. Going further, the formalism will work seamlessly with discrete probability distributions, probability densities, and perhaps most significantly, with difficult combinations of the two.
To see a simple example of what goal regression does, imagine a robot facing a wall and at a certain distance h to it, as in Fig. 1. The robot might initially believe h to be drawn from a uniform distribution on Assume the robot moves away by 2 units and is now interested in the belief about Regression would tell the robot that this is equivalent to its initial beliefs about which here would lead to a value of .1. To see a nontrivial example, imagine now the robot is also equipped with a sonar unit aimed at the wall, that adds Gaussian noise with mean μ and variance . After moving away by 2 units, if the sonar were now to provide a reading of 8, then regression would derive that belief about is equivalent to where γ is the normalization factor. Essentially, the posterior belief about is reformulated as the product of the prior belief about and the likelihood of given an observation of 6. (That is, observing 8 after moving away by 2 units is related here to observing 6 initially.)
We believe the broader implications of this result are two-fold. On the one hand, as we show later, simple cases of belief state evolution, as applicable to conjugate distributions for example, are special cases of regression's backward chaining procedure. Thus, regression could serve as a formal basis to study probabilistic belief change wrt limited forms of actions. On the other hand, our contribution can be viewed as a methodology for combining actions with recent advances in probabilistic inference, because reasoning about actions reduces to reasoning about the initial state.
To elaborate on the progression result, it has been argued that for long-lived agents like robots, continually updating the current view of the state of the world, is perhaps better suited. Lin and Reiter [47] show that progression is always second-order definable, and in general, it appears that second-order logic is unavoidable [76]. However, Lin and Reiter also identify some first-order definable cases by syntactically restricting situation calculus basic action theories, and since then, a number of other special cases have been studied [48].
While Lin and Reiter intended their work to be used on robots, one criticism leveled at their work, and indeed at much of the work in cognitive robotics, is that the theory is far removed from the kind of continuous uncertainty and noise seen in typical robotic applications. What exactly filtering mechanisms (such as Kalman filters) have to do with Lin and Reiter's progression has gone unanswered, although it has long been suspected that the two are related.
Our result remedies this situation. However, as we discuss later, progression in stochastic domains is complicated by the fact that actions can transform a continuous distribution to a mixed one. To obtain a closed-form result, we introduce a property of basic action theories called invertibility, closely related to invertible functions in real analysis [70]. We identify syntactic restrictions on basic action theories that guarantee invertibility. For our central result, we show a first-order progression of degrees of belief against noise in effectors and sensors for action theories that are invertible.
We structure this article as follows. In the preliminaries section, we cover the situation calculus, recap BHL and go over the essentials of its continuous extension. (This is taken with slight modifications from [10].) We then present regression for discrete domains, followed by regression for general domains. We then turn to a few special cases, such as conjugate distributions. Next, we turn to invertible theories, and discuss progression. Finally, we conclude after discussing related work.
We note that a preliminary version of this work has appeared in [7], [8]. In particular, the results on regression were first reported in [7], but that account was limited to noise-free actions only. The results on progression were first reported in [8].
Section snippets
Background
We work with the language of the situation calculus [51], as developed in [58]. It is a special-purpose knowledge representation formalism for reasoning about dynamical systems. The formalism is best understood by arranging the world in terms of three kinds of things: situations, actions and objects. Situations represent “snapshots,” and can be viewed as possible histories. A set of initial situations correspond to the ways the world can be prior to the occurrence of actions. The result of
Regression for discrete domains
We now investigate a computational mechanism for reasoning about beliefs after a trajectory. This is a generalization of the regression operator for knowledge over exact acting and sensing investigated in [62]. In this section, we focus on discrete domains, where a weight-based notion of belief would be appropriate. Domains with both discrete and continuous variables are reserved for the next section.
Formally, given a basic action theory , a sequence of actions δ, we might want to determine
Regression for general domains
We now generalize regression for domains with discrete and continuous variables, for which a density-based notion of belief is appropriate. (Physical actions are still noise-free for this section.) The main issue is that when formulating posterior beliefs after sensing, something like Definition 2's item 4(b) will not work. This is because over continuous spaces the belief about any individual point is 0. Therefore, we will be unpacking belief in terms of the density function, i.e. in terms of P
Two special cases
Regression is a general property for computing properties about posteriors in terms of priors after actions. It is therefore possible to explore limited cases, which might be appropriate for some applications. We present two such cases.
Regression over noisy actions
The regression operator thus far was limited to noise-free actions and noisy sensing. We first show how the logical account is extended to handle noisy actions (following [10]). We then extend the regression operator.
The idea behind noisy actions is that an agent might attempt a physical move of 3 units, but as a result of the limited accuracy of effectors, actually move (say) 3.094 units. Thus, unlike sensors, where the reading is nondeterministic, observable, but does not affect fluents, the
Progression
In the worst case, regressed formulas are exponentially long in the length of the action sequence [58], and so it has been argued that for long-lived agents like robots, continually updating the current view of the state of the world, is perhaps better suited. Lin and Reiter [47] proposed a theory of progression for the classical situation calculus. What we are after is an account of progression for probabilistic beliefs in the presence of stochastic noise. However, subtleties arise with the p
Computability of progression
In the general case [47], the computability of progression is a major concern, as it requires second-order logic. We are treating a special case here, and because it is defined over simple syntactic transformations, we have the following result immediately:
Theorem 29 Suppose is any invertible basic action theory. After the iterative progression of wrt a sequence δ, the size of the new initial theory is
Proof The result of progression is a theory which is essentially obtained by means
Related work
To the best our knowledge, this work is the first to fully generalize classical first-order regression and progression for degrees of belief, noisy acting and sensing. Below, we discuss related efforts in terms of the language, and the individual techniques. We note that although we restricted to nullary real-valued fluents, we suspect that both regression and invertibility and its connection to progression may apply more generally. This is left for future investigations.
At the outset, our
Conclusions
Planning and robotic applications have to deal with numerous sources of complexity regarding action and change. Along with efforts in related knowledge representation formalisms such as dynamic logic [73], the action language [30] and the fluent calculus [68], Reiter's [58] reconsideration of the situation calculus has proven enormously useful for the design of logical agents, essentially paving the way for cognitive robotics [42].
In this work, we obtained new results on how to handle
Declaration of Competing Interest
We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.
References (77)
- et al.
Reasoning about noisy sensors and effectors in the situation calculus
Artif. Intell.
(1999) - et al.
Reasoning about discrete and continuous noisy sensors and effectors in dynamical systems
Artif. Intell.
(2018) - et al.
Representing action and change by logic programs
Special Issue: Non-Monotonic Reasoning and Logic Programming, J. Log. Program.
(1993) An analysis of first-order logics of probability
Artif. Intell.
(1990)- et al.
Planning and acting in partially observable stochastic domains
Artif. Intell.
(1998) - et al.
An algorithm for probabilistic planning
Artif. Intell.
(1995) - et al.
How to progress a database
Artif. Intell.
(1997) Probabilistic logic
Artif. Intell.
(1986)- et al.
Some first-order probability logics
Theor. Comput. Sci.
(2000) - et al.
Knowledge, action, and the frame problem
Artif. Intell.
(2003)
Representing and Reasoning with Probabilistic Knowledge
Epistemic planning (Dagstuhl Seminar 17231)
Dagstuhl Rep.
A semantical account of progression in the presence of uncertainty
Reasoning about probabilities in unbounded first-order dynamical domains
Reasoning about continuous uncertainty in the situation calculus
Reasoning about probabilities in dynamic systems using goal regression
How to progress beliefs in continuous domains
PREGO: an action language for belief-based cognitive robotics in continuous domains
Probabilistic inference in hybrid domains by weighted model integration
Introduction to Probability
Calculus: Concepts and Methods
Symbolic dynamic programming for first-order MDPs
Decision-theoretic, high-level agent programming in the situation calculus
Bayesian Inference in Statistical Analysis
Tractable inference for complex stochastic processes
Statistical relational learning
Probabilistic temporal reasoning
A model for reasoning about persistence and causation
Comput. Intell.
Planning and Control
Logic and probabilistic update
Epistemic probabilistic logic
A tractable first-order probabilistic logic
A Mathematical Introduction to Logic
Reasoning about knowledge and probability
J. ACM
Reasoning About Knowledge
Bayesian filtering for location estimation
IEEE Pervasive Comput.
Monitoring plan optimality during execution
Learning probabilistic models of relational structure
Cited by (9)
On the progression of belief
2023, Artificial IntelligenceEpistemic planning: Perspectives on the special issue
2023, Artificial IntelligenceConcerning Measures in a First-order Logic with Actions and Meta-beliefs
2023, Proceedings of the International Conference on Knowledge Representation and ReasoningA Logic of Only-Believing over Arbitrary Probability Distributions
2023, Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMASFrom Actions to Programs as Abstract Actual Causes
2022, Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022The Nondeterministic Situation Calculus
2021, Proceedings of the 18th International Conference on Principles of Knowledge Representation and Reasoning, KR 2021
- 1
Vaishak Belle was partly supported by a Royal Society University Research Fellowship.