当前位置: X-MOL 学术Methods Ecol. Evol. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A primer on partially observable Markov decision processes (POMDPs)
Methods in Ecology and Evolution ( IF 6.3 ) Pub Date : 2021-08-02 , DOI: 10.1111/2041-210x.13692
Iadine Chades 1 , Luz V. Pascal 2 , Sam Nicol 1 , Cameron S. Fletcher 3 , Jonathan Ferrer Mestres 1
Affiliation  

  1. Partially observable Markov decision processes (POMDPs) are a convenient mathematical model to solve sequential decision-making problems under imperfect observations. Most notably for ecologists, POMDPs have helped solve the trade-offs between investing in management or surveillance and, more recently, to optimise adaptive management problems.
  2. Despite an increasing number of applications in ecology and natural resources, POMDPs are still poorly understood. The complexity of the mathematics, the inaccessibility of POMDP solvers developed by the Artificial Intelligence (AI) community, and the lack of introductory material are likely reasons for this.
  3. We propose to bridge this gap by providing a primer on POMDPs, a typology of case studies drawn from the literature, and a repository of POMDP problems.
  4. We explain the steps required to define a POMDP when the state of the system is imperfectly detected (state uncertainty) and when the dynamics of the system are unknown (model uncertainty). We provide input files and solutions to a selected number of problems, reflect on lessons learned applying these models over the last 10 years and discuss future research required on interpretable AI.
  5. Partially observable Markov decision processes are powerful decision models that allow users to make decisions under imperfect observations over time. This primer will provide a much-needed entry point to ecologists.


中文翻译:

部分可观察马尔可夫决策过程 (POMDP) 入门

  1. 部分可观察马尔可夫决策过程 (POMDP) 是一种方便的数学模型,用于解决不完美观察下的顺序决策问题。对于生态学家来说,最值得注意的是,POMDP 帮助解决了投资管理或监控与最近优化适应性管理问题之间的权衡。
  2. 尽管在生态学和自然资源中的应用越来越多,但 POMDPs 仍然知之甚少。数学的复杂性、人工智能 (AI) 社区开发的 POMDP 求解器的不可访问性以及介绍材料的缺乏可能是造成这种情况的原因。
  3. 我们建议通过提供 POMDP 入门、从文献中提取的案例研究类型以及 POMDP 问题库来弥合这一差距。
  4. 我们解释了在系统状态检测不完善(状态不确定性)和系统动态未知(模型不确定性)时定义 POMDP 所需的步骤。我们为选定数量的问题提供输入文件和解决方案,反思过去 10 年应用这些模型的经验教训,并讨论可解释人工智能所需的未来研究。
  5. 部分可观察的马尔可夫决策过程是强大的决策模型,允许用户随着时间的推移在不完美的观察下做出决策。这本入门书将为生态学家提供急需的切入点。
更新日期:2021-08-02
down
wechat
bug