当前位置: X-MOL 学术arXiv.cs.LO › 论文详情
DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning
arXiv - CS - Logic in Computer Science Pub Date : 2019-11-22 , DOI: arxiv-1911.10244
Mohammadhosein Hasanbeig; Natasha Yogananda Jeppu; Alessandro Abate; Tom Melham; Daniel Kroening

We propose a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires the attainment of an unknown sequence of high-level objectives. Our method employs a recently-published algorithm for synthesis of compact automata to uncover this sequential structure. We synthesise an automaton from trace data generated through exploration of the environment by the deep RL agent. A product construction is then used to enrich the state space of the environment so that generation of an optimal control policy by deep RL is guided by the discovered structure encoded in the automaton. Our experiments show that our method is able to achieve training results that are otherwise difficult with state-of-the-art RL techniques unaided by external guidance.
更新日期:2020-02-13

 

全部期刊列表>>
化学/材料学中国作者研究精选
ACS材料视界
南京大学
自然科研论文编辑服务
剑桥大学-
中国科学院大学化学科学学院
南开大学化学院周其林
课题组网站
X-MOL
北京大学分子工程苏南研究院
华东师范大学分子机器及功能材料
中山大学化学工程与技术学院
试剂库存
天合科研
down
wechat
bug