We propose a novel approach to interactive theorem-proving (ITP) using deep reinforcement learning. Unlike previous work, our framework is able to prove theorems both end-to-end and from scratch (i.e., without relying on example proofs from human experts). We formulate the process of ITP as a Markov decision process (MDP) in which each state represents a set of potential derivation paths. The agent learns to select promising derivations as well as appropriate tactics within each derivation using deep policy gradients. This structure allows us to introduce a novel backtracking mechanism which enables the agent to efficiently discard (predicted) dead-end derivations and restart the derivation from promising alternatives. Experimental results show that the framework provides comparable performance to that of the approaches that use human experts, and that it is also capable of proving theorems that it has never seen during training. We further elaborate the role of each component of the framework using ablation studies.

中文翻译：

---

TacticZero：通过深度强化学习从零开始学习证明定理

我们提出了一种使用深度强化学习的交互式定理证明（ITP）的新颖方法。与以前的工作不同，我们的框架能够从头到尾地证明定理（即，不依赖人类专家的示例证明）。我们将ITP的过程表述为马尔可夫决策过程（MDP），其中每个状态代表一组潜在的推导路径。代理学习使用深层次的策略梯度来选择有希望的派生以及每个派生中的适当策略。这种结构使我们能够引入一种新颖的回溯机制，该机制使代理能够有效地丢弃（预测）死胡同的派生，并从有前途的替代方案中重新开始派生。实验结果表明，该框架可提供与使用人类专家的方法相当的性能，并且它还能够证明训练期间从未见过的定理。我们使用消融研究进一步阐述了框架各部分的作用。