Verifying real-world programs often requires inferring loop invariants with nonlinear constraints. This is especially true in programs that perform many numerical operations, such as control systems for avionics or industrial plants. Recently, data-driven methods for loop invariant inference have shown promise, especially on linear invariants. However, applying data-driven inference to nonlinear loop invariants is challenging due to the large numbers of and magnitudes of high-order terms, the potential for overfitting on a small number of samples, and the large space of possible inequality bounds. In this paper, we introduce a new neural architecture for general SMT learning, the Gated Continuous Logic Network (G-CLN), and apply it to nonlinear loop invariant learning. G-CLNs extend the Continuous Logic Network (CLN) architecture with gating units and dropout, which allow the model to robustly learn general invariants over large numbers of terms. To address overfitting that arises from finite program sampling, we introduce fractional sampling---a sound relaxation of loop semantics to continuous functions that facilitates unbounded sampling on real domain. We additionally design a new CLN activation function, the Piecewise Biased Quadratic Unit (PBQU), for naturally learning tight inequality bounds. We incorporate these methods into a nonlinear loop invariant inference system that can learn general nonlinear loop invariants. We evaluate our system on a benchmark of nonlinear loop invariants and show it solves 26 out of 27 problems, 3 more than prior work, with an average runtime of 53.3 seconds. We further demonstrate the generic learning ability of G-CLNs by solving all 124 problems in the linear Code2Inv benchmark. We also perform a quantitative stability evaluation and show G-CLNs have a convergence rate of $97.5\%$ on quadratic problems, a $39.2\%$ improvement over CLN models.

中文翻译：

---

使用门控连续逻辑网络学习非线性循环不变量（扩展版）

验证现实世界的程序通常需要推断具有非线性约束的循环不变量。在执行许多数值运算的程序中尤其如此，例如航空电子设备或工业工厂的控制系统。最近，用于循环不变推理的数据驱动方法已经显示出前景，尤其是在线性不变量方面。然而，由于高阶项的数量和大小、在少量样本上过度拟合的可能性以及可能的不等式边界空间很大，将数据驱动推理应用于非线性循环不变量具有挑战性。在本文中，我们介绍了一种用于一般 SMT 学习的新神经架构，即门控连续逻辑网络 (G-CLN)，并将其应用于非线性循环不变学习。G-CLN 使用门控单元和 dropout 扩展了连续逻辑网络 (CLN) 架构，这使模型能够在大量项上稳健地学习一般不变量。为了解决有限程序采样引起的过度拟合，我们引入了分数采样——循环语义对连续函数的合理放松，有助于在实际域中进行无界采样。我们还设计了一个新的 CLN 激活函数，即分段偏置二次单元 (PBQU)，用于自然地学习严格的不等式边界。我们将这些方法合并到一个非线性循环不变推理系统中，该系统可以学习一般的非线性循环不变性。我们在非线性循环不变量的基准上评估我们的系统，并表明它解决了 27 个问题中的 26 个问题，比之前的工作多 3 个，平均运行时间为 53.3 秒。我们通过解决线性 Code2Inv 基准测试中的所有 124 个问题进一步证明了 G-CLN 的通用学习能力。我们还进行了定量稳定性评估，并表明 G-CLN 在二次问题上的收敛率为 97.5%$，比 CLN 模型提高了 39.2%$。