Arabshahi, Singh, and Anandkumar (2018) propose a method for creating a dataset of symbolic mathematical equations for the tasks of symbolic equation verification and equation completion. Unfortunately, a dataset constructed using the method they propose will suffer from two serious flaws. First, the class of true equations that the procedure can generate will be very limited. Second, because true and false equations are generated in completely different ways, there are likely to be artifactual features that allow easy discrimination. Moreover, over the class of equations they consider, there is an extremely simple probabilistic procedure that solves the problem of equation verification with extremely high reliability. The usefulness of this problem in general as a testbed for AI systems is therefore doubtful.

中文翻译：

---

用于符号方程验证的有缺陷的数据集

Arabshahi，Singh和Anandkumar（2018）提出了一种创建符号数学方程式数据集的方法，用于符号方程式验证和方程式完成的任务。不幸的是，使用他们提出的方法构建的数据集将遭受两个严重的缺陷。首先，该过程可以生成的真方程类非常有限。其次，由于对和错方程是以完全不同的方式生成的，因此可能存在人为因素，可以轻松区分。此外，在他们考虑的方程类别中，有一个非常简单的概率过程，可以极高的可靠性解决方程验证的问题。因此，一般而言，此问题作为AI系统测试平台的用处值得怀疑。