Recognition of handwritten mathematical expressions (MEs) is an important problem that has wide applications in practice. Handwritten ME recognition is challenging due to the variety of writing styles and ME formats. As a result, recognizers trained by optimizing the traditional supervision loss do not perform satisfactorily. To improve the robustness of the recognizer with respect to writing styles, in this work, we propose a novel paired adversarial learning method to learn semantic-invariant features. Specifically, our proposed model, named PAL-v2, consists of an attention-based recognizer and a discriminator. During training, handwritten MEs and their printed templates are fed into PAL-v2 simultaneously. The attention-based recognizer is trained to learn semantic-invariant features with the guide of the discriminator. Moreover, we adopt a convolutional decoder to alleviate the vanishing and exploding gradient problems of RNN-based decoder, and further, improve the coverage of decoding with a novel attention method. We conducted extensive experiments on the CROHME dataset to demonstrate the effectiveness of each part of the method and achieved state-of-the-art performance.

中文翻译：

---

基于配对对抗学习的手写数学表达式识别

手写数学表达式（ME）的识别是一个在实践中具有广泛应用的重要问题。由于书写风格和 ME 格式的多样性，手写 ME 识别具有挑战性。结果，通过优化传统监督损失训练的识别器的表现并不令人满意。为了提高识别器在书写风格方面的鲁棒性，在这项工作中，我们提出了一种新颖的配对对抗学习方法来学习语义不变特征。具体来说，我们提出的名为 PAL-v2 的模型由一个基于注意力的识别器和一个鉴别器组成。在训练期间，手写 ME 及其打印模板同时输入 PAL-v2。训练基于注意力的识别器以在鉴别器的指导下学习语义不变的特征。而且，我们采用卷积解码器来缓解基于 RNN 的解码器的梯度消失和爆炸问题，并进一步用一种新颖的注意力方法提高解码的覆盖率。我们对 CROHME 数据集进行了大量实验，以证明该方法每个部分的有效性并取得了最先进的性能。