Developing Semi-Supervised Seq2Seq ($S^4$) learning for sequence transduction tasks in natural language processing (NLP), e.g. semantic parsing, is challenging, since both the input and the output sequences are discrete. This discrete nature makes trouble for methods which need gradients either from the input space or from the output space. Recently, a new learning method called joint stochastic approximation is developed for unsupervised learning of fixed-dimensional autoencoders and theoretically avoids gradient propagation through discrete latent variables, which is suffered by Variational Auto-Encoders (VAEs). In this letter, we propose seq2seq Joint-stochastic-approximation Auto-Encoders (JAEs) and apply them to $S^4$ learning for NLP sequence transduction tasks. Further, we propose bi-directional JAEs (called bi-JAEs) to leverage not only unpaired input sequences (which is most commonly studied) but also unpaired output sequences. Experiments on two benchmarking datasets for semantic parsing show that JAEs consistently outperform VAEs in $S^4$ learning and bi-JAEs yield further improvements.

中文翻译：

---

半监督 Seq2seq 联合随机逼近自动编码器在语义解析中的应用

开发半监督 Seq2Seq ($S^4$) 学习自然语言处理 (NLP) 中的序列转导任务，例如语义解析，具有挑战性，因为输入和输出序列都是离散的。这种离散性质给需要来自输入空间或输出空间的梯度的方法带来麻烦。最近，开发了一种称为联合随机逼近的新学习方法，用于固定维自动编码器的无监督学习，并在理论上避免了通过离散潜在变量进行的梯度传播，这会受到变分自动编码器 (VAE) 的影响。在这封信中，我们提出了 seq2seq Joint-stochastic-approximation Auto-Encoders (JAEs) 并将它们应用于$S^4$学习 NLP 序列转导任务。此外，我们提出了双向 JAE（称为双向 JAE），不仅可以利用未配对的输入序列（这是最常研究的），还可以利用未配对的输出序列。在用于语义解析的两个基准数据集上进行的实验表明，JAEs 始终优于 VAEs$S^4$ 学习和双 JAE 产生了进一步的改进。