Text production (Reiter and Dale 2000; Gatt and Krahmer 2018) is also referred to as natural language generation (NLG). It is a subtask of natural language processing focusing on the generation of natural language text. Although as important as natural language understanding for communication, NLG had received relatively less research attention. Recently, the rise of deep learning techniques has led to a surge of research interest in text production, both in general and for specific applications such as text summarization and dialogue systems. Deep learning allows NLG models to be constructed based on neural representations, thereby enabling end-to-end NLG systems to replace traditional pipeline approaches, which frees us from tedious engineering efforts and improves the output quality. In particular, a neural encoder-decoder structure (Cho et al. 2014; Sutskever, Vinyals, and Le 2014) has been widely used as a basic framework, which computes input representations using a neural encoder, according to which a text sequence is generated token by token using a neural decoder. Very recently, pre-training techniques (Broscheit et al. 2010; Radford 2018; Devlin et al. 2019) have further allowed neural models to collect knowledge from large raw text data, further improving the quality of both encoding and decoding. This book introduces the fundamentals of neural text production, discussing both the mostly investigated tasks and the foundational neural methods. NLG tasks with different types of inputs are introduced, and benchmark datasets are discussed in detail. The encoder-decoder architecture is introduced together with basic neural network components such as convolutional neural network (CNN) (Kim 2014) and recurrent neural network (RNN) (Cho et al. 2014). Elaborations are given on the encoder, the decoder, and task-specific optimization techniques. A contrast is made between the neural solution and traditional solutions to the task. Towards the end of the book, more recent techniques such as self-attention networks (Vaswani et al. 2017) and pre-training are briefly discussed. Throughout the book, figures are given to facilitate understanding and references are provided to enable further reading. Chapter 1 introduces the task of text production, discussing three typical input settings, namely, generation from meaning representations (MR; i.e., realization), generation from data (i.e., data-to-text), and generation from text (i.e., text-to-text). At the end of the chapter, a book outline is given, and the scope, coverage and notation

中文翻译：

---

文本生成的深度学习方法

文本生成（Reiter 和 Dale 2000；Gatt 和 Krahmer 2018）也称为自然语言生成 (NLG)。它是自然语言处理的一个子任务，侧重于自然语言文本的生成。尽管与自然语言理解对交流同样重要，但 NLG 受到的研究关注相对较少。最近，深度学习技术的兴起导致了对文本生成的研究兴趣激增，无论是一般的还是特定的应用程序，如文本摘要和对话系统。深度学习允许基于神经表示构建 NLG 模型，从而使端到端 NLG 系统能够取代传统的管道方法，从而使我们摆脱繁琐的工程工作并提高输出质量。特别是，神经编码器-解码器结构（Cho 等人。2014; Sutskever、Vinyals 和 Le 2014）已被广泛用作基本框架，该框架使用神经编码器计算输入表示，根据该框架使用神经解码器逐个标记生成文本序列。最近，预训练技术（Broscheit et al. 2010; Radford 2018; Devlin et al. 2019）进一步允许神经模型从大型原始文本数据中收集知识，进一步提高编码和解码的质量。本书介绍了神经文本生成的基础知识，讨论了最常研究的任务和基本的神经方法。介绍了具有不同类型输入的 NLG 任务，并详细讨论了基准数据集。编码器-解码器架构与基本神经网络组件一起引入，例如卷积神经网络 (CNN) (Kim 2014) 和循环神经网络 (RNN) (Cho et al. 2014)。详细介绍了编码器、解码器和特定于任务的优化技术。对神经解决方案和传统的任务解决方案进行了对比。在本书的最后，简要讨论了自注意力网络（Vaswani 等人，2017 年）和预训练等最新技术。整本书都给出了数字以方便理解，并提供了参考资料以便进一步阅读。第 1 章介绍了文本生成的任务，讨论了三种典型的输入设置，即从意义表示生成（MR；即实现）、从数据生成（即数据到文本）、和从文本生成（即文本到文本）。在本章的最后，给出了本书的大纲，以及范围、覆盖面和符号