Dialogue state tracking (DST) is a crucial module in dialogue management. It is usually cast as a supervised training problem, which is not convenient for on-line optimization. In this paper, a novel companion teaching based deep reinforcement learning (DRL) framework for on-line DST optimization is proposed. To the best of our knowledge, this is the first effort to optimize the DST module within DRL framework for on-line task-oriented spoken dialogue systems. In addition, dialogue policy can be further jointly updated. Experiments show that on-line DST optimization can effectively improve the dialogue manager performance while keeping the flexibility of using predefined policy. Joint training of both DST and policy can further improve the performance.

中文翻译：

---

用于在线对话状态跟踪的深度强化学习

对话状态跟踪（DST）是对话管理中的一个关键模块。它通常被视为监督训练问题，不便于在线优化。在本文中，提出了一种用于在线 DST 优化的基于深度强化学习 (DRL) 的新型伴随教学框架。据我们所知，这是第一次在面向任务的在线语音对话系统的 DRL 框架内优化 DST 模块。此外，对话策略可以进一步联合更新。实验表明，在线 DST 优化可以有效提高对话管理器的性能，同时保持使用预定义策略的灵活性。DST 和策略的联合训练可以进一步提高性能。