Current technology for autonomous cars primarily focuses on getting the passenger from point A to B. Nevertheless, it has been shown that passengers are afraid of taking a ride in self-driving cars. One way to alleviate this problem is by allowing the passenger to give natural language commands to the car. However, the car can misunderstand the issued command or the visual surroundings which could lead to uncertain situations. It is desirable that the self-driving car detects these situations and interacts with the passenger to solve them. This paper proposes a model that detects the uncertain situations when a command is given and finds the visual objects causing it. Optionally, a question generated by the system describing the uncertain objects is included. We argue that if the car could explain the objects in a human-like way, passengers could gain more confidence in the car’s abilities. Thus, we investigate how to (1) detect uncertain situations and their underlying causes, and (2) how to generate clarifying questions for the passenger. When evaluating on the Talk2Car dataset, we show that the proposed model, Uncertainty Resolving System (URS), improves 12.6% in terms of $Io{U}_{.5}$ compared to not using URS. Furthermore, we designed a referring expression generator (REG) Attribute-Referring Expression Generator (A-REG) tailored to a self-driving car setting which yields a relative improvement of 6% METEOR and 8% ROUGE-l compared with state-of-the-art REG models, and is three times faster.

当前的自动驾驶汽车技术主要侧重于将乘客从 A 点带到 B 点。然而，已经表明乘客害怕乘坐自动驾驶汽车。缓解此问题的一种方法是允许乘客向汽车发出自然语言命令。但是，汽车可能会误解发出的命令或视觉环境，从而导致不确定的情况。希望自动驾驶汽车检测到这些情况并与乘客互动以解决这些问题。本文提出了一种模型，该模型可以检测发出命令时的不确定情况，并找到导致该情况的视觉对象。可选地，包括由系统生成的描述不确定对象的问题。我们认为，如果汽车可以像人类一样解释物体，乘客可以对汽车的性能更有信心。因此，我们研究如何 (1) 检测不确定情况及其根本原因，以及 (2) 如何为乘客提出澄清问题。在对 Talk2Car 数据集进行评估时，我们表明所提出的模型不确定性解决系统 (URS) 在以下方面提高了 12.6%$\mathrm{一世}○{你}_{.5}$与不使用 URS 相比。此外，我们设计了一个针对自动驾驶汽车设置的引用表达式生成器 (REG) 属性引用表达式生成器 (A-REG)，与 state-of- 相比，它产生了 6% METEOR 和 8% ROUGE-l 的相对改进 -最先进的 REG 模型，速度提高了三倍。