Different high-level robotics tasks require the robot to manipulate or interact with objects that are in an unexplored part of the environment or not already in its field of view. Although much works rely on searching for objects based on their colour or 3D context, we argue that text information is a useful and functional visual cue to guide the search. In this paper, we study the problem of active visual search (AVS) in large unknown environments. In this paper, we present an AVS system that relies on semantic information inferred from texts found in the environment, which allows the robot to reduce the search costs by avoiding not promising regions for the target object. Our semantic planner reasons over the numbers detected from door signs to decide either perform a goal-directed exploration towards unknown parts of the environment or carefully search in the already known parts. We compared the performance of our semantic AVS system with two other search systems in four simulated environments. First, we developed a greedy search system that does not consider any semantic information, and second, we invited human participants to teleoperate the robot while performing the search. Our results from simulation and real-world experiments show that text is a promising source of information that provides different semantic cues for AVS systems.

不同的高级机器人任务要求机器人操纵环境中未探索的部分或尚未在其视野中的物体或与之交互。尽管许多工作依赖于根据颜色或 3D 上下文来搜索对象，但我们认为文本信息是指导搜索的有用且实用的视觉提示。在本文中，我们研究了大型未知环境中的主动视觉搜索（AVS）问题。在本文中，我们提出了一种 AVS 系统，该系统依赖于从环境中发现的文本推断出的语义信息，该系统允许机器人通过避开目标对象不希望的区域来降低搜索成本。我们的语义规划器对从门标志检测到的数字进行推理，以决定是对环境的未知部分进行目标导向的探索，还是在已知的部分中仔细搜索。我们在四个模拟环境中将语义 AVS 系统的性能与其他两个搜索系统进行了比较。首先，我们开发了一个不考虑任何语义信息的贪婪搜索系统，其次，我们邀请人类参与者在执行搜索时远程操作机器人。我们的模拟和现实实验结果表明，文本是一种有前途的信息源，可为 AVS 系统提供不同的语义线索。