The success of learning techniques in solving a variety of hard AI problems promotes the flourish of recognition-based applications. Many state-of-the-art text localization systems, which can detect and report the positions of text segments in an image, are mainly implemented with learning-based techniques. Data-driven learning raises a series of questions on how to verify, validate and evaluate such learning-based systems. In this paper, we propose a methodology to automatically evaluate the stability of text localization systems via metamorphic relations, where a stable system should output consistent results for similar inputs with the same text segments. We introduce six metamorphic relations that should be preserved in a stable text localization system and define the corresponding metrics for stability evaluation. With the defined metamorphic relations, we apply metamorphic testing techniques to compare the inputs and outputs to evaluate system stability, and further diagnose the causes of inconsistency. The extensive experimentation on both academic and commercial text localization systems demonstrates the effectiveness of our method on stability evaluation for such systems.

学习技术在解决各种人工智能难题方面的成功促进了基于识别的应用程序的蓬勃发展。许多最先进的文本定位系统可以检测和报告图像中文本段的位置，主要是通过基于学习的技术实现的。数据驱动的学习提出了一系列关于如何验证、验证和评估这种基于学习的系统的问题。在本文中，我们提出了一种通过变形关系自动评估文本定位系统稳定性的方法，其中稳定的系统应该为具有相同文本段的相似输入输出一致的结果。我们介绍了稳定文本定位系统中应保留的六种变形关系，并定义了相应的稳定性评估指标。根据定义的变形关系，我们应用变形测试技术来比较输入和输出以评估系统稳定性，并进一步诊断不一致的原因。在学术和商业文本本地化系统上的广泛实验证明了我们的方法对此类系统的稳定性评估的有效性。