The automatic deep understanding of technical documents is a privilege only to humans so far, since it requires knowledge coming from many different modalities, like text, diagrams, formulas, tables, graphics, pictures, etc. Thus, in response to this very large and complex challenge, this paper investigates the synergistic association of only two modalities, the diagrams as main modality and natural language text as an assistive one in an effort to combine them together for deeper understanding of technical documents. In particular, it presents the formal modelling of a hybrid methodology capable to automatically extract the structural and functional behavior of a system described in a technical document without the use of original code. By system here we mean the block diagram(s) of a system. The methodology presented here is based on a formal language, called Synergy, to efficiently represent and synthesize the structural features of the system, and convert them into a Stochastic Petri-nets (SPN) model as for expressing the functional behavior of the understudy system. The overall methodology will contribute to an automatic deep understanding of technical documents (TD) without the main involvement of human users.

中文翻译：

---

深入理解技术文档：第一部分图结构-功能建模

迄今为止，对技术文档的自动深入理解是人类的一项特权，因为它需要来自许多不同形式的知识，如文本、图表、公式、表格、图形、图片等。因此，为了应对这个非常庞大和复杂的挑战，本文研究了两种模式的协同关联，图表作为主要模式，自然语言文本作为辅助模式，努力将它们结合在一起，以更深入地理解技术文档。特别是，它展示了一种混合方法的正式建模，该方法能够自动提取技术文档中描述的系统的结构和功能行为，而无需使用原始代码。这里的系统是指系统的框图。这里介绍的方法基于一种称为 Synergy 的形式语言，以有效地表示和综合系统的结构特征，并将它们转换为随机 Petri 网 (SPN) 模型，以表达研究系统的功能行为。整个方法将有助于自动深入理解技术文档 (TD)，而无需人类用户的主要参与。