当前位置: X-MOL 学术Expert Syst. Appl. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A cross-disciplinary comparison of multimodal data fusion approaches and applications: Accelerating learning through trans-disciplinary information sharing
Expert Systems with Applications ( IF 8.5 ) Pub Date : 2020-08-27 , DOI: 10.1016/j.eswa.2020.113885
Rohit Bokade , Alfred Navato , Ruilin Ouyang , Xiaoning Jin , Chun-An Chou , Sarah Ostadabbas , Amy V. Mueller

Multimodal data fusion (MMDF) is the process of combining disparate data streams (of different dimensionality, resolution, type, etc.) to generate information in a form that is more understandable or usable. Despite the explosion of data availability in recent decades, as yet there is no well-developed theoretical basis for multimodal data fusion, i.e., no way to determine a priori which approach is best suited to combine an arbitrary set of available data to achieve a stated goal for a given application. This has resulted in exploration of a wide variety of approaches across numerous domains but as yet very little integration of conclusions at a meta (cross-disciplinary) level. In response, this manuscript poses the following questions: (1) How convergent (or divergent) are approaches within single disciplines? (2) How similar are the challenges posed across different disciplines, i.e., might there be opportunity for successes in MMDF achieved in one field to inform progress in other areas as well? and (3) Where are the outstanding gaps in MMDF research, and what does this imply as targets for high impact research in the coming years? To begin to answer these questions, an apples-to-apples comparison of the literature of nine stakeholder-centric engineering domains (civil engineering, transportation, energy, environmental engineering, food engineering, critical care (healthcare), neuroscience, manufacturing/automation, and robotics) was created by quantifying the numbers and dimensionalities of modalities and sensors in each published project and classifying the algorithms used and purposes for which they are used. Within disciplines, it is shown there is often a tendency for use of similar methodologies, both in choice of level of fusion and data algorithm class. Yet this analysis also reveals that many problem types (defined by data dimensionality, modality number and type, and fusion purpose) are shared across different domains and are approached differently in those domains, e.g., transportation problems have similar characteristics to critical care, food science, robotics, and civil engineering. Of the disciplines studied, most (>75%) share problem characteristics with 3–5 others; to support leveraging these resources, lookup tables indexed by data dimensions, number of modalities, etc. are provided as a starting point for cross-disciplinary MMDF literature searches for new applications. Critical gaps identified are (1) a drop off of the number of published studies with increasing number of distinct modalities and (2) a dearth of publications tackling challenges with high dimensionality inputs, especially time-series 2D and 3D data. These gaps may point to topics where algorithm development will be fruitful to enable future solutions as video and other high-dimensionality sensors decrease in price. Finally, the lack of a shared vocabulary across disciplines makes analyses like the one conducted here challenging, as does the often implicit incorporation of expert knowledge into design; therefore progress toward a better leveraging of the current state of knowledge and toward a theoretical MMDF framework depends critically on improved cross-disciplinary communication and coordination on this topic.



中文翻译:

多模式数据融合方法和应用的跨学科比较:通过跨学科信息共享加速学习

多模式数据融合(MMDF)是组合不同数据流(具有不同维度,分辨率,类型等)的过程,以更易于理解或使用的形式生成信息。尽管近几十年来数据可用性爆炸式增长,但到目前为止,多模式数据融合尚无完善的理论基础,即无法确定先验条件哪种方法最适合组合任意组可用数据以实现给定应用程序的既定目标。这导致对许多领域的多种方法进行了探索,但在元(跨学科)水平上结论的整合却很少。作为回应,该手稿提出了以下问题:(1)单一学科中方法的趋同性(或趋异性)?(2)不同学科之间所面临的挑战有多相似,即,是否有机会在一个领域取得MMDF的成功,从而为其他领域的进展提供信息?(3)MMDF研究中的突出差距在哪里?这意味着未来几年高影响力研究的目标是什么?要开始回答这些问题,通过量化创建了以利益相关者为中心的九个工程领域(土木工程,交通运输,能源,环境工程,食品工程,重症监护(卫生保健),神经科学,制造/自动化和机器人技术)文献的逐一比较。每个已发布项目中模态和传感器的数量和维度,并对所使用的算法和用途进行分类。在学科内部,表明在融合级别的选择和数据算法类别的选择上,往往存在使用相似方法的趋势。然而,该分析还表明,许多问题类型(由数据维,模态数量和类型以及融合目的定义)在不同领域之间共享,并且在这些领域中的处理方式不同,例如,运输问题与重症监护,食品科学,机器人技术和土木工程具有相似的特征。在所研究的学科中,大多数(>75%)与其他3-5个人分享问题特征;为了支持利用这些资源,提供了按数据维度,模态数量索引的查找表,作为跨学科MMDF的起点文献搜索新的应用。所确定的关键差距是(1)随着不同模式数量的增加,发表的研究数量下降;(2)缺乏应对高维度输入(尤其是时间序列2D和3D数据)挑战的出版物。这些差距可能指向一些主题,在这些主题中,随着视频和其他高维度传感器的价格下降,算法开发将为将来的解决方案提供丰硕的成果。最后,由于缺乏跨学科的共享词汇,使得像此处进行的分析一样具有挑战性,因为通常会隐含地将专家知识整合到设计中。

更新日期:2020-08-27
down
wechat
bug