当前位置: X-MOL 学术Image Vis. Comput. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A Survey on Object Detection for the Internet of Multimedia Things (IoMT) using Deep Learning and Event-based Middleware: Approaches, Challenges, and Future Directions
Image and Vision Computing ( IF 4.7 ) Pub Date : 2020-12-29 , DOI: 10.1016/j.imavis.2020.104095
Asra Aslam , Edward Curry

An enormous amount of sensing devices (scalar or multimedia) collect and generate information (in the form of events) over the Internet of Things (IoT). Present research on IoT mainly focus on the processing of scalar sensor data events and barely considers the challenges posed by multimedia based events. In this paper, we systematically review the existing solutions available for the Internet of Multimedia Things (IoMT) by analyzing sensing, networking, service, and application-level services provided by IoT. We present state-of-the-art event-based middleware methods and their suitability for multimedia event processing methods. We observe that existing IoT event-based middleware solutions focus on structured (scalar) events and possess only domain-specific characteristics for unstructured (multimedia) events. A case study for object detection is also presented to demonstrate the requirements associated with the processing of multimedia events within smart cities, even with common image recognition based applications. In order to validate the existing issues in the detection of objects, we also presented an evaluation of object detection models using existing datasets. At the end of each section, we shed light on trends, gaps, and possible solutions based on our analysis, experiments, and review of the existing research. Finally, we summarize the challenges and future research directions for the generalized multimedia event processing (by taking detection of each and every object as an example) based on applications using IoMT. Our experiments demonstrate that existing models are very slow to respond to any unseen class, and existing rich datasets do not have a sufficient number of classes to meet the requirements of real-time applications of smart cities. We show that although there is a significantly large technical literature on IoT, and research on IoMT is also quite actively growing, there have not been much research efforts directed towards the processing of multimedia events. As an example, although deep learning techniques have been shown to achieve impressive performance in applications like image recognition, the methods are deficient in detecting new (previously unseen) objects for multimedia based applications in smart cities. In light of these facts, it becomes imperative to conduct research on bringing together the abilities of event-based middleware for IoMT, and low response-time based online training and adaptation techniques.



中文翻译:

使用深度学习和基于事件的中间件对多媒体物联网(IoMT)进行对象检测的调查:方法,挑战和未来方向

大量传感设备(标量或多媒体)通过物联网(IoT)收集并生成信息(以事件的形式)。当前对物联网的研究主要集中在标量传感器数据事件的处理上,而很少考虑基于多媒体的事件带来的挑战。在本文中,我们通过分析传感网络服务应用程序级别,系统地回顾了可用于多媒体物联网(IoMT)的现有解决方案。物联网提供的服务。我们介绍了基于事件的最新中间件方法及其对多媒体事件处理方法的适用性。我们观察到,现有的基于IoT事件的中间件解决方案专注于结构化(标量)事件,而对于非结构化(多媒体)事件仅具有特定于域的特征。还提出了一个用于对象检测的案例研究,以证明与智能城市内的多媒体事件处理相关的要求,即使是基于普通图像识别的应用程序也是如此。为了验证对象检测中的现有问题,我们还提出了使用现有数据集的对象检测模型的评估。在每个部分的结尾,我们都会根据我们的分析,实验和对现有研究的回顾,阐明趋势,差距和可能的解决方案。最后,我们总结了基于基于IoMT的应用程序的通用多媒体事件处理(以检测每个对象为例)的挑战和未来的研究方向。我们的实验表明,现有模型对任何看不见的类别的响应都非常慢,并且现有的丰富数据集没有足够数量的类别来满足智能城市实时应用的要求。我们表明,尽管有关物联网的技术文献非常多,并且有关IoMT的研究也在相当活跃地发展,但是针对多媒体事件的处理却没有进行太多的研究。举例来说,尽管已证明深度学习技术在图像识别等应用中取得了令人印象深刻的性能,这些方法不足以为智能城市中基于多媒体的应用检测新的(以前看不见的)对象。鉴于这些事实,有必要进行研究以将基于事件的中间件IoMT的功能以及基于低响应时间的在线培训和适应技术结合在一起。

更新日期:2020-12-29
down
wechat
bug