Abstract In agriculture, many types of weeds have a harmful impact on agricultural productivity. Recognizing weeds and understanding the threat they pose to farmlands is a significant challenge because many weeds are quite similar in their external structure, making it difficult to classify them. A weeds classification approach with high accuracy and quick processing should be incorporated into automatic devices in smart agricultural systems to solve this problem. In this study, we develop a novel classification approach via a voting method by using the late fusion of multimodal Deep Neural Networks (DNNs). The score vector used for voting is calculated by either using Bayesian conditional probability-based method or by determining priority weights so that better DNNs models have a higher contribution to scoring. We experimentally studied the Plant Seedlings and Chonnam National University (CNU) Weeds datasets with 5 DNN models: NASNet, Resnet, Inception–Resnet, Mobilenet, and VGG. The results show that our methods achieved an accuracy of 97.31% on the Plant Seedlings dataset, and 98.77% accuracy on the CNU Weeds dataset. Furthermore, our framework can classify an image in near real-time.

中文翻译：

---

用于杂草分类的多模态深度神经网络的后期融合

摘要 在农业中，多种杂草对农业生产力产生有害影响。识别杂草并了解它们对农田构成的威胁是一项重大挑战，因为许多杂草的外部结构非常相似，因此很难对它们进行分类。应在智能农业系统的自动化设备中加入一种精度高、处理速度快的杂草分类方法来解决这个问题。在这项研究中，我们通过使用多模态深度神经网络 (DNN) 的后期融合，通过投票方法开发了一种新的分类方法。用于投票的得分向量是通过使用基于贝叶斯条件概率的方法或通过确定优先权重来计算的，以便更好的 DNN 模型对得分有更高的贡献。我们使用 5 个 DNN 模型对植物幼苗和全南大学 (CNU) 杂草数据集进行了实验研究：NASNet、Resnet、Inception–Resnet、Mobilenet 和 VGG。结果表明，我们的方法在 Plant Seedlings 数据集上达到了 97.31% 的准确率，在 CNU Weeds 数据集上达到了 98.77% 的准确率。此外，我们的框架可以近乎实时地对图像进行分类。