Non-numerical data, such as images and inspection records, contain information about industrial system degradation, but they are rarely used for failure prognostic tasks given the difficulty of automatic analysis. In this work, we present a novel method for prognostics using multimodal data, i.e. both numerical and non-numerical data. The proposed method is based on the development of a multi-branch Deep Neural Network (DNN), each branch of which is a neural network designed for processing a certain type of data. The method is applied to a case study properly designed to reproduce the problem of prognostics using multimodal data by referring to the operation of steam generators. The results show that it is able to accurately predict future degradation level using multimodal data, outperforming other methods using fewer sources of information.

非数字数据（例如图像和检查记录）包含有关工业系统退化的信息，但是鉴于自动分析的困难，它们很少用于故障预测任务。在这项工作中，我们提出了一种使用多峰数据（即数值和非数值数据）进行预测的新颖方法。所提出的方法基于多分支深度神经网络（DNN）的开发，该网络的每个分支都是为处理某种类型的数据而设计的神经网络。该方法适用于案例研究，该案例研究经过适当设计，可通过参考蒸汽发生器的操作，使用多模式数据重现预后问题。结果表明，它能够使用多模态数据准确预测未来的降解水平，而使用较少信息源的性能优于其他方法。