A real-time deep-learning model is proposed to classify the volume of cuttings from a shale shaker on an offshore drilling rig by analyzing the real-time monitoring video stream. Compared with the traditional video-analytics method, which is time-consuming, the proposed model is able to implement a real-time classification and achieve remarkable accuracy. Our approach is composed of three modules: a multithread engine for decoding/encoding real-time video stream. The video streaming is provided by a modularized service named Rig-Site Virtual Presence, which enables aggregating, storing, transrating/transcoding, streaming, and visualization of video data from the rig; an automatic region-of-interest (ROI) selector. A deep-learning-based object-detection approach is implemented to help the classification model find the region containing the cutting flow; and a convolutional-neural-network-based classification model, which is pretrained with videos collected from previous drilling operations. Normalization and principal-component analyses (PCAs) are conducted before every video frame is fed into the classification model. The classification model classifies each frame into four labels (Extra Heavy, Heavy, Light, and None) in real time. The overall workflow has been tested on a video stream directed from an offshore drilling rig. The video stream has a bitrate of 137 Kbps, approximately 6 frames/sec (fps), and a frame size of 720 × 486. The training process is conducted on an Nvidia GeForce 1070 graphics processing unit (GPU). The testing process (classification inference) runs with only an i5-8500 central processing unit (CPU). Because of the multithreads processing and proper adaptation on the classification model, we are able to handle the entire workflow in real time. This allows us to receive a real-time video stream and display the classification results with encoded frames on the user-side screen at the same time. We use the confusion matrix as the metric to evaluate the performance of our model. Compared with results manually labeled by engineers, our model can achieve highly accurate results in real time without dropping frames.

提出了一种实时深度学习模型，通过分析实时监控视频流，对来自海上钻井平台的页岩振动筛的碎屑量进行分类。与费时的传统视频分析方法相比，该模型能够实现实时分类，并且具有很高的准确性。我们的方法由三个模块组成：用于解码/编码实时视频流的多线程引擎。视频流由名为Rig-Site Virtual Presence的模块化服务提供，该服务可对钻机中的视频数据进行汇总，存储，转换/转码，流化和可视化；自动关注区域（ROI）选择器。实施了一种基于深度学习的对象检测方法，以帮助分类模型找到包含切削流的区域。以及基于卷积神经网络的分类模型，该模型预先训练了从先前钻井操作中收集的视频。在将每个视频帧输入分类模型之前，先进行归一化和主成分分析（PCA）。分类模型将每个帧实时分类为四个标签（超重，重，轻和无）。整个工作流程已在海上钻井平台的视频流上进行了测试。视频流的比特率为137 Kbps，约为6帧/秒（fps），帧大小为720×486。训练过程在Nvidia GeForce 1070图形处理单元（GPU）上进行。测试过程（分类推断）仅在i5-8500中央处理器（CPU）上运行。由于多线程处理和对分类模型的适当调整，我们能够实时处理整个工作流程。这使我们可以接收实时视频流，并同时在用户侧屏幕上显示带有编码帧的分类结果。我们使用混淆矩阵作为衡量模型性能的指标。与工程师手动标记的结果相比，我们的模型可以实时获得高度准确的结果，而不会丢帧。这使我们可以接收实时视频流，并同时在用户侧屏幕上显示带有编码帧的分类结果。我们使用混淆矩阵作为衡量模型性能的指标。与工程师手动标记的结果相比，我们的模型可以实时获得高度准确的结果，而不会丢帧。这使我们能够接收实时视频流，并同时在用户侧屏幕上显示带有编码帧的分类结果。我们使用混淆矩阵作为衡量模型性能的指标。与工程师手动标记的结果相比，我们的模型可以实时获得高度准确的结果，而不会丢帧。