Background/foreground separation is one of the most fundamental tasks in computer vision, especially for video data. Robust PCA (RPCA) and its tensor extension, namely, Robust Tensor PCA (RTPCA), provide an effective framework for background/foreground separation by decomposing the data into low-rank and sparse components, which contain the background and the foreground (moving objects), respectively. However, in real-world applications, the video data is contaminated with noise. For example, in metal additive manufacturing (AM), the processed X-ray video to study melt pool dynamics is very noisy. RPCA and RTPCA are not able to separate the background, foreground, and noise simultaneously. As a result, the noise will contaminate the background or the foreground or both. There is a need to remove the noise from the background and foreground. To achieve the three components decomposition, a smooth sparse Robust Tensor Decomposition (SS-RTD) model is proposed to decompose the data into static background, smooth foreground, and noise, respectively. Specifically, the static background is modeled by the low-rank tucker decomposition, the smooth foreground (moving objects) is modeled by the spatio-temporal continuity, which is enforced by the total variation regularization, and the noise is modeled by the sparsity, which is enforced by the $\ell _{1}$ norm. An efficient algorithm based on alternating direction method of multipliers (ADMM) is implemented to solve the proposed model. Extensive experiments on both simulated and real data demonstrate that the proposed method significantly outperforms the state-of-the-art approaches for background/foreground separation in noisy cases. Note to Practitioners—This work is motivated by melt pool detection in metal additive manufacturing where the processed X-ray video from the monitoring system is very noisy. The objective is to recover the background with porosity defects and the foreground with melt pool in the presence of noise. Existing methods fail to separate the noise from the background and foreground since RPCA and RTPCA have only two components, which cannot explain the three components in the data. This paper puts forward a smooth sparse Robust Tensor Decomposition by decomposing the tensor data into low-rank, smooth, and sparse components, respectively. It is a highly effective method for background/foreground separation in noisy case. In the case studies on simulated video and X-ray data, the proposed method can handle non-additive noise, and even the case of high noise-ratio. In the proposed algorithm, there is only one tuning parameter $\lambda $ . Based on the case studies, our method achieves satisfying performance by taking any $\lambda \in [{0.2,1}]$ with anisotropic total variation regularization. With this observation, practitioners can apply the proposed method without extensive parameter tuning work. Furthermore, the proposed method is also applicable to other popular industrial applications. Practitioners can use the proposed SS-RTD for degradation processes monitoring, where the degradation image contains the static background, anomaly, and random disturbance, respectively.

中文翻译：

---

基于鲁棒张量分解的噪声视频背景/前景分离及其在增材制造中的应用

背景/前景分离是计算机视觉中最基本的任务之一，尤其是对于视频数据。鲁棒 PCA (RPCA) 及其张量扩展，即鲁棒张量 PCA (RTPCA)，通过将数据分解为包含背景和前景（移动对象）的低秩和稀疏分量，为背景/前景分离提供了有效的框架）， 分别。然而，在实际应用中，视频数据被噪声污染。例如，在金属增材制造 (AM) 中，用于研究熔池动力学的经过处理的 X 射线视频非常嘈杂。RPCA 和 RTPCA 不能同时分离背景、前景和噪声。结果，噪声将污染背景或前景或两者。需要去除背景和前景的噪声。为了实现三分量分解，提出了一种平滑稀疏鲁棒张量分解（SS-RTD）模型，将数据分别分解为静态背景、平滑前景和噪声。具体来说，静态背景由低秩塔克分解建模，平滑前景（运动物体）由时空连续性建模，由全变正则化强制执行，噪声由稀疏性建模，这由 $\ell _{1}$规范。一种基于交替方向乘数法 (ADMM) 的高效算法被用来求解所提出的模型。对模拟和真实数据的广泛实验表明，所提出的方法在嘈杂情况下明显优于背景/前景分离的最新方法。从业者须知——这项工作的动机是金属增材制造中的熔池检测，其中来自监控系统的处理过的 X 射线视频非常嘈杂。目的是在存在噪声的情况下恢复具有孔隙率缺陷的背景和具有熔池的前景。现有方法无法将噪声从背景和前景中分离出来，因为 RPCA 和 RTPCA 只有两个分量，无法解释数据中的三个分量。本文通过将张量数据分别分解为低秩、平滑和稀疏分量，提出了平滑稀疏的鲁棒张量分解。这是一种在嘈杂情况下进行背景/前景分离的高效方法。在模拟视频和X射线数据的案例研究中，所提出的方法可以处理非加性噪声，甚至是高噪声比的情况。在提出的算法中，只有一个调整参数 $\λ$ . 基于案例研究，我们的方法通过采取任何方式获得令人满意的性能 $\lambda\in [{0.2,1}]$具有各向异性总变差正则化。通过这种观察，从业者可以应用所提出的方法，而无需进行大量的参数调整工作。此外，所提出的方法也适用于其他流行的工业应用。从业者可以使用所提出的 SS-RTD 进行退化过程监测，其中退化图像分别包含静态背景、异常和随机干扰。