We propose a novel framework that combines state-of-the-art deep learning approaches with pre- and post-processing algorithms for particle detection in complex/heterogeneous backgrounds common in the manufacturing domain. Traditional methods, like size analyzers and those based on dilution, image processing, or deep learning, typically excel with homogeneous backgrounds. Yet, they often fall short in accurately detecting particles against the intricate and varied backgrounds characteristic of heterogeneous particle–substrate (HPS) interfaces in manufacturing. To address this, we've developed a flexible framework designed to detect particles in diverse environments and input types. Our modular framework hinges on model selection and AI-guided particle detection as its core, with preprocessing and postprocessing as integral components, creating a four-step process. This system is versatile, allowing for various preprocessing, AI model selections, and post-processing strategies. We demonstrate this with an entrainment-based particle delivery method, transferring various particles onto substrates that mimic the HPS interface. By altering particle and substrate properties (e.g., material type, size, roughness, shape) and process parameters (e.g., capillary number) during particle entrainment, we capture images under different ambient lighting conditions, introducing a range of HPS background complexities. In the preprocessing phase, we apply image enhancement and sharpening techniques to improve detection accuracy. Specifically, image enhancement adjusts the dynamic range and histogram, while sharpening increases contrast by combining the high pass filter output with the base image. We introduce an image classifier model (based on the type of heterogeneity), employing Transfer Learning with MobileNet as a Model Selector, to identify the most appropriate AI model (i.e., YOLO model) for analyzing each specific image, thereby enhancing detection accuracy across particle–substrate variations. Following image classification based on heterogeneity, the relevant YOLO model is employed for particle identification, with a distinct YOLO model generated for each heterogeneity type, improving overall classification performance. In the post-processing phase, domain knowledge is used to minimize false positives. Our analysis indicates that the AI-guided framework maintains consistent precision and recall across various HPS conditions, with the harmonic mean of these metrics comparable to those of individual AI model outcomes. This tool shows potential for advancing in-situ process monitoring across multiple manufacturing operations, including high-density powder-based 3D printing, powder metallurgy, extreme environment coatings, particle categorization, and semiconductor manufacturing.

我们提出了一种新颖的框架，它将最先进的深度学习方法与预处理和后处理算法相结合，用于制造领域常见的复杂/异构背景中的粒子检测。传统方法，如尺寸分析仪和基于稀释、图像处理或深度学习的方法，通常在均匀背景下表现出色。然而，它们通常无法准确检测制造过程中异质颗粒-基底 (HPS) 界面复杂且多样的背景特征的颗粒。为了解决这个问题，我们开发了一个灵活的框架，旨在检测不同环境和输入类型中的粒子。我们的模块化框架以模型选择和人工智能引导的粒子检测为核心，以预处理和后处理为组成部分，创建了一个四步流程。该系统用途广泛，可进行各种预处理、AI 模型选择和后处理策略。我们通过基于夹带的颗粒输送方法证明了这一点，将各种颗粒转移到模仿 HPS 界面的基材上。通过在颗粒夹带过程中改变颗粒和基底特性（例如材料类型、尺寸、粗糙度、形状）和工艺参数（例如毛细管数），我们在不同的环境照明条件下捕获图像，引入一系列 HPS 背景复杂性。在预处理阶段，我们应用图像增强和锐化技术来提高检测精度。具体来说，图像增强调整动态范围和直方图，而锐化则通过将高通滤波器输出与基础图像相结合来增加对比度。我们引入了图像分类器模型（基于异质性类型），采用 MobileNet 作为模型选择器的迁移学习，来识别最合适的 AI 模型（即 YOLO 模型）来分析每个特定图像，从而提高跨粒子的检测精度– 基材变化。基于异质性的图像分类后，采用相关的YOLO模型进行颗粒识别，针对每种异质性类型生成不同的YOLO模型，提高了整体分类性能。在后处理阶段，使用领域知识来最大限度地减少误报。我们的分析表明，人工智能引导的框架在各种 HPS 条件下保持一致的精度和召回率，这些指标的调和平均值与单个人工智能模型结果的调和平均值相当。该工具显示了在多个制造业务中推进现场过程监控的潜力，包括高密度粉末基 3D 打印、粉末冶金、极端环境涂层、颗粒分类和半导体制造。