Automatic morphological extraction of fibers from SEM images for quality control of short fiber-reinforced composites manufacturing

Highlights

• We propose five different methods, namely, the opening method, simple Hough transform (HT), partitioning HT, gradient based HT and break-merge method to automatically segment short fibers from SEM images.

• This methodology can segment fibers in scanning electron microscope (SEM) images and extract useful information such as fiber orientation distribution and their agglomeration.

• The robustness and accuracy of the proposed methods are evaluated and compared through both simulation and real case studies.

• The extracted results (orientation distribution and fiber agglomeration) are statistically evaluated.

Abstract

The properties of fiber-reinforced composite materials greatly depend on the morphology of reinforcing fibers within the base materials, i.e., spatial uniformity, orientation, and length distribution. Accurately extracting this information in an automated manner from SEM images is essential for quality assessment, quality control, and process optimization. However, due to overlapping or cross-linking issue, morphological fiber extraction is very challenging and has not been well addressed in the existing literature. This paper takes into account this research gap and proposes five different methods, namely, the opening method, simple Hough transform (HT), partitioning HT, gradient-based HT, and break-merge method to automatically extract the straight fibers from SEM images to facilitate the morphological analysis. The robustness and accuracy of the proposed methods are evaluated and compared through both simulation and real case studies.

Introduction

Micro/nanofibers are noble reinforcements to increase the mechanical properties of composite materials. Fiber-reinforced composites exhibit promising advantages, such as high strength, high stiffness, and lightweight in comparison with conventional materials. These outstanding properties have led to their desirability and exploration in a wide range of applications. The spatial homogeneity, length, and alignment orientation of fibers in the base material play a decisive role in determining the final properties of composites [1], [2], [3]. For instance, with the increase of mean fiber length, the tensile strength increases significantly [4]. Besides, the composites have stronger mechanical properties in the direction of fiber alignment than other directions [5]. The desired orientation and length distribution largely depend on the application of composites. In structural applications, uniform distribution of fibers in terms of both spatial location and orientation is desirable to achieve the best isotropic mechanical properties. However, in other applications, the alignment of fibers in one direction may be preferable. Literature reports that the alignment of dielectric fillers in the direction of the applied electric field can significantly enhance the dielectric properties of the base material, especially the dielectric permittivity and breakdown strength [6], [7]. Besides the alignment, the spatial distribution is also crucial for material properties. Almost in all applications, the homogeneous spatial distribution of fibers in the specimen is required to achieve optimal performance. Nevertheless, similar to particle-reinforced nanocomposites [8], [9], [10], the clustering or aggregation of fibers often exists due to imperfectly controlled processing, which may significantly reduce the material properties [11]. Therefore, it is highly desirable to extract the fiber morphology for quality evaluation and process control.

Nowadays, the conventional quality inspection approach is to perform the morphology analysis of fibers through visual checking of the scanning electron microscope (SEM) images of the composite material. However, this approach is subjective and often very time-consuming. Besides, it may not be realistic to collect all the quantitative information of interest manually. Therefore, an automated fiber extraction method for morphological analysis is desirable. Image processing techniques have been successfully used for the morphology analysis of nanoparticles in the biomedical and material science field. For example, ImageJ, a popular freeware tool provided by NIH, is used for the extraction and quantification of fluorescence particles [12]. A Machine learning approach has also been introduced to locate the particles’ position in the images based on Haar features [13]. Ellipse fitting techniques and watershed algorithm are used to segment uniformly distributed particles from the background [14], [15]. The problem of segmenting overlapped particles is also well tackled [16]. However, there are very limited morphological extraction methodologies for fibers in the existing literature. Kimura et al. [17] proposed an algorithm to measure the root length of plants through skeleton or thinning operation. Kawabata et al. [18] developed an image processing technique to detect and count asbestos fibers using both color and shape information. Peng et al. [19] presented a set of image processing techniques to estimate the length, position, and orientation of nanowires in the fluidic workspace from optical section microscopy images. However, all of these methods are not applicable to fiber extraction of SEM images, where fibers are often overlapped or cross-linked with each other. Jeon et al. [20] developed a method to characterize the nanowire alignment in a microchannel using ridge detection, texton analysis, and autocorrelation function (ACF) calculation. The textons of different orientation angles are convoluted with the autocorrelation field to detect the distribution of wire alignment angles. This method can approximately estimate the orientation distribution. Nevertheless, it is not capable of identifying the fiber locations, which are essential for spatial homogeneity assessment.

To address the aforementioned issues, this paper develops and compares five different methods to automatically identify the fibers from SEM images. Of the five methods, one is based on the morphological opening operation, three are based on the Hough transform (HT) algorithm, namely, the simple Hough transform approach, partitioning Hough transform and gradient-based Hough transform, and the last one is based on the identification of cross points. Hough transform (HT) algorithm is a very efficient tool to identify a certain class of shapes, such as lines, circles, and ellipses, by a voting procedure [21]. The classic HT was used to detect straight lines in the image. Intuitively, the straight fibers can be identified by detecting the long boundaries or by detecting the skeleton after thinning process through the simple HT method. However, various issues make the simple HT not effective, especially when the fiber density is high. To solve these issues, this paper proposes two improved approaches, the partitioning Hough transform and gradient-based Hough transform. Besides, another innovative approach named break-merge (BM) method has also been proposed. In this method, the partitioning and morphological thinning operations are performed first, and then the DBSCAN clustering algorithm [22] is used to classify the skeleton into cross points and straight line points. After that, the cross points are removed to break connected fibers into shorter segments, and then the DBSCAN is used again to identify these segments as clusters. Finally, these short segments are matched or merged based on their distance and orientation to form complete fibers for morphological information extraction. The contribution of this paper is twofold. First, it develops four tailored techniques by nontrivially customizing the natural morphological operations, and also develops a brand new technique (BM) to fill the research gap of fiber extraction. Second, the shortcomings of these methods are discussed, the performance is evaluated and compared to provide insight and guidance for practitioners.

The rest of the paper is organized as follows. In Section “Methodologies for automatic fiber extraction”, the technical details of the proposed methods are presented. In Section “Simulation study for performance evaluation and comparison”, the proposed methods are evaluated and compared in terms of identification accuracy through simulation studies. The real case studies are given in Section “Application to real images”. The conclusion and discussion are provided in Section “Conclusion and discussion”.