Deep learning-based optimal segmentation of 3D printed product for surface quality improvement and support structure reduction

doi:10.1016/j.jmsy.2021.06.007

Journal of Manufacturing Systems

Volume 60, July 2021, Pages 252-264

https://doi.org/10.1016/j.jmsy.2021.06.007 Get rights and content

Highlights

•
Features of 3D printing products are considered in the model segmentation.
•
Sub-graph data structure is proposed based on sub-graphs and printing features.
•
A deep-learning system is proposed to extract product features using Stacked Auto-encoders.
•
AP clustering is introduced to group sub-graphs into clusters.
•
The segmentation improves surface quality and reduces support structure of printed product.

Abstract

Large-sized product cannot be printed as one piece by a 3D printer because of the volume limitation of most 3D printers. Some products with the complex structure and high surface quality should also not be printed into one piece to meet requirement of the printing quality. For increasing the surface quality and reducing support structure of 3D printed models, this paper proposes a 3D model segmentation method based on deep learning. Sub-graphs are generated by pre-segmenting 3D triangular mesh models to extract printing features. A data structure is proposed to design training data sets based on the sub-graphs with printing features of the original 3D model including surface quality, support structure and normal curvature. After training a Stacked Auto-encoder using the training set, a 3D model is pre-segmented to build an application set by the sub-graph data structure. The application set is applied by the trained deep-learning system to generate hidden features. An Affinity Propagation clustering method is introduced in combining hidden features and geometric information of the application set to segment a product model into several parts. In the case study, samples of 3D models are segmented by the proposed method, and then printed using a 3D printer for validating the performance.

Introduction

Additive Manufacturing (AM) has made significant contributions to manufacturing practices over the last thirty years [1]. As most 3D printers for AM use a fixed printing plane to grow material in a single direction, the surface quality of a 3D printed product relies on its directionally layered printing process. It is therefore ideal that the surface normal of a 3D printing product model is perpendicular to the printing direction for a high quality of the product surface. In addition, when the surface normal vector is toward to the negative printing direction, or there is an angle larger than a standard between the surface and printing direction, it causes face overhang and needs adding support structure [2]. The support structure will deteriorate the product surface finish when the support material is removed in post-processing. The support structure also increases the material use.

Therefore, a 3D model should be printed following the surface direction for the high surface quality and less support structure. The model segmentation decomposes a 3D model into a set of parts to enable each part to be printed with an optimal printing direction for the high surface quality and less support structure [3]. In addition, a large-sized product model has to be printed in a few pieces because of the size limitation of the 3D printer. Thus, the model segmentation is proposed in this research.

Printing direction plays an important role in the printing time, printing surface quality and weights of support materials in 3D printing. Traditional methods only optimize a single global printing direction for the whole model to improve limitations of the surface quality and support materials [3]. To further improve the surface quality and reduce support materials, model segmentation methods have been proposed for the 3D printing process. For example, the curvature analysis was applied in the segmentation of a large-sized 3D mesh model for meeting the size limitation of 3D printers [4]. 3D models were divided into smaller shapes similar to the pyramid for reducing support materials and removing difficultly [5]. 3D mesh models were separated into multiple printable pieces to save the packing space of 3D printed products [6]. A 3D model segmentation method was formed for 3D printing models by the Ncut algorithm based on features of the surface quality [3]. A 3D model segmentation method was developed for 3D printed models based on spectral clustering by considering features of the surface quality and support materials [7]. The existing methods of the surface segmentation for 3D printing models have limitations in the feature extraction and clustering number decision. Self-tuning spectral clustering has been applied to estimate the cluster number [3]. Features including the curvature, surface quality and support materials can be considered respectively in the model segmentation for 3D printing, but there is not a method combing all the features.

The segmentation of a 3D printing model should consider relations of surface quality, support structure and shape of the 3D model. This paper proposes a 3D model segmentation method based on deep learning to increase the surface quality and reduce the support structure. Printing features of a product model are abstracted from the original 3D model including surface quality, support structure and normal curvature. A 3D product triangular mesh model is pre-segmented into sub-graphs. The sub-graph data structure is built by features extracting from the 3D mesh model to train a Stacked Auto-encoder (SAE) that form a training set for deep learning. After the SAE is trained using the training set, a 3D model is pre-segmented to build an application set with the sub-graph data structure. The application set is then used in the trained deep-learning system to generate hidden features. Affinity Propagation (AP) clustering method is finally used to combine the hidden features and geometric information of the application set into several clusters for printing parts of the model. Contributions of this paper are as follows.

1)
Printing features of the surface quality, support structure and normal curvature are introduced in the model segmentation for extracting hidden features.
2)
A 3D product triangular mesh model is segmented into sub-graphs based on triangles to reduce dimensions of the clustering process and increase the clustering speed. The sub-graphs are abstracted from a few connected triangles rather than a single triangle.
3)
A sub-graph data structure is proposed based on sub-graphs to extract hidden features to be combined with the geometric information based on a trained SAE.
4)
Hidden features are combined with the geometric distance to build a similarity matrix in the clustering process to increase the surface quality and reduce the support structure of 3D printed parts.
5)
AP clustering is introduced to cluster sub-graphs of a 3D product model into several clusters for improved automatic decision-making of the number of clusters.

Following parts of this paper are organized as follows. Section 2 reviews the existing research on the model segmentation. Section 3 introduces the proposed model segmentation method. Section 4 describes the extraction of 3D printing features. A deep-learning system is proposed in Section 5 to elaborate the proposed method. Common surface optimization and connector design are discussed in Sections 6, followed by case studies in Section 7 to validate performance of the proposed method. Conclusions and future work are discussed in Section 8.

Section snippets

Literature review

Product model segmentation is widely applied in many areas of geometrical modeling and applications such as the component-based shape synthesis, style transfer, modeling by example, modeling from interchangeable parts, 3D scene analysis, part-based recognition, 3D video compression and 3D object retrieval using methods of the region growing, surface fitting, clustering, spectral analysis, and neural networks [8].

As a common technique of the 3D model segmentation, the region growing usually

Proposed method of the model segmentation

This paper proposes a deep learning-based segmentation method to improve the surface quality and reduce support structure of 3D printing models. The method consists of three main steps: printing feature extraction, system training, and model segmentation.

Printing features are generated from an original 3D model considering the normal curvature, surface quality and support structure. In the training process, the 3D model is first pre-segmented into sub-graphs to build a training model base. The

Surface quality in 3D printing

A 3D triangular mesh model M is composed of a triple ${V, E, F}$ with m vertices $V = {v_{i} | v_{i} \in R^{3}, 1 \leq i \leq m}$ , edges $E = {e_{i j} = (v_{i}, v_{j}) | v_{i}, v_{j} \in V, i \neq j}$ , and triangles $F = {f_{i j k} = (v_{i}, v_{j}, v_{k}) | v_{i}, v_{j}, v_{k} \in V, i \neq j, j \neq k, i \neq k}$ .

3D printing is a layered manufacturing process and the surface quality relies on the printing direction. Fig. 2(a) shows the relation between outer surfaces of a 3D printed model and printing directions. An outer surface of the 3D printing model should be parallel to the printing direction for a high surface

Sub-graph data structure

A sub-graph data structure is proposed to generate a data set for training the SAE to form an application set of hidden features for the model segmentation. For example, a sub-graph with 9 triangles is shown in Fig. 5(a).

The sub-graph data structure is a matrix formed by triangles, adjacent triangles and features. In a sub-graph, an inner triangle has three adjacent triangles, and a boundary triangle has one or two adjacent triangles. If a triangle has three adjacent triangles, the position of

Common surface optimization

As a common surface between two adjacent parts is non-planar after clustering, the difficulty of assembling all the parts together increases. For reducing the difficulty of shaping and assembly of parts in the segmented model, Support Vector Machines (SVM) is introduced to fit a planar between two adjacent parts. The 3D model is finally segmented based on the fitted planes.

A common plane P: $x^{T} W + b = 0$ is generated for each pair of two adjacent patches. Each SVM plane P_ij can generate an optimal

Simulation of the 3D model segmentation

3D triangular mesh models are used to validate the proposed method. Eight models are applied including models of airplane, ant, bird, chair, fish, fourleg, glass and teddy as listed in Table 1. For segmenting the model with the proposed method, all the eight models are transferred into the application data set based on the method proposed in Section 5.1. As some triangles are not included in the data structure, the triangle usage rate is defined for the percentage of triangles used in the data

Conclusions and future work

This paper proposed a 3D model segmentation method based on deep learning to increase surface quality and reduce support structure for large-sized models and complex models that cannot be printed in one piece by a 3D printer. Printing features were abstracted from each triangle of a 3D mesh model including normal curvature, surface quality and support structure. Sub-graphs were formed by pre-segmenting 3D triangular mesh models for printing features. The sub-graph data structure was proposed to

Declaration of Competing Interest

The authors report no declarations of interest.

Acknowledgements

The authors wish to acknowledge that this research has been supported by the Discovery Grants from the Natural Sciences and Engineering Research Council (NSERC) of Canada, and the Graduate Enhancement of Tri-Council Stipends (GETS) program from the University of Manitoba.

References (39)

Y. Jin et al.
Optimization of process planning for reducing material consumption in additive manufacturing
J Manuf Syst
(2017)
N. Gardan et al.
Topological optimization of internal patterns and support in additive manufacturing
J Manuf Syst
(2015)
P. Theologou et al.
A comprehensive overview of methodologies and performance evaluation frameworks in 3D mesh segmentation
Comput Vis Image Underst
(2015)
M. Vieira et al.
Surface mesh segmentation and smooth surface extraction through region growing
Comput Aided Geom Des
(2005)
D.M. Yan et al.
Variational mesh segmentation via quadric surface fitting
Comput Aided Des
(2012)
J. Chen et al.
Automatic extraction of discontinuity orientation from rock mass surface 3D point cloud
Comput Geosci
(2016)
X. Jiao et al.
Mesh segmentation by combining mesh saliency with spectral clustering
J Comput Appl Math
(2018)
J. Wang et al.
Deep learning for smart manufacturing: methods and applications
J Manuf Syst
(2018)
P. Wang et al.
3D shape segmentation via shape fully convolutional networks
Comput Geosci
(2018)
Z. Shu et al.
Unsupervised 3D shape segmentation and co-segmentation via deep learning
Comput Aided Geom Des
(2016)

J.L. Shih et al.

A new 3D model retrieval approach based on the elevation descriptor

Pattern Recognit

(2007)

W. Mao et al.

Online detection of bearing incipient fault with semi-supervised architecture and deep feature representation

J Manuf Syst

(2020)

L. Ren et al.

Bearing remaining useful life prediction based on deep autoencoder and deep neural networks

J Manuf Syst

(2018)

L. Zhang et al.

Comparison of auto-encoders with different sparsity regularizers

2015 International Joint Conference on Neural Networks (IJCNN)

(2015)

J. Zabalza et al.

Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging

Neurocomputing

(2016)

W.M. Wang et al.

Improved surface quality in 3D printing by optimizing the printing direction

Comput Graph Forum

(2016)

J. Hao et al.

An efficient curvature-based partitioning of large-scale STL models

Rapid Prototyp J

(2011)

R. Hu et al.

Approximate pyramidal shape decomposition

ACM Trans Graph

(2014)

M. Yao et al.

Level-set-based partitioning and packing optimization of a printable model

ACM Trans Graph

(2015)

Cited by (7)

Contrastive learning-based semantic segmentation for In-situ stratified defect detection in additive manufacturing
2023, Journal of Manufacturing Systems
Supervised semantic segmentation has been widely utilized for the quality assurance of the additive manufacturing process. However, abnormal states (those with defects) happen significantly less frequently than normal ones (those without defects), resulting in the collected dataset usually containing fewer defect samples than normal ones, which deteriorates the performance of defect detection in additive manufacturing. To address this issue, a novel contrastive learning-based semantic segmentation model, termed cLass-aware Semantic Contrast and Attention amalgamation (LSCA), is proposed to accurately detect the in-situ stratified defect with imbalanced data. LSCA is equipped with a defect memory bank which is devised to store and retrieve class-aware semantic memory from poorly labeled training data. In particular, class-aware semantic contrast is devised to drive network learning by contrasting all categorical defect regions, leading to a more holistic defect pattern understanding. Further, class-aware attention amalgamation is developed to assemble various relationship contexts in the defect memory to enrich semantic representations. The extensive experiments demonstrate that, despite the imbalanced labeling of the training dataset, our LSCA consistently outperforms cutting-edge semantic segmentation techniques for detecting in-situ stratified defects in additive manufacturing.
Federated learning-based semantic segmentation for pixel-wise defect detection in additive manufacturing
2022, Journal of Manufacturing Systems
Citation Excerpt :
A three-dimensional convolutional neural network (CNN) is used for volumetric defect segmentation in X-ray computed tomography (XCT) images of AM samples [7]. Other applications include fabrication of medical constructs [8], measurement and monitoring of melt pool geometry [9], and surface quality improvement [10]. Most conventional semantic segmentation models are built around CNN architectures which are inherently data hungry and require large high-quality datasets for training [11].
Semantic segmentation is a promising machine learning (ML) method for highly precise fine-scale defect detection and part qualification in additive manufacturing (AM). Most existing segmentation methods utilize convolutional neural network architectures that require large quantities of training data. However, obtaining sufficient data—both in quality and quantity—to train such models is expensive and time-consuming for individual AM practitioners, which severely limits the deployment of semantic segmentation in a data-scarce production environment. Similar data may be readily available with other AM practitioners that cannot be pooled together for conventional centralized learning (CL) due to its sensitive nature or conflicts of interest. This paper develops a federated learning (FL)-based method to simultaneously alleviate the constraints of data availability and data privacy. A U-Net architecture is created for semantic segmentation and is trained under the FL framework. The effectiveness of the developed FL-based semantic segmentation approach is demonstrated using case studies on layer-wise images from the laser powder bed fusion process. Results show that the proposed technique achieves a comparable defect detection performance with CL, which shares data among manufacturers/clients but does not preserve data privacy, and significantly outperforms individual learning, where each manufacturer trains a model using its own data. Additionally, the impact of data distribution across clients, incentives to participate in FL, and the learning dynamics of FL are discussed in detail. It is found that data diversity within and across clients improves FL performance, and FL does not involve a significantly higher training cost compared to CL. Lastly, transfer learning is shown to enhance FL generalizability, thus allowing more manufacturers with heterogeneous machines or technologies to benefit from participating in a data federation. Overall, this work puts forth FL as a promising paradigm for privacy-preserving collaborative ML in AM process control.
A kinematics-aware decomposition approach for complex CAD parts in additive manufacturing
2022, Additive Manufacturing
Citation Excerpt :
To minimize the visual quality affected by support structures and decomposition, Filoscia et al. [39] defined non-planar decompositions that adapt to the shape of a part and decomposed the part into pieces mainly using over-segmentation, merging, and refining operations. To increase a part’s surface quality while decreasing its support structures, Li et al. [40] proposed a part decomposition approach based on deep learning, which combines a stacked auto-encoder, the descriptor of each pre-segmented sub-graph, and an affinity propagation clustering method. Generally speaking, the works about part decomposition in AM at present are mainly designed by considering printability, productivity, and/or artistry [34].
Improving the surface quality of each assembly interface on a CAD part in Additive Manufacturing (AM) can effectively promote the part’s kinematic performance in a product or a prototype. To achieve this goal, decomposition is frequently unavoidable for a complex CAD part when it has two or more assembly interfaces respectively requiring different fabrication orientations to achieve their best surface quality. Accordingly, a smart and kinematics-aware part decomposition approach is proposed in this paper, in viewing that the related automatic works seldom ensure the achievement of the above goal while the manual operation is usually experience-intensive and time-consuming. First, the new criteria for kinematics-aware part decomposition are defined under the standard guidelines in ISO/ASTM 52910. Meanwhile, the heuristic method to determine the kinematics-aware fabrication orientation (to achieve the best surface quality) for an assembly interface is also presented. Based on them, a novel dual-process method is designed to decompose a CAD part from coarse to fine. It presents an assembly-interface-group-based method to plan the preliminary decomposition scheme, and then develops an optimization method to accurately determine all the decomposition planes based on the above scheme. Finally, experiments on six representative CAD parts are implemented to verify the effectiveness of the proposed approach. Comparing with the approaches that improve global and local surface quality based on optimizing fabrication orientation, the proposed approach increases more than 82% in the surface quality of the assembly interfaces on each part respectively. Besides, the methodological comparisons with the state-of-the-art approaches are also implemented. The results show that the proposed approach is more suitable for the applications where parts are fabricated using AM for the products or prototypes with mechanisms.
The evaluation of enterprise supply chain intelligent manufacturing system for agricultural interconnection data based on machine learning
2024, Expert Systems
Power spectral density moment of having defective 3D printed plastic beams under moving load based on deep learning
2024, Journal of Intelligent Manufacturing
An autonomous intelligent framework for optimal orientation detection in 3D printing
2023, International Journal of Computer Integrated Manufacturing

View all citing articles on Scopus

View full text

Deep learning-based optimal segmentation of 3D printed product for surface quality improvement and support structure reduction

Highlights

Abstract

Introduction

Section snippets

Literature review

Proposed method of the model segmentation

Surface quality in 3D printing

Sub-graph data structure

Common surface optimization

Simulation of the 3D model segmentation

Conclusions and future work

Declaration of Competing Interest

Acknowledgements

J Manuf Syst

J Manuf Syst

Comput Vis Image Underst

Comput Aided Geom Des

Comput Aided Des

Comput Geosci

J Comput Appl Math

J Manuf Syst

Comput Geosci

Comput Aided Geom Des

Pattern Recognit

J Manuf Syst

J Manuf Syst

2015 International Joint Conference on Neural Networks (IJCNN)

Neurocomputing

Improved surface quality in 3D printing by optimizing the printing direction

Comput Graph Forum

An efficient curvature-based partitioning of large-scale STL models

Rapid Prototyp J

Approximate pyramidal shape decomposition

ACM Trans Graph

Level-set-based partitioning and packing optimization of a printable model

ACM Trans Graph