Skip to main content
SpringerLink
Log in

A Framework for the Systematic Design of Segmentation Workflows

  • Technical Article
  • Published:
Integrating Materials and Manufacturing Innovation Aims and scope Submit manuscript

Abstract

Segmentation of microscopy images is an essential step in most experimental studies of process–structure–property relationships in advanced materials. Currently employed segmentation approaches require the user to identify and string together a sequence of algorithms (and codes) into customized workflows that need extensive tweaking and optimization (often accomplished through repeated trials) for producing the most reliable results for each set of images. Recent advances in materials characterization instruments have significantly increased the throughput and variety of microscopy images that could be generated in the efforts to document and understand the material internal structure. There is a critical need for a guiding framework for the systematic design of segmentation workflows that can eventually lead to fully automated segmentation workflows. In this work, we propose one such modular framework consisting of five sequential steps that is applicable to segmentation of a broad variety of microscopy images. Each step is designed to accomplish a specific subtask in the overall segmentation using available functions in popular software packages. Furthermore, the modular nature of the framework allows the user to explore alternate functions in each step, while systematically comparing their relative efficacies. We describe this new segmentation framework in this paper and demonstrate its value through case studies involving a variety of microstructures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Santofimia M, Zhao L, Petrov R, Sietsma J (2008) Characterization of the microstructure obtained by the quenching and partitioning process in a low-carbon steel. Mater Charact 59(12):1758–1764

    CAS  Google Scholar 

  2. Uchic MD, Groeber MA, Dimiduk DM, Simmons J (2006) 3D microstructural characterization of nickel superalloys via serial-sectioning using a dual beam FIB-SEM. Scr Mater 55(1):23–28

    CAS  Google Scholar 

  3. Cantor B, Chang I, Knight P, Vincent A (2004) Microstructural development in equiatomic multicomponent alloys. Mater Sci Eng A 375:213–218

    Google Scholar 

  4. Otto F, Dlouhý A, Pradeep KG et al (2016) Decomposition of the single-phase high-entropy alloy CrMnFeCoNi after prolonged anneals at intermediate temperatures. Acta Mater 112:40–52

    CAS  Google Scholar 

  5. Tan C, Zhou K, Ma W et al (2017) Microstructural evolution, nanoprecipitation behavior and mechanical properties of selective laser melted high-performance grade 300 maraging steel. Mater Des 134:23–34

    CAS  Google Scholar 

  6. Morito S, Huang X, Furuhara T et al (2006) The morphology and crystallography of lath martensite in alloy steels. Acta Mater 54(19):5323–5331

    CAS  Google Scholar 

  7. Timokhina IB, Hodgson PD, Pereloma E (2007) Transmission electron microscopy characterization of the bake-hardening behavior of transformation-induced plasticity and dual-phase steels. Metall Mater Trans A 38(10):2442–2454

    Google Scholar 

  8. Sarvghad-Moghaddam M, Parvizi R, Davoodi A et al (2014) Establishing a correlation between interfacial microstructures and corrosion initiation sites in Al/Cu joints by SEM–EDS and AFM–SKPFM. Corros Sci 79:148–158

    CAS  Google Scholar 

  9. Hu T, Shi H, Hou D et al (2019) A localized approach to study corrosion inhibition of intermetallic phases of AA 2024-T3 by cerium malate. Appl Surf Sci 467:1011–1032

    Google Scholar 

  10. Li D, Guo Q, Guo S et al (2011) The microstructure evolution and nucleation mechanisms of dynamic recrystallization in hot-deformed Inconel 625 superalloy. Mater Des 32(2):696–705

    CAS  Google Scholar 

  11. Paredes-Orta CA, Mendiola-Santibañez JD, Manriquez-Guerrero F, Terol-Villalobos IR (2019) Method for grain size determination in carbon steels based on the ultimate opening. Measurement 133:193–207

    Google Scholar 

  12. Campbell A, Murray P, Yakushina E et al (2018) New methods for automatic quantification of microstructural features using digital image processing. Mater Des 141:395–406

    CAS  Google Scholar 

  13. Smith T, Bonacuse P, Sosa J et al (2018) A quantifiable and automated volume fraction characterization technique for secondary and tertiary γ′ precipitates in Ni-based superalloys. Mater Charact 140:86–94

    CAS  Google Scholar 

  14. Collins PC, Welk B, Searles T et al (2009) Development of methods for the quantification of microstructural features in α + β-processed α/β titanium alloys. Mater Sci Eng A 508(1–2):174–182

    Google Scholar 

  15. Tiley J, Kim S, Parthasarathy T et al (2017) Quantifying the effect of microstructure variability on the yield strength predictions of Ni-base superalloys. Mater Sci Eng A 685:178–186

    CAS  Google Scholar 

  16. Peregrina-Barreto H, Terol-Villalobos I, Rangel-Magdaleno J et al (2013) Automatic grain size determination in microstructures using image processing. Measurement 46(1):249–258

    Google Scholar 

  17. Cecen A, Wargo E, Hanna A et al (2012) 3-D microstructure analysis of fuel cell materials: spatial distributions of tortuosity, void size and diffusivity. J Electrochem Soc 159(3):B299–B307

    CAS  Google Scholar 

  18. Payton E, Phillips P, Mills M (2010) Semi-automated characterization of the γ′ phase in Ni-based superalloys via high-resolution backscatter imaging. Mater Sci Eng A 527(10–11):2684–2692

    Google Scholar 

  19. Yang D, Liu Z (2016) Quantification of microstructural features and prediction of mechanical properties of a dual-phase Ti–6Al–4V alloy. Materials 9(8):628

    Google Scholar 

  20. Zhou W, Apkarian R, Wang ZL, Joy D (2006) Fundamentals of scanning electron microscopy (SEM). In: Zhou W, Wang ZL (eds) Scanning microscopy for nanotechnology. Springer, New York, pp 1–40

    Google Scholar 

  21. Behrooz A, Tseng J-C, Meganck J, Hopkinton M (2016) Image resolution in micro-CT: principles and characterization of the quantum FX and quantum GX system

  22. Ishitani T, Kamiya C, Sato M (2005) Influence of random noise on the contrast-to-gradient image resolution in scanning electron microscopy. J Electron Microsc 54(2):85–97

    Google Scholar 

  23. Barrett JF, Keat N (2004) Artifacts in CT: recognition and avoidance. Radiographics 24(6):1679–1691

    Google Scholar 

  24. Iskakov A, Yabansu YC, Rajagopalan S et al (2018) Application of spherical indentation and the materials knowledge system framework to establishing microstructure–yield strength linkages from carbon steel scoops excised from high-temperature exposed components. Acta Mater 144:758–767

    CAS  Google Scholar 

  25. Yabansu YC, Steinmetz P, Hötzer J et al (2017) Extraction of reduced-order process–structure linkages from phase-field simulations. Acta Mater 124:182–194

    CAS  Google Scholar 

  26. de Oca Zapiain DM, Popova E, Abdeljawad F et al (2018) Reduced-order microstructure-sensitive models for damage initiation in two-phase composites. Integr Mater Manuf Innov 7(3):97–115

    Google Scholar 

  27. Khosravani A, Cecen A, Kalidindi SR (2017) Development of high throughput assays for establishing process–structure–property linkages in multiphase polycrystalline metals: application to dual-phase steels. Acta Mater 123:55–69

    CAS  Google Scholar 

  28. Latypov MI, Kalidindi SR (2017) Data-driven reduced order models for effective yield strength and partitioning of strain in multiphase materials. J Comput Phys 346:242–261

    CAS  Google Scholar 

  29. Paulson NH, Priddy MW, McDowell DL, Kalidindi SR (2017) Reduced-order structure–property linkages for polycrystalline microstructures based on 2-point statistics. Acta Mater 129:428–438

    CAS  Google Scholar 

  30. Perez F, Granger BE, Hunter JD (2011) Python: an ecosystem for scientific computing. Comput Sci Eng 13(2):13–21

    Google Scholar 

  31. Higham DJ, Higham NJ (2016) MATLAB guide. Siam, Philadelphia

    Google Scholar 

  32. Sosa JM, Huber DE, Welk B, Fraser HL (2014) Development and application of MIPAR™: a novel software package for two- and three-dimensional microstructural characterization. Integr Mater Manuf Innov 3(1):123–140

    Google Scholar 

  33. Jackson M (2014) EM/MPM, Dayton: BlueQuartz software. http://www.bluequartz.net/projects/EIM_Segmentation/

  34. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66

    Google Scholar 

  35. Bradley D, Roth G (2007) Adaptive thresholding using the integral image. J Graph Tools 12(2):13–21

    Google Scholar 

  36. Davies ER (2012) Computer and machine vision: theory, algorithms, practicalities. Academic Press, Cambridge

    Google Scholar 

  37. Soille P (2013) Morphological image analysis: principles and applications. Springer, Berlin

    Google Scholar 

  38. Shapiro L (1992) Computer vision and image processing. Academic Press, Cambridge

    Google Scholar 

  39. Zuiderveld K (1994) Contrast limited adaptive histogram equalization. In: Heckbert PS (ed) Graphics gems IV. Academic Press, San Diego, pp 474–485

    Google Scholar 

  40. Deshpande S, Kulkarni A, Sampath S, Herman H (2004) Application of image analysis for characterization of porosity in thermal spray coatings and correlation with small angle neutron scattering. Surf Coat Technol 187(1):6–16

    CAS  Google Scholar 

  41. Surekha K, Murty B, Rao KP (2008) Microstructural characterization and corrosion behavior of multipass friction stir processed AA2219 aluminium alloy. Surf Coat Technol 202(17):4057–4068

    CAS  Google Scholar 

  42. Liu J, Li C, Liu J et al (2013) Study on 3D spatial distribution of steel fibers in fiber reinforced cementitious composites through micro-CT technique. Constr Build Mater 48:656–661

    Google Scholar 

  43. Saadatfar M, Garcia-Moreno F, Hutzler S et al (2009) Imaging of metallic foams using X-ray micro-CT. Colloids Surf A 344(1–3):107–112

    CAS  Google Scholar 

  44. Gigan S (2017) Optical microscopy aims deep. Nat Photonics 11(1):14–16

    CAS  Google Scholar 

  45. Butt H-J, Cappella B, Kappl M (2005) Force measurements with the atomic force microscope: technique, interpretation and applications. Surf Sci Rep 59(1–6):1–152

    CAS  Google Scholar 

  46. Wortmann T (2009) Fusion of AFM and SEM scans. In: 2009 International symposium on optomechatronic technologies. IEEE, Istanbul

  47. Kalidindi SR (2015) Hierarchical materials informatics: novel analytics for materials data. Elsevier, Amsterdam

    Google Scholar 

  48. Niezgoda SR, Turner DM, Fullwood DT, Kalidindi SR (2010) Optimized structure based representative volume element sets reflecting the ensemble-averaged 2-point statistics. Acta Mater 58(13):4432–4445

    CAS  Google Scholar 

  49. Chen C-Y, Klette R (1999) Image stitching—comparisons and new techniques. In: Solina F, Leonardis A (eds) International conference on computer analysis of images and patterns. Springer, Berlin, pp 615–622

    Google Scholar 

  50. Ma B, Zimmermann T, Rohde M et al (2007) Use of autostitch for automatic stitching of microscope images. Micron 38(5):492–499

    Google Scholar 

  51. Juntu J, Sijbers J, Van Dyck D, Gielen J (2005) Bias field correction for MRI images. In: Kurzyński M, Puchała E, Woźniak M, Żołnierek A (eds) Computer recognition systems. Springer, Berlin, pp 543–551

    Google Scholar 

  52. Likar B, Ja Maintz, Viergever MA, Pernus F (2000) Retrospective shading correction based on entropy minimization. J Microsc 197(Pt 3):285–295

    CAS  Google Scholar 

  53. Peters RA (1995) A new algorithm for image noise reduction using mathematical morphology. IEEE Trans Image Process 4(5):554–568

    CAS  Google Scholar 

  54. Sarode MV, Deshmukh PR (2011) Reduction of speckle noise and image enhancement of images using filtering technique. Int J Adv Technol 2(1):30–38

    Google Scholar 

  55. Van De Ville D, Nachtegael M, Van der Weken D et al (2003) Noise reduction by fuzzy image filtering. IEEE Trans Fuzzy Syst 11(4):429–436

    Google Scholar 

  56. Verma R, Ali J (2013) A comparative study of various types of image noise and efficient noise removal techniques. Int J Adv Res Comput Sci Softw Eng 3(10):617–622

    Google Scholar 

  57. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Proceedings of the 1998 IEEE international conference on computer vision, Bombay

  58. Lim JS (1990) Two-dimensional signal and image processing. Prentice Hall, Englewood Cliffs, pp 469–476

    Google Scholar 

  59. Bovik AC, Huang TS, Munson DC (1987) The effect of median filtering on edge estimation and detection. IEEE Trans Pattern Anal Mach Intell 2:181–194

    Google Scholar 

  60. Alkinani MH, El-Sakka MR (2017) Patch-based models and algorithms for image denoising: a comparative review between patch-based images denoising methods for additive noise reduction. EURASIP J Image Video Process 1:1–27

    Google Scholar 

  61. Buades A, Coll B, Morel J-M (2005) A non-local algorithm for image denoising. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), IEEE

  62. Chatterjee P, Milanfar P (2011) Patch-based near-optimal image denoising. IEEE Trans Image Process 21(4):1635–1649

    Google Scholar 

  63. Kervrann C, Boulanger J (2006) Optimal spatial adaptation for patch-based image denoising. IEEE Trans Image Process 15(10):2866–2878

    Google Scholar 

  64. Zhang L, Dong W, Zhang D, Shi G (2010) Two-stage image denoising by principal component analysis with local pixel grouping. Pattern Recogn 43(4):1531–1549

    Google Scholar 

  65. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095

    Google Scholar 

  66. Deledalle C-A, Denis L, Tupin F (2009) Iterative weighted maximum likelihood denoising with probabilistic patch-based weights. IEEE Trans Image Process 18(12):2661–2672

    Google Scholar 

  67. Van der Walt S, Schönberger JL, Nunez-Iglesias J et al (2014) scikit-image: image processing in Python. PeerJ 2:e453. https://doi.org/10.7717/peerj.453

    Article  Google Scholar 

  68. Jain AK (1989) Fundamentals of digital image processing. Prentice Hall, Englewood Cliffs

    Google Scholar 

  69. Gonzalez RC, Woods RE (2002) Digital image processing. Prentice Hall, Englewood Cliffs

    Google Scholar 

  70. Dutta S, Barat K, Das A et al (2014) Characterization of micrographs and fractographs of Cu-strengthened HSLA steel using image texture analysis. Measurement 47:130–144

    Google Scholar 

  71. Gupta S, Panda A, Naskar R et al (2017) Processing and refinement of steel microstructure images for assisting in computerized heat treatment of plain carbon steel. J Electron Imaging 26(6):063010. https://doi.org/10.1117/1.JEI.26.6.063010

    Article  Google Scholar 

  72. Papa JP, De Albuquerque VHC, Falcão AX, Tavares JMR (2010) Fast automatic microstructural segmentation of ferrous alloy samples using optimum-path forest. In: Barneva RP, Brimkov VE, Hauptman HA, Natal Jorge RM, Tavares JMRS (eds) International symposium computational modeling of objects represented in images. Springer, Berlin, pp 210–220

    Google Scholar 

  73. Moon KH, Falchetto AC, Jeong JH (2013) Microstructural analysis of asphalt mixtures using digital image processing techniques. Can J Civ Eng 41(1):74–86

    Google Scholar 

  74. Shafei B, Steidl G (2012) Segmentation of images with separating layers by fuzzy c-means and convex optimization. J Vis Commun Image Represent 23(4):611–621

    Google Scholar 

  75. Kapur JN, Sahoo PK, Wong AK (1985) A new method for gray-level picture thresholding using the entropy of the histogram. Comput Vis Graph Image Process 29(3):273–285

    Google Scholar 

  76. Kurita T, Otsu N, Abdelmalek N (1992) Maximum likelihood thresholding based on population mixture models. Pattern Recogn 25(10):1231–1240

    Google Scholar 

  77. Tsai D-M, Chen Y-H (1992) A fast histogram-clustering approach for multi-level thresholding. Pattern Recogn Lett 13(4):245–252

    Google Scholar 

  78. Parker JR (2010) Algorithms for image processing and computer vision. Wiley, New Jersey

    Google Scholar 

  79. Canny J (1987) A computational approach to edge detection. Readings in computer vision. Elsevier, Amsterdam, pp 184–203

    Google Scholar 

  80. Serra J (1982) Image analysis and math. Academic Press, Morphology

    Google Scholar 

  81. Han Y, Lai C, Wang B, Gu H (2019) Segmenting images with complex textures by using hybrid algorithm. J Electron Imaging 28(1):013030

    Google Scholar 

  82. Paulic M, Mocnik D, Ficko M et al (2015) Intelligent system for prediction of mechanical properties of material based on metallographic images. Tehnički Vjesnik 22(6):1419–1424

    Google Scholar 

  83. Testing ASf, Materials (2011) ASTM E562-11: standard test method for determining volume fraction by systematic manual point count, ASTM

  84. Klein T, Schachermayer M, Mendez-Martin F et al (2015) Carbon distribution in multi-phase γ-TiAl based alloys and its influence on mechanical properties and phase formation. Acta Mater 94:205–213

    CAS  Google Scholar 

  85. Potgieter J, Cortie M (1991) Determination of the microstructure and alloy element distribution in experimental duplex stainless steels. Mater Charact 26(3):155–165

    CAS  Google Scholar 

  86. Joseph C, Persson C, Colliander MH (2017) Influence of heat treatment on the microstructure and tensile properties of Ni-base superalloy Haynes 282. Mater Sci Eng A 679:520–530

    CAS  Google Scholar 

  87. Melenka G, Hunt A, van Ravenhorst J et al (2017) Manufacturing processes for braided composite materials. In: Carey JP (ed) Handbook of advances in braided composite materials. Elsevier, Amsterdam, pp 47–153

    Google Scholar 

  88. Standard A (2013) E112, standard test method for determining average grain size. ASTM International, West Conshohocken, 2010. https://doi.org/10.1520/E0112-10

  89. Standard A E2567-13a (2013) Standard test method for determining nodularity and nodule count in ductile iron. ASTM Internationals, West Conshohocken, 2013. https://doi.org/10.1520/E2567-13A

  90. Fullwood D, Kalidindi S, Niezgoda S et al (2008) Gradient-based microstructure reconstructions from distributions using fast Fourier transforms. Mater Sci Eng A 494(1–2):68–72

    Google Scholar 

  91. Niezgoda S, Fullwood D, Kalidindi S (2008) Delineation of the space of 2-point correlations in a composite material system. Acta Mater 56(18):5285–5292

    CAS  Google Scholar 

  92. Turner DM, Kalidindi SR (2016) Statistical construction of 3-D microstructures from 2-D exemplars collected on oblique sections. Acta Mater 102:136–148

    CAS  Google Scholar 

  93. Jiao Y, Stillinger F, Torquato S (2009) A superior descriptor of random textures and its predictive capacity. Proc Natl Acad Sci 106(42):17634–17639

    CAS  Google Scholar 

  94. Torquato S, Lu B (1993) Chord-length distribution function for two-phase random media. Phys Rev E 47(4):2950

    CAS  Google Scholar 

  95. Jiao Y, Stillinger F, Torquato S (2007) Modeling heterogeneous materials via two-point correlation functions: basic principles. Phys Rev E 76(3):031110

    CAS  Google Scholar 

  96. Berryman JG, Blair SC (1986) Use of digital image analysis to estimate fluid permeability of porous materials: application of two-point correlation functions. J Appl Phys 60(6):1930–1938

    CAS  Google Scholar 

  97. Kozar R, Suzuki A, Milligan W et al (2009) Strengthening mechanisms in polycrystalline multimodal nickel-base superalloys. Metal Mater Trans A 40(7):1588–1603

    Google Scholar 

  98. Francis E, Grant B, da Fonseca JQ et al (2014) High-temperature deformation mechanisms in a polycrystalline nickel-base superalloy studied by neutron diffraction and electron microscopy. Acta Mater 74:18–29

    CAS  Google Scholar 

  99. Unocic R, Kovarik L, Shen C et al (2008) Deformation mechanisms in Ni-base disk superalloys at higher temperatures. Superalloys 8:377

    Google Scholar 

Download references

Acknowledgements

The authors acknowledge support from ONR Grant N00014-18-1-2879. The authors acknowledge members of the MINED research group for contributing images for this study.

Author information

Authors and Affiliations

  1. George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Almambet Iskakov & Surya R. Kalidindi

  2. School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Surya R. Kalidindi

Authors
  1. Almambet Iskakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Surya R. Kalidindi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Surya R. Kalidindi.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Iskakov, A., Kalidindi, S.R. A Framework for the Systematic Design of Segmentation Workflows. Integr Mater Manuf Innov 9, 70–88 (2020). https://doi.org/10.1007/s40192-019-00166-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40192-019-00166-z

Keywords

Search

Navigation