Abstract
Fossil single-celled marine organisms known as foraminifera are widely used in oceanographic research. The identification of species is one of the most common tasks when analyzing ocean samples. One of the primary criteria for species identification is their morphology. Automatic segmentation of images of foraminifera would aid on the identification task as well as on other morphological studies. We pose this problem as an edge detection task for which capturing the correct topological structure is essential. Due to the presence of soft edges and even unclosed segments, state-of-the-art techniques have problems capturing the correct edge structure. Standard pixel-based loss functions are also sensitive to small deformations and shifts of the edges penalizing location more heavily than actual structure. Hence, we propose a homology-based detector of local structural difference between two edge maps with a tolerable deformation. This detector is employed as a new criterion for the training and design of data-driven approaches that focus on enhancing these structural differences. Our approaches demonstrate significant improvement on morphological segmentation of foraminifera when considering region-based and topology-based metrics. Human ranking of the quality of the results by marine researchers also supports these findings.
Similar content being viewed by others
References
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., et al. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. https://mwww.tensorflow.org. Accessed 27 Oct 2020.
Ahmed M., Fasy B. T., & Wenk C. (2014). Local persistent homology based distance between maps. In Proceedings of the 22nd ACM SIGSPATIAL international conference on advances in geographic information systems, SIGSPATIAL ’14 (pp. 43–52). New York, NY: ACM.
Arbelaez, P., Maire, M., Fowlkes, C., & Malik, J. (2011). Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5), 898–916.
AROS Lab, NCSU. (2020). NCSU-CUB ForaBot Project. https://research.ece.ncsu.edu/aros/foram-identification/.
Bansal, A., Chen, X., Russell, B. C., Gupta, A., & Ramanan, D. (2017). Pixelnet: Representation of the pixels, by the pixels, and for the pixels. CoRR, arXiv: 1702.06506.
Beksi, W. J., & Papanikolopoulos, N. (2016). 3D region segmentation using topological persistence. In 2016 IEEE/RSJ International conference on intelligent robots and systems (IROS) (pp. 1079–1084).
BenTaieb, A., & Hamarneh, G. (2016). Aware fully convolutional networks for histology gland segmentation. In International conference on medical image computing and computer-assisted intervention (pp. 460–468). Berlin: Springer.
Berggren, W. A. (1972). A cenozoic time-scale-some implications for regional geology and paleobiogeography. Lethaia, 5(2), 195–215.
Berggren, W. A. (1992). Ecology and palaeoecology of benthic foraminifera. The Journal of Protozoology, 39(4), 537.
Bertasius, G., Shi, J., & Torresani, L. (2015). Deepedge: A multi-scale bifurcated deep network for top-down contour detection. In 2015 IEEE Conference on computer vision and pattern recognition (CVPR) (pp. 4380–4389). IEEE.
Boltovskoy, E., Scott, D. B., & Medioli, F. (1991). Morphological variations of benthic foraminiferal tests in response to changes in ecological parameters: A review. Journal of Paleontology, 65(02), 175–185.
Bubenik, P. (2015). Statistical topological data analysis using persistence landscapes. The Journal of Machine Learning Research, 16(1), 77–102.
Chaurasia, A., & Culurciello, E. (2017). Linknet: Exploiting encoder representations for efficient semantic segmentation. In 2017 IEEE Visual communications and image processing (VCIP) (pp. 1–4). IEEE.
Chen, L., Barron, J. T., Papandreou, G., Murphy, K., & Yuille, A. L. (2015). Semantic image segmentation with task-specific edge detection using cnns and a discriminatively trained domain transform. CoRR, arXiv: 1511.03328.
Choi, C., & Christensen, H. I. (2012). 3D textureless object detection and tracking: An edge-based approach. In 2012 IEEE/RSJ International conference on intelligent robots and systems (pp. 3877–3884).
Clough, J. R., Öksüz, I., Byrne, N., Schnabel, J. A., & King, A. P. (2019). Explicit topological priors for deep-learning based image segmentation using persistent homology. CoRR, arXiv: 1901.10244.
Corliss, B. H. (1991). Morphology and microhabitat preferences of benthic foraminifera from the northwest Atlantic Ocean. Marine Micropaleontology, 17(3–4), 195–236.
Deng, J., Dong, W., Socher, R., Li, L., Kai, L., & Li F.-F. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on computer vision and pattern recognition (pp. 248–255).
Dollár, P., Tu, Z., & Belongie, S. (2006). Supervised learning of edges and object boundaries. In 2006 IEEE Computer Society conference on computer vision and pattern recognition (CVPR’06) (Vol. 2, pp. 1964–1971).
Dollár, P., & Zitnick, C. L. (2015). Fast edge detection using structured forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(8), 1558–1570.
Edelsbrunner, H., Letscher, D., & Zomorodian, A. (2000). Topological persistence and simplification. In 41st Annual symposium on foundations of computer science, 2000. Proceedings (pp. 454–463). IEEE.
Ferrari, V., Fevrier, L., Jurie, F., & Schmid, C. (2008). Groups of adjacent contour segments for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(1), 36–51.
Ge, Q., Zhong, B., Kanakiya, B., Mitra, R., Marchitto, T., & Lobaton, E. (2017). Coarse-to-fine foraminifera image segmentation through 3D and deep features. In 2017 IEEE Symposium series on computational intelligence (SSCI) (pp. 1–8). IEEE.
Girshick, R. B. (2015). Fast R-CNN. CoRR, arXiv: 1504.08083.
Hu, X., Li, F., Samaras, D., & Chen, C. (2019). Topology-preserving deep image segmentation. CoRR, arXiv: 1906.05404.
Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T., & Keutzer, K. (2014). Densenet: Implementing efficient convnet descriptor pyramids. arXiv preprint arXiv:1404.1869.
Kennett, J., & Srinivasan, M. (1983). Neogene planktonic foraminifera: A phylogenetic atlas. Stroudsburg: Hutchinson Ross.
Kingma , D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Kingma , D. P., & Welling, M. (2014). Auto-encoding variational bayes. arXiv:1312.6114.
Kirillov, A., Levinkov, E., Andres, B., Savchynskyy, B., & Rother, C. (2016). Instancecut: From edges to instances with multicut. CoRR, arXiv:1611.08272.
Letscher, D., & Fritts, J. (2007). Image segmentation using topological persistence. In W. G. Kropatsch, M. Kampel, & A. Hanbury (Eds.), Computer analysis of images and patterns (pp. 587–595). Berlin: Springer.
Lim, J. J., Zitnick, C. L., & Dollar, P. (2013). Sketch tokens: A learned mid-level representation for contour and object detection. In The IEEE conference on computer vision and pattern recognition (CVPR).
Lin, T., Goyal, P., Girshick, R. B., He, K., & Dollár, P. (2017). Focal loss for dense object detection. CoRR, arXiv:1708.02002.
Liu, Y., Cheng, M., Hu, X., Bian, J., Zhang, L., Bai, X., et al. (2019). Richer convolutional features for edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8), 1939–1946.
Liu, Y., & Lew, M. S. (2016). Learning relaxed deep supervision for better edge detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 231–240).
Lobaton, E., Vasudevan, R., Bajcsy, R., & Alterovitz, R. (2010). Local occlusion detection under deformations using topological invariants. In European conference on computer vision (ECCV).
Lobaton, E., Vasudevan, R., Bajcsy, R., & Alterovitz, R. (2011). Robust topological features for deformation invariant image matching. In International conference on computer vision (ICCV).
Mnih, V., Heess, N., Graves, A., & Kavukcuoglu, K. (2014). Recurrent models of visual attention. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence & K. Q. Weinberge (Eds.), Advances in Neural Information Processing Systems 27 (pp. 2204–2212). http://papers.nips.cc/paper/5542-recurrent-models-of-visual-attention.pdf.
Mosinska, A., Márquez-Neila, P., Kozinski, M., & Fua, P. (2018). Beyond the pixel-wise loss for topology-aware delineation. In The IEEE conference on computer vision and pattern recognition (CVPR).
Qu, G., Zhang, W., Wang, Z., Dai, X., Shi, J., He, J., et al. (2018). Stripnet: Towards topology consistent strip structure segmentation. In 2018 ACM Multimedia conference on multimedia conference (pp. 283–291). ACM.
Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement.
Rohling, E. J., & Cooke, S. (1999). Stable oxygen and carbon isotopes in foraminiferal carbonate shells. In Modern foraminifera (pp. 239–258).
Rojas-Moraleda, R., Xiong, W., Halama, N., Breitkopf-Heinlein, K., Dooley, S., Salinas, L., et al. (2017). Robust detection and segmentation of cell nuclei in biomedical images based on a computational topology framework. Medical Image Analysis, 38, 90–103.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International conference on medical image computing and computer-assisted intervention (pp. 234–241). Berlin: Springer.
Shrivastava, A., Gupta, A., & Girshick, R. (2016). Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 761–769).
Sohn, K., Lee, H., & Yan, X. (2015). Learning structured output representation using deep conditional generative models. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in neural information processing systems (Vol. 28, pp. 3483–3491). Boston: Curran Associates Inc.
Tipsword, H. L. (1962). Tertiary foraminifera in Gulf Coast petroleum exploration and development. In Geology of the Gulf Coast and Central Texas, and guidebook of excursions (pp. 16–57). http://archives.datapages.com/data/hgssp/data/013/013001/i_hgs013i.htm.
Walker, J., Doersch, C., Gupta, A., & Hebert, M. (2016). An uncertain future: Forecasting from static images using variational autoencoders. CoRR, arXiv:1606.07873.
Wegner, J. D., Montoya-Zegarra, J. A., & Schindler, K. (2013). A higher-order CRF model for road network extraction. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1698–1705).
Wiedemann, C., Heipke, C., Mayer, H., & Jamet, O. (1998). Empirical evaluation of automatically extracted road axes. In Empirical evaluation techniques in computer vision (pp. 172–187).
Xie, S., & Tu, Z. (2017). Holistically-nested edge detection. International Journal of Computer Vision, 125(1–3), 3–18.
Xu, D., Ouyang, W., Alameda-Pineda, X., Ricci, E., Wang, X., & Sebe, N. (2017). Learning deep structured multi-scale features using attention-gated CRFs for contour prediction. In I. Guyon , U. V. Luxburg , S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan & R. Garnett (Eds.), Advances in neural information processing systems (pp. 3961–3970). http://papers.nips.cc/paper/6985-learning-deep-structured-multi-scale-features-using-attention-gated-crfs-for-contour-prediction.pdf.
Zhong, B., Ge, Q., Kanakiya, B., Marchitto, R. M. T., & Lobaton, E. (2017). A comparative study of image classification algorithms for foraminifera identification. In 2017 IEEE Symposium series on computational intelligence (SSCI) (pp. 1–8). IEEE.
Zhou, L., Zhang, C., & Wu, M. (2018). D-linknet: Linknet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. In The IEEE conference on computer vision and pattern recognition (CVPR) workshops.
Zhu, G., Porikli, F., & Li, H. (2015). Tracking randomly moving objects on edge box proposals. CoRR, arXiv:1507.08085.
Zhu, Z., Xia, Y., Shen, W., Fishman, E., & Yuille, A. (2018). A 3D coarse-to-fine framework for volumetric medical image segmentation. In 2018 International conference on 3D vision (3DV) (pp. 682–690). IEEE.
Zitnick, L., & Dollár, P. (2014). Edge boxes: Locating object proposals from edges. In ECCV.
Zomorodian, A., & Carlsson, G. (2005). Computing persistent homology. Discrete & Computational Geometry, 33(2), 249–274.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This is one of the several papers published in Autonomous Robots comprising the Special Issue on Topological Methods in Robotics.
This work was supported by US National Science Foundation Grants OCE-1637023, OCE-1637039, OCE-1829970 and OCE-1829930.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Ge, Q., Richmond, T., Zhong, B. et al. Enhancing the morphological segmentation of microscopic fossils through Localized Topology-Aware Edge Detection. Auton Robot 45, 709–723 (2021). https://doi.org/10.1007/s10514-020-09950-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-020-09950-9