Skip to main content
SpringerLink
Log in

Multiple instance learning combined with label invariant synthetic data for guiding systematic prostate biopsy: a feasibility study

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Ultrasound imaging is routinely used in prostate biopsy, which involves obtaining prostate tissue samples using a systematic, yet, non-targeted approach. This approach is blinded to individual patient intraprostatic pathology, and unfortunately, has a high rate of false negatives.

Methods

In this paper, we propose a deep network for improved detection of prostate cancer in systematic biopsy. We address several challenges associated with training such network: (1) Statistical labels: Since biopsy core’s pathology report only represents a statistical distribution of cancer within the core, we use multiple instance learning (MIL) networks to enable learning from ultrasound image regions associated with those data; (2) Limited labels: The number of biopsy cores are limited to at most 12 per patient. As a result, the number of samples available for training a deep network is limited. We alleviate this issue by effectively combining Independent Conditional Variational Auto Encoders (ICVAE) with MIL. We train ICVAE to learn label-invariant features of RF data, which is subsequently used to generate synthetic data for improved training of the MIL network.

Results

Our in vivo study includes data from 339 prostate biopsy cores of 70 patients. We achieve an area under the curve, sensitivity, specificity, and balanced accuracy of 0.68, 0.77, 0.55 and 0.66, respectively.

Conclusion

The proposed approach is generic and can be applied to several other scenarios where unlabeled data and noisy labels in training samples are present.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bjurlin MA, Taneja SS (2014) Standards for prostate biopsy. Curr Opin Urol 24(2):155–161

    Article  Google Scholar 

  2. Polascik TJ, Passoni NM, Villers A, Choyke PL (2014) Modernizing the diagnostic and decision-making pathway for prostate cancer. Clin Cancer Res 20(24):6254–6257

    Article  CAS  Google Scholar 

  3. Heidenreich A, Bastian PJ, Bellmunt J, Bolla M, Joniau S, van der Kwast T, Mason M, Matveev V, Wiegel T, Zattoni F, Mottet N, European Association of Urology (2014) EAU guidelines on prostate cancer. Part 1: screening, diagnosis, and local treatment with curative intent-update 2013. Eur Urol 65(1):124–137

  4. Eichler K, Hempel S, Wilby J, Myers L, Bachmann LM, Kleijnen J (2006) Diagnostic value of systematic biopsy methods in the investigation of prostate cancer: a systematic review. J Urol 175(5):1605–1612

    Article  Google Scholar 

  5. Hodge KK, McNeal JE, Stamey TA (1989) Ultrasound guided transrectal core biopsies of the palpably abnormal prostate. J Urol 142(1):66–70

    Article  CAS  Google Scholar 

  6. Rapiti E, Schaffar R, Iselin C, Miralbell R, Pelte MF, Weber D, Zanetti R, Neyroud-Caspar I, Bouchardy C (2013) Importance and determinants of gleason score undergrading on biopsy sample of prostate cancer in a population-based study. BMC Urol 13(1):13–19

    Article  Google Scholar 

  7. Siddiqui MM, Rais-Bahrami S, Turkbey B, George AK, Rothwax J, Shakir N, Okoro C, Raskolnikov D, Parnes HL, Linehan WM, Merino MJ, Simon RM, Choyke PL, Wood BJ, Pinto PA (2015) Comparison of MR/ultrasound fusion-guided biopsy with ultrasound-guided biopsy for the diagnosis of Prostate CancerMR/ultrasound fusion biopsy for Prostate CancerMR/ultrasound fusion biopsy for prostate Cancer. JAMA 313(4):390–397

    Article  CAS  Google Scholar 

  8. Natarajan S, Marks LS, Margolis DJA, Huang J, Macairan ML, Lieu P, Fenster A (2011) Clinical application of a 3D ultrasound-guided prostate biopsy system. Urol Oncol 29(3):334–342

    Article  Google Scholar 

  9. Ahmed HU, Bosaily AES, Brown LC, Gabe R, Kaplan R, Parmar MK, Collaco-Moraes Y, Ward K, Hindley RG, Freeman A, Kirkham AP, Oldroyd R, Parker C, Emberton M (2017) Diagnostic accuracy of multi-parametric mri and trus biopsy in prostate cancer PROMIS: a paired validating confirmatory study. The Lancet 389(10071):815–822

    Article  Google Scholar 

  10. Feleppa E, Porter C, Ketterling J, Dasgupta S, Ramachandran S, Sparks D (2007) Recent advances in ultrasonic tissue-type imaging of the prostate. In: Acoustical imaging, Springer, pp 331–339

  11. Correas JM, Tissier AM, Khairoune A, Khoury G, Eiss D, Hélénon O (2013) Ultrasound elastography of the prostate: state of the art. Diagn Interv Imaging 94(5):551–560

    Article  Google Scholar 

  12. Moradi M, Mousavi P, Abolmaesumi P (2007) Tissue characterization using fractal dimension of high frequency ultrasound rf time series. In: Medical image computing and computer-assisted intervention-MICCAI 2007, pp 900–908

  13. Imani F, Abolmaesumi P, Gibson E, Khojaste A, Gaed M, Moussa M, Gomez JA, Romagnoli C, Leveridge M, Chang S et al (2015) Computer-aided prostate cancer detection using ultrasound RF time series: in vivo feasibility study. IEEE Trans Med Imaging 34(11):2248–2257

    Article  Google Scholar 

  14. Imani F, Zhuang B, Tahmasebi A, Kwak JT, Xu S, Agarwal H, Bharat S, Uniyal N, Turkbey IB, Choyke P et al (2015) Augmenting MRI-transrectal ultrasound-guided prostate biopsy with temporal ultrasound data: a clinical feasibility study. Int J Comput Assist Radiol Surg 10(6):727–735

    Article  Google Scholar 

  15. Nahlawi L, Imani F, Gaed M, Gomez JA, Moussa M, Gibson E, Fenster A, Ward AD, Abolmaesumi P, Mousavi P, Others (2015) Using hidden markov models to capture temporal aspects of ultrasound data in prostate cancer. In: 2015 IEEE international conference on bioinformatics and biomedicine (BIBM), pp 446–449

  16. Moradi M, Abolmaesumi P, Mousavi P (2010) Tissue typing using ultrasound RF time series: experiments with animal tissue samples. Med Phys 37(8):4401–4413

    Article  Google Scholar 

  17. Azizi S, Imani F, Zhuang B, Tahmasebi A, Kwak JT, Xu S, Uniyal N, Turkbey B, Choyke P, Pinto P, Others (2015) Ultrasound-based detection of prostate cancer using automatic feature selection with deep belief networks. In: International conference on medical image computing and computer-assisted intervention, pp 70–77

  18. Bayat S, Azizi S, Daoud MI, Nir G, Imani F, Gerardo CD, Yan P, Tahmasebi A, Vignon F, Sojoudi S, Wilson S, Iczkowski KA, Lucia MS, Goldenberg L, Salcudean SE, Abolmaesumi P, Mousavi P (2018) Investigation of physical phenomena underlying temporal-enhanced ultrasound as a new diagnostic imaging technique: theory and simulations. IEEE Trans Ultrason Ferroelectr Freq Control 65(3):400–410

    Article  Google Scholar 

  19. Bayat S, Imani F, Gerardo CD, Nir G, Azizi S, Yan P, Tahmasebi A, Wilson S, Iczkowski KA, Lucia MS, Goldenberg L, Salcudean SE, Mousavi P, Abolmaesumi P (2017) Tissue mimicking simulations for temporal enhanced ultrasound-based tissue typing. In: Medical imaging 2017: ultrasonic imaging and tomography, international society for optics and photonics, vol 10139, pp 79–84

  20. Sedghi A, Pesteie M, Javadi G, Azizi S, Yan P, Kwak JT, Xu S, Turkbey B, Choyke P, Pinto P, Wood B, Rohling R, Abolmaesumi P, Mousavi P (2019) Deep neural maps for unsupervised visualization of high-grade cancer in prostate biopsies. Int J Comput Assist Radiol Surgery 14:1009–1016

    Article  Google Scholar 

  21. Azizi S, Bayat S, Yan P, Tahmasebi A, Kwak JT, Xu S, Turkbey B, Choyke P, Pinto P, Wood B, Mousavi P, Abolmaesumi P (2018) Deep recurrent neural networks for prostate cancer detection: analysis of temporal enhanced ultrasound. IEEE Trans Med Imaging 37(12):2695–2703

    Article  Google Scholar 

  22. Ing N, Tomczak JM, Miller E, Garraway IP, Welling M, Knudsen BS, Gertych A (2018) A deep multiple instance model to predict prostate cancer metastasis from nuclear morphology

  23. Khan SS, Madden MG (2004) A review of Multi-Instance learning assumptions. Knowl Eng Rev 00(January):1–24

  24. Ilse M, Tomczak JM, Welling M (2018) Attention-based deep multiple instance learning. arXiv preprint arXiv:1802.04712

  25. Weidmann N, Frank E, Pfahringer B (2003) A Two-Level learning method for generalized multi-instance problems. In: ECML

  26. Dietterich TG, Lathrop RH, Lozano-Pérez T (1997) Solving the multiple instance problem with axis-parallel rectangles. Artif Intell 89:31–71

    Article  Google Scholar 

  27. Cireşan DC, Meier U, Gambardella LM, Schmidhuber J (2010) Deep, big, simple neural nets for handwritten digit recognition. Neural Comput 22(12):3207–3220

    Article  Google Scholar 

  28. Tran T, Pham T, Carneiro G, Palmer L, Reid I (2017) A bayesian data augmentation approach for learning deep models. In: Advances in neural information processing systems, pp 2794–2803

  29. Ding J, Li X, Gudivada VN (2017) Augmentation and evaluation of training data for deep learning. In: 2017 IEEE international conference on big data (big data), pp 2603–2611. https://doi.org/10.1109/bigdata.2017.8258220

  30. Wong SC, Gatt A, Stamatescu V, McDonnell MD (2016) Understanding data augmentation for classification: When to warp? In: 2016 International conference on digital image computing: techniques and applications (DICTA), pp 1–6, https://doi.org/10.1109/dicta.2016.7797091

  31. DeVries T, Taylor GW (2017) Dataset augmentation in feature space. arXiv preprint arXiv:1702.05538

  32. Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114

  33. Nishizaki H (2017) Data augmentation and feature extraction using variational autoencoder for acoustic modeling. In: Asia-Pacific signal and information processing association annual summit and conference (APSIPA ASC), IEEE, pp 1222–1227

  34. Wan Z, Zhang Y, He H (2017) Variational autoencoder based synthetic data generation for imbalanced learning. In: 2017 IEEE symposium series on computational intelligence (SSCI), pp 1–7. https://doi.org/10.1109/ssci.2017.8285168

  35. Pesteie M, Abolmaesumi P, Rohling RN (2019) Adaptive augmentation of medical data using independently conditional variational auto-encoders.In: IEEE Transactions on Medical Imaging

  36. Wang X, Yan Y, Tang P, Bai X, Liu W (2018) Revisiting multiple instance neural networks. Pattern Recogn 74:15–24

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of British Columbia, Vancouver, BC, Canada

    Golara Javadi, Samareh Samadi, Sharareh Bayat, Mehran Pesteie, Mohammad H. Jafari, Samira Sojoudi & Purang Abolmaesumi

  2. Queen’s University, Kingston, ON, Canada

    Parvin Mousavi

  3. Vancouver General Hospital, Vancouver, BC, Canada

    Claudia Kesch, Antonio Hurtado, Silvia Chang & Peter Black

Authors
  1. Golara Javadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samareh Samadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sharareh Bayat
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mehran Pesteie
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohammad H. Jafari
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Samira Sojoudi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Claudia Kesch
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Antonio Hurtado
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Silvia Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Parvin Mousavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Peter Black
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Purang Abolmaesumi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Golara Javadi.

Ethics declarations

Conflict of interest

All authors confirm that there are no known conflicts of interest with this publication.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is funded in part by the Canadian Institutes of Health Research (CIHR) and in part by the Natural Sciences and Engineering Research Council of Canada (NSERC)

Golara Javadi and Samareh Samadi are Joint first authors. Parvin Mousavi, Peter Black and Purang Abolmaesumi are Joint senior authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Javadi, G., Samadi, S., Bayat, S. et al. Multiple instance learning combined with label invariant synthetic data for guiding systematic prostate biopsy: a feasibility study. Int J CARS 15, 1023–1031 (2020). https://doi.org/10.1007/s11548-020-02168-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-020-02168-1

Keywords

Search

Navigation