Abstract
It is a known fact that infrared radiation is produced by all objects with a temperature above absolute zero. In fruits, IR light sensor senses the invisible areas and exposes obscure objects in the image. In normal RGB images, it is very difficult to predict the internal defect of the fruit accurately without chopping it into pieces. In this paper, a thermal IR imaging-based fruit defect detection technique is proposed to identify and estimate the internal defect in the pome fruits. The technique is non-invasive and non-destructive which helps in minimizing the fruit wastage during the quality check. To achieve high accuracy, a three-level validation process is adopted. First level involves the prediction made by the proposed Deep learning-based expert system using RGB and thermal images of apple respectively. In second level, the validation of results is done using Fourier Transform Infrared spectroscopy technique. And finally, in third level, an invasive destructive method is used for inspection of the fruit quality by cutting them into pieces. The apple defect detection accuracy by the proposed Naïve Bayes classifier is observed to be 97.6% for thermal IR imaging samples whereas true color based achieved only 59% for the same sample. An apple seems to be healthy externally but internally there is a probability of a defect. Thermal IR imaging detects the heat from the surface of the fruit. Due to defective tissues of fruit pulp, the non-uniform temperature difference is observed and sensed on the surface of the fruit.
Similar content being viewed by others
References
Thangavelns, K., Dhivya, K.: Determination of curcumin, starch and moisture content in turmeric by Fourier transform near infrared spectroscopy (FT-NIR). Eng. Agric. Environ. Food 12, 264–269 (2019)
Mohd, M.R.S., Herman, S.H., Shariff, Z.: Application of discrete wavelet transform in thermal infrared image processing. In: 2016 IEEE Conference on Systems, Process and Control (ICSPC), Bandar Hilir, 2016, pp. 186–191. https://doi.org/10.1109/SPC.2016.7920727.
Mulaveesala, R., Ghali, V.S., Arora, V.: Applications of non-stationary thermal wave imaging methods for characterisation of fibre-reinforced plastic materials. Electron. Lett. 49(2), 118–119 (2013)
Liu, Q., Hong, X., Zou, B., Chen, J., Chen, Z., Zhao, G.: Hierarchical contour closure-based holistic salient object detection. IEEE Trans. Image Process. 26(9), 4537–4552 (2017). https://doi.org/10.1109/TIP.2017.2703081
Meng, F., Li, H., Wu, Q., Luo, B., Ngan, K.N.: Weakly supervised part proposal segmentation from multiple images. IEEE Trans. Image Process. 26(8), 4019–4031 (2017). https://doi.org/10.1109/TIP.2017.2708839
Raghavendra, A., Rao, M.: A survey on internal defect detection in fruits by non-intrusive methods. Int. J. Latest Trends Eng. Technol. 6(3), 343–348 (2016)
Wang, F., Pan, X.: Image segmentation for somatic cell of milk based on niching particle swarm optimization Otsu. Eng. Agric. Environ. Food 12, 141–149 (2019)
Leena, N., Saju, K.K.: Classification of macronutrient deficiencies in maize plants using optimized multi class support vector machines. Eng. Agric. Environ. Food 12, 126–139 (2019)
Serrato, K.L.R., Estrada, J.A.S., Gonzalez, M.T.R.: Automatic pest detection on bean and potato crops by applying neural classifiers. Eng. Agric. Environ. Food 11, 245–255 (2018)
Sa, J., Sun, X., Zhang, T., Li, H., Zeng, H.: Improved Otsu segmentation based on Sobel operator. In: 2016 3rd Intl. Conf. on Systems and Informatics (ICSAI), Shanghai, pp. 886–890 (2016).
Khan, M.B., Nisar, H., Aun, N.C., Lo, P.K.: Iterative region-based Otsu thresholding of bright-field microscopic images of activated sludge. In: 2016 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES), Kuala Lumpur, pp. 533–538 (2016). https://doi.org/10.1109/IECBES.2016.7843507.
Wang, Q., Hu, S., Wang, J., Ren, K.: Secure surfing: privacy-preserving speeded-up robust feature extractor. In: 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS), Nara, pp. 700–710 (2016). https://doi.org/10.1109/ICDCS.2016.84.
Abedin, M.Z., Dhar, P., Deb, K.: Traffic sign recognition using SURF: speeded up robust feature descriptor and artificial neural network classifier. In: 2016 9th International Conference on Electrical and Computer Engineering (ICECE), Dhaka, pp. 198–201 (2016). https://doi.org/10.1109/ICECE.2016.7853890.
Verma, N.K., Sharma, T., Sevakula, R.K., Salour, A.: Vision based object counting using speeded up Robust features for inventory control. In: 2016 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, pp. 709–714 (2016). https://doi.org/10.1109/CSCI.2016.0139.
Melendez, J., Girones, X., Puig, D.: Supervised texture segmentation through a multi-level pixel-based classifier based on specifically designed filters. In: 2011 18th IEEE International Conference on Image Processing, Brussels, pp. 2869–2872 (2011). https://doi.org/10.1109/ICIP.2011.6116147.
Zhang, K., Xu, Z., Dong, S., Con, C., Wu, Q.: Identification of peach leaf disease infected by Xanthomonas campestris with deep learning. Eng. Agric. Environ. Food 12(4), 388–396 (2019)
Wdowiak, M., Slodkowska, J., Markiewicz, T.: Cell segmentation in desmoglein-3 stained specimen microscopic images using GVF and watershed algorithm. In: 2016 17th International Conference Computational Problems of Electrical Engineering (CPEE), Sandomierz, pp. 1-3 (2016). https://doi.org/10.1109/CPEE.2016.7738760
Mohana Rao, K.N.R., Dempster, A.G.: Modification on distance transform to avoid over-segmentation and under-segmentation. In: Intl. Symposium on VIPromCom Video/Image Processing and Multimedia Communications, Zadar, Croatia, pp. 295–301 (2002). https://doi.org/10.1109/VIPROM.2002.1026672.
Duarte, A., Carrão, L., Espanha, M., Viana, T., Freitas, D., Bártolo, P., Faria, P., Almeida, H.A.: Segmentation algorithms for thermal images. Procedia Technol. 16, 1560–1569 (2014)
Font-Aragones, X., Faundez-Zanuy, M., Mekyska, J.: Thermal hand image segmentation for biometric recognition. IEEE Aerosp. Electron. Syst. Mag. 28(6), 4–14 (2013). https://doi.org/10.1109/MAES.2013.6533739
Chen, Y., Chen, W., Ni, H.: Image segmentation in thermal images. In: 2016 IEEE International Conference on Industrial Technology (ICIT), Taipei, pp. 1507–1512 (2016). https://doi.org/10.1109/ICIT.2016.7474983.
Shaikh,S., Gite, H., Manza, R.R., Kale, K.V., Akhter, N.: Segmentation of thermal images using thresholding-based methods for detection of malignant tumours. In: Intl. Symposium on Intelligent Systems Technologies and Applications, Advances in Intelligent Systems and Computing (AISC), vol. 530, pp. 131–146 (2016).
Jaffery, Z.A., Dubey, A.K.: Design of early fault detection technique for electrical assets using infrared thermograms. Int. J. Electric. Power Energy Syst. 63, 753–759 (2014)
Jaffery, Z.A., Dubey, A.K., Irshad, Haque, A.: Scheme for predictive fault diagnosis in photo-voltaic modules using thermal imaging. Infrared Phys. Technol. 83, 182–187 (2017)
Spahn, J.G., Nuguru, K.: Grayscale thermographic imaging. US patent, US 20160213304 A1 (2016).
Usamentiaga, R., Venegas, P., Guerediaga, J., Vega, L., Molleda, J., Bulnes, F.G.: Infrared thermography for temperature measurement and non-destructive testing. Sensors 14(7), 12305–12348 (2014)
Santhi, V.: Recent advances in applied thermal imaging for industrial applications. IGI Global (2017). https://doi.org/10.4018/978-1-5225-2423-6
http://www.giangrandi.ch/optics/blackbody/blackbody.shtml. Accessed 19 Aug 2017
http://www.physicscentral.com/explore/action/infraredlight.cfm. Accessed 19 Aug 2017
https://physics.stackexchange.com/questions/116302/how-is-temperature-related-to-color. Accessed 19 Aug 2017
http://www.iar.unicamp.br/lab/luz/ld/Cor/color_vision.pdf. Accessed 19 Aug 2017
Jeroen, et al.: Light penetration properties of NIR radiation in fruit with respect to non-destructive quality assessment. Postharv. Biol. Technol. 18(2), 121–132 (2000)
Davidhazy, A.: Infrared photography. https://people.rit.edu/andpph/text-infraredbasics.html. Accessed 19 Aug 2017
https://www.simac.com/en/type-oplossingen/thermal-vision-inspection. Accessed 19 Aug 2017
http://www6.dict.cc/wp_examples.php?lp_id=1&lang=en&s=human%20eye. Accessed 19 Aug 2017
Prabhakar, C.J., Mohana, S.H.: Computer vision based technique for surface defect detection of apples. In: Srivastava, R., Singh, S.K., Shukla, K.K. (eds.) Research Developments in Computer Vision and Image Processing: Methodologies and Applications, pp. 111–121. Idea Group, New York (2014)
Using Regions of Interest as Masks. http://www.xinapse.com/Manual/roi_mask.html. Accessed 19 Aug 2017
Kim, H.S.: FPGA-based of thermogram enhancement algorithm for non-destructive thermal characterization. IJE Trans. A Basics 31(10), 1675–1681 (2018)
Shavandi, M., Afrakoti, I.E.P.: Face recognition in thermal images based on sparse classifier. IJE Trans. A Basics 32(1), 78–84 (2019)
Mysteries of Neural Networks. https://towardsdatascience.com/gentle-dive-into-math-behind-convolutional-neural-networks-79a07dd44cf9. Accessed 27 Jan 2020.
Hahn, F.: Actual pathogen detection: sensors and algorithms—a review. Algorithms 2, 301–338 (2009)
Pan, J., et al.: Image segmentation based on 2D OTSU and simplified swarm optimization. In: 2016 International Conference on Machine Learning and Cybernetics (ICMLC), Jeju, pp. 1026–1030 (2016). https://doi.org/10.1109/ICMLC.2016.7873020.
Stefan, J.: Über die Beziehung zwischen der Wärmestrahlung und der Temperatur [On the relationship between heat radiation and temperature] (PDF), Sitzungsberichte der mathematisch-naturwissenschaftlichen Classe der kaiserlichen Akademie der Wissenschaften (in German). Vienna 79, 391–428 (1879)
Hassanpour, H., Yousefian, H.: An improved pixon-based approach for image segmentation. IJE Trans. A Basics 24(1), 25–35 (2011)
Yogesh, Dubey, A.K., Ratan, R., Rocha, R.: Computer vision based analysis and detection of defects in fruits causes due to nutrients deficiency. Cluster Comput. (2019). https://doi.org/10.1007/s10586-019-03029-6
Acknowledgements
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors are gratefully acknowledging for the laboratory support provided by Amity Institute of Nano Technology, Amity University Uttar Pradesh, Noida, UP, India to carry out the FTIR analysis of the sample fruits. The equipment used was funded by ICAR through NASF for funded project NASF/Nano-5020/2016-17. The authors are gratefully acknowledging to Prof. Z. A. Jaffery, Head of Electrical Engineering Department, Jamia Millia Islamia, New Delhi, India for providing the thermal imaging camera to take the thermal images of the fruit samples.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yogesh, Dubey, A.K., Arora, R.R. et al. Fruit Defect Prediction Model (FDPM) based on Three-Level Validation. J Nondestruct Eval 40, 45 (2021). https://doi.org/10.1007/s10921-021-00778-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10921-021-00778-6