Skip to main content

Advertisement

SpringerLink
Log in

DroneRTEF:development of a novel adaptive framework for railroad track extraction in drone images

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Railroad track health monitoring is a challenging yet important task as it affects the safety of railroad systems. Railroad track extraction presents an immediate advantage during railroad inspections in an efficient and cost-effective manner. At present, human inspectors, inspection trains, and rail-mounted vehicles equipped with cameras are prevalent image acquisition systems(IAS) in the track extraction module. However, these IAS face various challenges such as high operability cost, railroad closed for normal traffic, and inaccessibility to certain geographical locations. In such scenarios, drones act as effective IAS. Therefore, this paper presents a novel and adaptive railroad track extraction framework for drone images (DI) captured under uneven illumination, at different drone flight heights, with varying rail line orientations, and in complex railroad environments. We termed this framework as DroneRTEF. This work primarily focuses on two aspects of drone-based railroad track images: image enhancement and image analysis. With regard to the first aspect, a global image enhancement algorithm named adaptive colour space-based masking (ACSM) is developed to enhance railroad track images and identify rail lines. The rail lines and background can be highlighted and homogenized, respectively, in DI captured under various sunlight intensity using ACSM due to its illuminance independence. With regard to the second aspect, the Hough parameter space analysis-based novel Hough transform-ground sample distance(HT-GSD) method is presented in this paper. The proposed HT-GSD method emphasizes on rail line detections at varying line orientations and different flight heights. The track extraction is then performed by a coordinate transformation technique. The approach has been successfully tested and validated on various DI. The efficacy of our framework for rail line detection is identified by comparing it with other line detection model. Performances of these methods are tested using metrics such as precision, recall and accuracies of the detections. Results obtained show that our method is superior to another model. Therefore, DroneRTEF is an efficient and feasible method for railroad track extraction in DI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Min Y, Xiao B, Dang J, Yue B, Cheng T (2018) Real time detection system for rail surface defects based on machine vision. EURASIP J Image Video Process 2018(1):3.

    Article  Google Scholar 

  2. Kaleli F, Akgul YS (2009) Vision-based railroad track extraction using dynamic programming. In: 2009 12th International IEEE Conference on Intelligent Transportation Systems, (IEEE, 2009), pp 1–6

  3. Kim Z, Cohn TE (2004) Pseudoreal-time activity detection for railroad grade-crossing safety. IEEE Trans Intell Transportation Syst 5(4):319–324.

    Article  Google Scholar 

  4. Feng H, Jiang Z, Xie F, Yang P, Shi J, Chen L (2013) Automatic fastener classification and defect detection in vision-based railway inspection systems. IEEE Trans Instrumentation Meas 63(4):877–888.

    Article  Google Scholar 

  5. Espino JC, Stanciulescu B (2012) Rail extraction technique using gradient information and a priori shape model. In: 2012 15th International IEEE Conference on Intelligent Transportation Systems, (IEEE, 2012), pp. 1132–1136

  6. Espino JC, Stanciulescu B, Forin P (2013) Rail and turnout detection using gradient information and template matching. In: Intelligent Rail Transportation (ICIRT), 2013 IEEE International Conference on (IEEE, 2013), pp 233–238

  7. Ashish, J., Wang, J., Yang, X., et al.: ‘TrackNet-A Deep Learning Based Fault Detection for Railway Track Inspection’2018.

  8. Qi Z, Tian Y, Shi Y (2013) Efficient railway tracks detection and turnouts recognition method using HOG features. Neural Comput Appl 23(1):245–254.

    Article  Google Scholar 

  9. Li Q, Ren S (2012) A visual detection system for rail surface defects. IEEE Trans Syst Man Cybern C (Applications and Reviews) 42(6):1531–1542

    Article  Google Scholar 

  10. Wang Z, Cai B, Chunxiao J et al (2016) Geometry constraints-based visual rail track extraction. In: 2016 12th World Congress on Intelligent Control and Automation (WCICA), (IEEE, 2016), pp 993–998

  11. Li Q, Ren S (2012) A real-time visual inspection system for discrete surface defects of rail heads. IEEE Trans Instrumentation Meas 61(8):2189–2199.

    Article  Google Scholar 

  12. Wu H, Siu W-C (2015) Real time railway extraction by angle alignment measure. In: 2015 IEEE International Conference on Image Processing (ICIP) (IEEE, 2015), pp 4560–4564

  13. Warsop T, Singh S (2012) A framework for computing asset-sighting distance. Pattern Anal Appl 15(4):427–444.

    Article  MathSciNet  Google Scholar 

  14. Nyberg RG, Gupta NK, Yella S, Dougherty MS (2014) Machine vision for condition monitoring vegetation on railway embankments, 2014.

  15. Higgins C, Liu X (2018) Modeling of track geometry degradation and decisions on safety and maintenance: a literature review and possible future research directions. Proc Inst Mech Eng F J Rail Rapid Transit 232(5):1385–1397

    Article  Google Scholar 

  16. Leizer GKK, Tokody D (2017) Radiofrequency identification by using Drones in railway accidents and disaster situations. Interdisciplinary Description Complex Syst 15(2):114.

    Article  Google Scholar 

  17. Máthé K, Buşoniu L (2015) Vision and control for UAVs: a survey of general methods and of inexpensive platforms for infrastructure inspection. Sensors 15(7):14887–14916.

    Article  Google Scholar 

  18. Jang J, Shin M, Lim S, Park J, Kim J, Paik J (2019) Intelligent image-based railway inspection system using deep learning-based object detection and Weber contrast-based image comparison. Sensors 19(21):4738.

    Article  Google Scholar 

  19. Wu Y, Qin Y, Wang Z, Jia L (2018) A UAV-based visual inspection method for rail surface defects. Appl Sci 8(7):1028.

    Article  Google Scholar 

  20. Smith EM (2016) A collection of computer vision algorithms capable of detecting linear infrastructure for the purpose of UAV control. PhD Thesis, Virginia Tech

  21. Wei X, Wei D, Suo D, Jia L, Li Y (2020) Multi-target defect identification for railway track line based on image processing and improved YOLOv3 Model. IEEE Access, 2020.

  22. Frauenthal JM (2015) Design and exploration of a computer vision based unmanned aerial vehicle for railroad health applications. PhD Thesis, Virginia Tech

  23. Taştimur C, Karaköse M, Akın E (2016) A vision based condition monitoring approach for Rail Switch and level crossing using hierarchical SVM in railways. Int J Appl Math Electron Comput 2016, (Special Issue-1):319–325.

  24. Teng Z, Liu F, Zhang B (2016) Visual railway detection by superpixel based intracellular decisions. Multimedia Tools Appl 75(5):2473–2486.

    Article  Google Scholar 

  25. Arun K, Anushka S, Ankush A, Dharmendra S (2017) Vision based rail track extraction and monitoring through drone imagery (KICS, 2017)

  26. Jarrett C, Perry K, Stol KA (2015) Controller comparisons for autonomous railway following with a fixed-wing UAV. In: 2015 6th International Conference on Automation, Robotics and Applications (ICARA) (IEEE, 2015), pp 104–109

  27. Goldenshluger A, Zeevi A (2004) The Hough transform estimator. Ann Stat 32(5):1908–1932.

    Article  MathSciNet  Google Scholar 

  28. Le Saux B, Beaupère A, Boulch A, Brossard J, Manier A, Villemin G (2018) Railway detection: from filtering to segmentation networks. In: ‘IGARSS 2018–2018 IEEE International Geoscience and Remote Sensing Symposium, (IEEE, 2018), pp 4819–4822

  29. Banić M, Miltenović A, Pavlović M, Ćirić I (2019) Intelligent machine vision based railway infrastructure inspection and monitoring using UAV. Facta Universitatis Series: Mech Eng 17(3):357–364.

    Article  Google Scholar 

  30. Ikshwaku S, Srinivasan A, Varghese A, Gubbi J (2019) Railway corridor monitoring using deep drone vision. In: Computational Intelligence: Theories, Applications and Future Directions, vol II. Springer, Cham, pp 361–372

  31. Purica AI, Pesquet-Popescu B, Dufaux F (2017) A railroad detection algorithm for infrastructure surveillance using enduring airborne systems. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2017), pp 2187–2191

  32. Hassanein AS, Mohammad S, Sameer M, Ragab ME (2015) A survey on Hough transform, theory, techniques and applications. arXiv preprint arXiv:1502.02160.

  33. Páli E, Mathe K, Tamas L, Buşoniu L (2014) Railway track following with the AR. Drone using vanishing point detection. In: 2014 IEEE international conference on automation, quality and testing, Robotics’ (IEEE, 2014), pp 1–6

  34. Wang W, Tan-Torres A, Hamledari H (2017) Lecture# 06: edge detection, computer vision: foundations and applications (CS 131), Stanford University

  35. Ruzgienė B, Berteška T, Gečyte S, Jakubauskienė E, Aksamitauskas VČ (2015) The surface modelling based on UAV Photogrammetry and qualitative estimation. Measurement 73:619–627.

    Article  Google Scholar 

  36. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66.

    Article  MathSciNet  Google Scholar 

  37. Wong AK, Sahoo PK (1989) A gray-level threshold selection method based on maximum entropy principle. IEEE Trans Syst Man Cybern 19(4):866–871

    Article  Google Scholar 

  38. Shafarenko L, Petrou H, Kittler J (1998) Histogram-based segmentation in a perceptually uniform color space. IEEE Trans Image Process 7(9):1354–1358.

    Article  Google Scholar 

  39. Dev S, Lee YH, Winkler S (2017) Color-based segmentation of sky/cloud images from ground-based cameras. IEEE J Selected Topics Appl Earth Observations Remote Sensing 10(1):231–242.

    Article  Google Scholar 

  40. Danelljan M, Shahbaz Khan F, Felsberg M, Van de Weijer J (2014) Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014), pp 1090–1097

  41. Shaik KB, Ganesan P, Kalist V, Sathish BS, Jenitha JMM (2015) Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Proc Comput Sci 57:41–48.

    Article  Google Scholar 

  42. Ford A, Roberts A (1998) Colour space conversions. Westminster University, London, pp 1–31

  43. Poynton CA (1996) A technical introduction to digital video. Wiley, New York

    Google Scholar 

  44. Zheng S, Chai X, An X, Li L (2012) Railway track gauge inspection method based on computer vision. In: 2012 IEEE International conference on mechatronics and automation (IEEE, 2012), pp 1292–1296

  45. Kotoulas L, Andreadis I (2003) ‘Colour histogram content-based image retrieval and hardware implementation. IEE Proc-Circuits Devices Syst 150(5):387–393

    Article  Google Scholar 

  46. John C (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intekkugence 8(6).

  47. Hough PV (1962) Patent and Trademark Office. U.S. Patent No. 3,069,654. Washington, DC: U.S

  48. Bradski G, Kaehler A (2008) Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, Inc., Sebastopol

  49. Szeliski R (2010) Computer vision: algorithms and applications. Springer Science & Business Media

Download references

Acknowledgements

The authors are thankful to RailTel, India for supporting this work.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Indian Institute of Technology Roorkee, Roorkee, Uttarakhand, India

    Aradhya Saini & Dharmendra Singh

  2. RailTel - IIT Roorkee Center of Excellence in Telecommunication, Roorkee, Uttarakhand, India

    Dharmendra Singh

Authors
  1. Aradhya Saini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dharmendra Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dharmendra Singh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A

Appendix A

Fig. A1
figure a

HSV histograms for (i) D3 (ii) D4 (iii) D2 (iv) D1

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saini, A., Singh, D. DroneRTEF:development of a novel adaptive framework for railroad track extraction in drone images. Pattern Anal Applic 24, 1549–1568 (2021). https://doi.org/10.1007/s10044-021-00994-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-021-00994-w

Keywords

Search

Navigation