Skip to main content
SpringerLink
Log in

Deep multiple instance learning for airplane detection in high-resolution imagery

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Automatic airplane detection in aerial imagery has a variety of applications. Two of the significant challenges in this task are variations in the scale and direction of the airplanes. To solve these challenges, we present a rotation-and-scale-invariant airplane proposal generator. We call this generator symmetric line segments (SLS) that is developed based on the symmetric and regular boundaries of airplanes from the top view. Then, the generated proposals are used to train a deep convolutional neural network for removing non-airplane proposals. Since each airplane can have multiple SLS proposals, where some of them are not in the direction of the fuselage, we collect all proposals corresponding to one ground truth as a positive bag and the others as the negative instances. To have multiple instance deep learning, we modify the loss function of the network to learn from each positive bag at least one instance as well as all negative instances. Finally, we employ non-maximum suppression to remove duplicate detections. Our experiments on NWPU VHR-10 and DOTA datasets show that our method is a promising approach for automatic airplane detection in very high-resolution images. Moreover, we estimate the direction of the airplanes using box-level annotations as an extra achievement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Amores, J.: Multiple instance classification: review, taxonomy and comparative study. Artif. Intell. 201, 81–105 (2013)

    Article  MathSciNet  Google Scholar 

  2. An, Z., Shi, Z., Teng, X., Yu, X., Tang, W.: An automated airplane detection system for large panchromatic image with high spatial resolution. Optik Int. J. Light Electron Opt. 125(12), 2768–2775 (2014)

    Article  Google Scholar 

  3. Bai, X., Zhang, H., Zhou, J.: Vhr object detection based on structural feature extraction and query expansion. IEEE Trans. Geosci. Remote Sens. 52(10), 6508–6520 (2014)

    Article  Google Scholar 

  4. Bo, S., Jing, Y.: Region-based airplane detection in remotely sensed imagery. In: 2010 3rd International Congress on Image and Signal Processing (CISP), vol. 4, pp. 1923–1926. IEEE (2010)

  5. Cai, H., Su, Y.: Airplane detection in remote sensing image with a circle-frequency filter. In: International Conference on Space Information Technology, vol. 5985, p. 59852T. International Society for Optics and Photonics (2006)

  6. Chen, J., Wan, L., Zhu, J., Xu, G., Deng, M.: Multi-scale spatial and channel-wise attention for improving object detection in remote sensing imagery. IEEE Geosci. Remote Sens. Lett. 17(4), 681–685 (2019)

    Article  Google Scholar 

  7. Cheng, G., Han, J.: A survey on object detection in optical remote sensing images. ISPRS J. Photogram. Remote Sens. 117, 11–28 (2016)

    Article  Google Scholar 

  8. Cheng, G., Han, J., Guo, L., Qian, X., Zhou, P., Yao, X., Hu, X.: Object detection in remote sensing imagery using a discriminatively trained mixture model. ISPRS J. Photogram. Remote Sens. 85, 32–43 (2013)

    Article  Google Scholar 

  9. Cheng, G., Han, J., Zhou, P., Guo, L.: Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS J. Photogram. Remote Sens. 98, 119–132 (2014)

    Article  Google Scholar 

  10. Cheng, G., Han, J., Zhou, P., Guo, L.: Scalable multi-class geospatial object detection in high-spatial-resolution remote sensing images. In: Geoscience and Remote Sensing Symposium (IGARSS), 2014 IEEE International, pp. 2479–2482. IEEE (2014)

  11. Cheng, G., Zhou, P., Han, J.: Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Trans. Geosci. Remote Sens. 54(12), 7405–7415 (2016)

    Article  Google Scholar 

  12. Cheng, G., Zhou, P., Yao, X., Yao, C., Zhang, Y., Han, J.: Object detection in VHR optical remote sensing images via learning rotation-invariant hog feature. In: 2016 4th International Workshop on Earth Observation and Remote Sensing Applications (EORSA), pp. 433–436. IEEE (2016)

  13. Cheng, M.M., Zhang, Z., Lin, W.Y., Torr, P.: Bing: binarized normed gradients for objectness estimation at 300fps. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3286–3293 (2014)

  14. Chollet, F., et al.: Keras. https://keras.io (2015)

  15. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)

  16. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 248–255. IEEE (2009)

  17. Deng, Z., Sun, H., Zhou, S., Zhao, J., Lei, L., Zou, H.: Multi-scale object detection in remote sensing imagery with convolutional neural networks. ISPRS J. Photogram. Remote Sens. 145, 3–22 (2018)

    Article  Google Scholar 

  18. Ding, J., Xue, N., Long, Y., Xia, G.S., Lu, Q.: Learning roi transformer for detecting oriented objects in aerial images. arXiv preprint arXiv:1812.00155 (2018)

  19. Duda, R.O., Hart, P.E.: Use of the Hough transformation to detect lines and curves in pictures. Commun. ACM 15(1), 11–15 (1972)

    Article  Google Scholar 

  20. Farooq, A., Hu, J., Jia, X.: Efficient object proposals extraction for target detection in VHR remote sensing images. In: 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 3337–3340. IEEE (2017)

  21. Gao, F., Xu, Q., Li, B.: Aircraft detection from VHR images based on circle-frequency filter and multilevel features. Sci. World J. 2013, 1–7 (2013)

    Google Scholar 

  22. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)

  23. Hosang, J., Benenson, R., Dollár, P., Schiele, B.: What makes for effective detection proposals? IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 814–830 (2016)

    Article  Google Scholar 

  24. Hosomura, T., et al.: Airplane extraction from high resolution satellite image using boundary feature. In: Proceedings of ISPRS Technical Com. VIII Symposium (2010)

  25. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  26. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243

  27. Humayun, A., Li, F., Rehg, J.M.: Rigor: Reusing inference in graph cuts for generating object regions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 336–343 (2014)

  28. Inglada, J.: Automatic recognition of man-made objects in high resolution optical remote sensing images by SVM classification of geometric image features. ISPRS J. Photogram. Remote Sens. 62(3), 236–248 (2007)

    Article  Google Scholar 

  29. James, J., Ford, J.J., Molloy, T.L.: Below horizon aircraft detection using deep learning for vision-based sense and avoid. In: 2019 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 965–970. IEEE (2019)

  30. Kawato, S., Tetsutani, N.: Circle-frequency filter and its application. In: ITE Technical Report 25.12, pp. 49–54. The Institute of Image Information and Television Engineers (2001)

  31. Khan, M.J., Khan, H.S., Yousaf, A., Khurshid, K., Abbas, A.: Modern trends in hyperspectral image analysis: a review. IEEE Access 6, 14118–14129 (2018)

    Article  Google Scholar 

  32. Khan, M.J., Yousaf, A., Javed, N., Nadeem, S., Khurshid, K.: Automatic target detection in satellite images using deep learning. J. Space Technol. 7(1), 44–49 (2017)

    Google Scholar 

  33. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  34. Kumar, G., Bhatia, P.K.: A detailed review of feature extraction in image processing systems. In: 2014 Fourth International Conference on Advanced Computing & Communication Technologies (ACCT), pp. 5–12. IEEE (2014)

  35. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  36. Li, Y., Fu, K., Sun, H., Sun, X.: An aircraft detection framework based on reinforcement learning and convolutional neural networks in remote sensing images. Remote Sensing 10(2), 243 (2018)

    Article  Google Scholar 

  37. Li, Y., Sun, X., Wang, H., Sun, H., Li, X.: Automatic target detection in high-resolution remote sensing images using a contour-based spatial model. IEEE Geosci. Remote Sens. Lett. 9(5), 886–890 (2012)

    Article  Google Scholar 

  38. Li, Z., Itti, L.: Saliency and gist features for target detection in satellite images. IEEE Trans. Image Process. 20(7), 2017–2029 (2011)

    Article  MathSciNet  Google Scholar 

  39. Li, Z., Xu, K., Xie, J., Bi, Q., Qin, K.: Deep multiple instance convolutional neural networks for learning robust scene representations. IEEE Trans. Geosci. Remote Sens. 58(5), 3685–3702 (2020)

    Article  Google Scholar 

  40. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. (2018). https://doi.org/10.1109/TPAMI.2018.2858826

    Article  Google Scholar 

  41. Liu, Q., Xiang, X., Wang, Y., Luo, Z., Fang, F.: Aircraft detection in remote sensing image based on corner clustering and deep learning. Eng. Appl. Artif. Intell. 87, 103333 (2020)

    Article  Google Scholar 

  42. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer, Berlin (2016)

  43. Long, Y., Gong, Y., Xiao, Z., Liu, Q.: Accurate object localization in remote sensing images based on convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 55(5), 2486–2498 (2017)

    Article  Google Scholar 

  44. Luo, Q., Shi, Z.: Airplane detection in remote sensing images based on object proposal. In: 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 1388–1391. IEEE (2016)

  45. Petridis, S., Geyer, C., Singh, S.: Learning to detect aircraft at low resolutions. In: International Conference on Computer Vision Systems, pp. 474–483. Springer (2008)

  46. Qiu, S., Wen, G., Deng, Z., Fan, Y., Hui, B.: Automatic and fast PCM generation for occluded object detection in high-resolution remote sensing images. IEEE Geosci. Remote Sens. Lett. 14, 1730–1734 (2017)

    Article  Google Scholar 

  47. Qiu, S., Wen, G., Fan, Y.: Occluded object detection in high-resolution remote sensing images using partial configuration object model. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 10(5), 1909–1925 (2017)

    Article  Google Scholar 

  48. Qiu, S., Wen, G., Liu, J., Deng, Z., Fan, Y.: Unified partial configuration model framework for fast partially occluded object detection in high-resolution remote sensing images. Remote Sens. 10(3), 464 (2018)

    Article  Google Scholar 

  49. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  50. Seelan, S.K., Laguette, S., Casady, G.M., Seielstad, G.A.: Remote sensing applications for precision agriculture: a learning community approach. Remote Sens. Environ. 88(1–2), 157–169 (2003)

    Article  Google Scholar 

  51. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  52. Sizkouhi, A.M.M., Aghaei, M., Esmailifar, S.M., Mohammadi, M.R., Grimaccia, F.: Automatic boundary extraction of large-scale photovoltaic plants using a fully convolutional network on aerial imagery. IEEE J. Photovolt. 10(4), 1061–1067 (2020)

    Article  Google Scholar 

  53. Sun, H., Sun, X., Wang, H., Li, Y., Li, X.: Automatic target detection in high-resolution remote sensing images using spatial sparse coding bag-of-words model. IEEE Geosci. Remote Sens. Lett. 9(1), 109–113 (2012)

    Article  Google Scholar 

  54. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  55. Tan, M., Le, Q.V.: Efficientnet: rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946 (2019)

  56. Uijlings, J.R., Van De Sande, K.E., Gevers, T., Smeulders, A.W.: Selective search for object recognition. Int. J. Comput. Vision 104(2), 154–171 (2013)

    Article  Google Scholar 

  57. Von Gioi, R.G., Jakubowicz, J., Morel, J.M., Randall, G.: LSD: a fast line segment detector with a false detection control. IEEE Trans. Pattern Anal. Mach. Intell. 32(4), 722–732 (2010)

    Article  Google Scholar 

  58. Wang, S., Wang, M., Zhao, X., Liu, D.: Two-stage object detection based on deep pruning for remote sensing image. In: International Conference on Knowledge Science, Engineering and Management, pp. 137–147. Springer (2018)

  59. Wu, H., Zhang, H., Zhang, J., Xu, F.: Fast aircraft detection in satellite images based on convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4210–4214. IEEE (2015)

  60. Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: Dota: A large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3974–3983 (2018)

  61. Xu, C., Li, C., Cui, Z., Zhang, T., Yang, J.: Hierarchical semantic propagation for object detection in remote sensing imagery. IEEE Trans. Geosci. Remote Sens. 58(6), 4353–4364 (2020)

    Article  Google Scholar 

  62. Xu, Z., Xu, X., Wang, L., Yang, R., Pu, F.: Deformable convnet with aspect ratio constrained NMS for object detection in remote sensing imagery. Remote Sens. 9(12), 1312 (2017)

    Article  Google Scholar 

  63. Yu, Y., Guan, H., Ji, Z.: Rotation-invariant object detection in high-resolution satellite imagery using superpixel-based deep Hough forests. IEEE Geosci. Remote Sens. Lett. 12(11), 2183–2187 (2015)

    Article  Google Scholar 

  64. Yu, Y., Guan, H., Zai, D., Ji, Z.: Rotation-and-scale-invariant airplane detection in high-resolution satellite images based on deep-Hough-forests. ISPRS J. Photogram. Remote Sens. 112, 50–64 (2016)

    Article  Google Scholar 

  65. Zhang, W., Jiao, L., Liu, X., Liu, J.: Multi-scale feature fusion network for object detection in vhr optical remote sensing images. In: IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 330–333. IEEE (2019)

  66. Zhang, W., Lv, W., Zhang, Y., Tian, J., Ma, J.: Unsupervised-learning airplane detection in remote sensing images. In: MIPPR 2015: Remote Sensing Image Processing, Geographic Information Systems, and Other Applications, vol. 9815, p. 981503. International Society for Optics and Photonics (2015)

  67. Zhang, X., Zhu, K., Chen, G., Tan, X., Zhang, L., Dai, F., Liao, P., Gong, Y.: Geospatial object detection on high resolution remote sensing imagery based on double multi-scale feature pyramid network. Remote Sens. 11(7), 755 (2019)

    Article  Google Scholar 

  68. Zheng, J., Xi, Y., Feng, M., Li, X., Li, N.: Object detection based on bing in optical remote sensing images. In: 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 504–509. IEEE (2016)

  69. Zhong, Y., Han, X., Zhang, L.: Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery. ISPRS J. Photogram. Remote Sens. 138, 281–294 (2018)

    Article  Google Scholar 

  70. Zitnick, C.L., Dollár, P.: Edge boxes: locating object proposals from edges. In: European Conference on Computer Vision, pp. 391–405. Springer, Berlin (2014)

  71. Zou, Z., Shi, Z.: Random access memories: a new paradigm for target detection in high resolution aerial remote sensing images. IEEE Trans. Image Process. 27(3), 1100–1111 (2018)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran

    Mohammad Reza Mohammadi

Authors
  1. Mohammad Reza Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Reza Mohammadi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohammadi, M.R. Deep multiple instance learning for airplane detection in high-resolution imagery. Machine Vision and Applications 32, 36 (2021). https://doi.org/10.1007/s00138-020-01153-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01153-7

Keywords

Search

Navigation