Abstract
In this paper, we consider object classification and detection problems. We propose an algorithm that is effective from the point of view of computational complexity and memory consumption. The proposed algorithm can be successfully used as a basic tool for building different remote sensing systems which are, in general, installed on UAVs. The algorithm is based on the Viola–Jones method. It is shown in the paper, that the Viola–Jones method is the most preferable approach to detect objects on-board UAVs, because it needs the least amount of memory and the number of computational operations to solve the object detection problem. To ensure sufficient accuracy, we use a modified feature: rectangular Haar-like features, calculated over the magnitude of the image gradient. To increase computational efficiency, the L1 norm was used to calculate the magnitude of the image gradient. To train orientation-independent complex classifier we use a more generic decision tree form of complex classifier instead of a cascade scheme. All mentioned improvements were evaluated during detection of the following objects: the PSN-10 inflatable life raft (an example of an object that is detected during rescue operations using UAVs), oil tank storage (such kind of objects are usually detected during the inspection of industrial infrastructure), and aircraft on an area of hardstand. The performance of the trained detectors was estimated on real data (including data obtained during the rescue operation of the trawler Dalniy Vostok and a subset of real images from the DOTA dataset).
Similar content being viewed by others
REFERENCES
I. I. Afanasyev, V. N. Laptev, and V. P. Pirogov, “Analysis of the rescue assets range of the Russian navy,” Nauchn. Vestn. Vol’skogo Voen. Inst. Mater. Obespecheniya: Voen.-Nauchn. Zh., No. 2, 150–154 (2015).
J. Cho, G. Lim, T. Biobaku, S. Kim, and H. Parsaei, “Safety and security management with unmanned aerial vehicle (UAV) in oil and gas industry,” Procedia Manuf. 3, 1343–1349 (2015).
D. Du et al., “The unmanned aerial vehicle benchmark: Object detection and tracking,” in Proceedings of the European Conference on Computer Vision (ECCV) (2018), pp. 370–386.
A. V. Gayer, Y. S. Chernyshova, and A. V. Sheshkus, “Effective real-time augmentation of training dataset for the neural networks learning,” in Proceedings of the Eleventh International Conference on Machine Vision (2019). https://doi.org/10.1117/12.2522969
E. Granger, M. Kiran, and L. A. Blais-Morin, “A comparison of CNN-based face and head detectors for real-time video surveillance applications,” in 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA) (IEEE, 2017), pp. 1–7.
D. Hulens, T. Goedemé, and J. Verbeke, “How to choose the best embedded processing platform for on-board UAV image processing?,” in Proceedings VISAPP (2015), pp. 1–10.
V. Jain and E. Learned-Miller, FDDB: A Benchmark for Face Detection in Unconstrained Settings. Technical Report UM-CS-2010-009 (Univ. Massachusetts, Amherst, 2010).
A. A. Kotov, S. A. Usilin, S. A. Gladilin, and D. P. Nikolaev, “Construction of robust features for detection and classification of objects without characteristic brightness contrasts,” J. Inf. Technol. Comput. Syst. 1, 53–60 (2014).
F. S. Leira, T. A. Johansen, and T. I. Fossen, “Automatic detection, classification and tracking of objects in the ocean surface from UAVs using a thermal camera,” in 2015 IEEE Aerospace Conference (IEEE, 2015), pp. 1–10.
J. P. Lewis, “Fast template matching,” in Proceedings of the Vision Interface (1995), pp. 120–123.
D. P. Matalov, S. A. Usilin, and V. V. Arlazarov, “Modification of the Viola-Jones approach for the detection of the government seal stamp of the Russian Federation,” in Proceedings of Eleventh International Conference on Machine Vision (ICMV 2018) (2019), Vol. 1104. https://doi.org/10.1117/12.2522793
A. Minkina, D. Nikolaev, S. Usilin, and V. Kozyrev, “Generalization of the Viola-Jones method as a decision tree of strong classifiers for real-time object recognition in video stream,” in Seventh International Conference on Machine Vision (ICMV 2014) (2015), Vol. 9445. https://doi.org/10.1117/12.2180941
C. P. Papageorgiou, M. Oren, and T. Poggio, “A general framework for object detection,” in Proceedings of the Sixth Int. Conf. Comput. Vis. (IEEE, 1998), Vol. 6, pp. 555–562.
S. A. Usilin, V. V. Arlazarov, N. S. Rokhlin, S. A. Rudyka, S. A. Matveev, and A. A. Zatsarinny, “Training Viola-Jones detectors for 3D objects based on fully synthetic data for use in rescue missions with UAV,” Vestn. YuUrGU, Ser. Mat. Model. Progr. 13 (4), 94–106 (2020).
S. A. Usilin, P. V. Bezmaternykh, and V. V. Arlazarov, “Fast approach for QR code localization on images using Viola-Jones method,” in Proceedings of the Twelfth International Conference on Machine Vision (2020). https://doi.org/10.1117/12.2559386.
P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pat-tern Recognition (CVPR 2001) (2001), Vol. 1, pp. 511–518.
P. Viola and M. Jones, “Robust real-time object detection,” Proc. Int. J. Comput. Vision 4, 34–47 (2001).
G. S. Xia, X. Bai, J. Ding, Z. Zhu, S. Belongie, J. Luo, M. Datcu, M. Pelillo, and L. Zhang, “DOTA: A large-scale dataset for object detection in aerial images,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018), pp. 3974–3983.
H. Yu, G. Li, W. Zhang, Q. Huang, D. Du, Q. Tian, and N. Sebe, “The unmanned aerial vehicle benchmark: Object detection, tracking and baseline,” Int. J. Comput. Vision 128 (5), 1141–1159 (2020).
A. Zhao, K. Fu, H. Sun, X. Sun, F. Li, D. Zhang, and H. Wang, “An effective method based on ACF for aircraft detection in remote sensing images,” IEEE Geosci. Remote Sens. Lett. 14 (5), 744–748 (2017).
Funding
This work is partially supported by the Russian Foundation for Basic Research (project nos. 18-29-26022 and 18-29-26020).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
COMPLIANCE WITH ETHICAL STANDARDS
This manuscript is a completely original work of its authors; it has not been published before and will not be published in other sources.
CONFLICT OF INTEREST
The content of the article does not give grounds for raising the issue of a conflict of interest.
Additional information
Sergey Alexandrovich Usilin. Born in 1986. Graduated from Moscow Institute of Physics and Technology in 2009. Obtained Candidate’s degree in 2018. Works as a Senior Researcher at the Federal Research Center Computer Science and Control, Russian Academy of Sciences. Scope of scientific interests: object detection, machine learning, recognition systems, and digital image processing.
Oleg Anatolevich Slavin. Born in 1963. Graduated from Moscow Institute of Radio Engineering, Electronics, and Automation in 1988. Obtained candidate’s and doctoral degrees in 2000 and 2011, respectively. Works as a Chief Research Officer and Head of Department no. 92 at the Federal Research Center Computer Science and Control, Russian Academy of Sciences. Scope of scientific interests: information systems and pattern recognition.
Vladimir Viktorovich Arlazarov. Born in 1976. Graduated from Moscow Institute of Steel and Alloys in 1999. Received his Candidate’s degrees in 2005. Works as a Head of Department no. 93 at the Federal Research Center Computer Science and Control, Russian Academy of Sciences. Scope of scientific interests: artificial intelligence, machine learning, recognition systems, and information technology.
Rights and permissions
About this article
Cite this article
Usilin, S.A., Slavin, O.A. & Arlazarov, V.V. Memory Consumption and Computation Efficiency Improvements of Viola–Jones Object Detection Method for Remote Sensing Applications. Pattern Recognit. Image Anal. 31, 571–579 (2021). https://doi.org/10.1134/S1054661821030238
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1054661821030238