Abstract
Among the tasks solved by artificial neural networks are the tasks of analyzing objects on the images of the underlying Earth’s surface, obtained by the on-board equipment of unmanned aerial vehicle (UAV). For the solution of such problems, the convolutional neural networks (CNN), operating semantic segmentation of the received image, are widely used. In this case, the designer of such networks has to solve the difficult task of selecting hyperparameter values for them. These values’ choice is one of the most critical tasks that have to be solved when forming a CNN. Existing attempts to solve this problem are usually based on one of two approaches. The first one involves a set of experiments with different values of hyperparameters of the CNN with learning each of the network variants. These experiments are performed until a CNN with acceptable characteristics is obtained. This approach is simple to implement but does not guarantee a CNN with high performance. The second approach treats the selection of hyperparameter values in the network as an optimization problem. If this problem is successfully solved, it is possible to obtain a CNN with sufficiently high characteristics. However, this task has a significant complexity and also requires a large consumption of computing resources. Images in the form of multidimensional arrays are used as source data to analyze objects on the underlying surface. It means that CNN will contain a significant number of parameters. Accordingly, it will take considerable time to find a suitable CNN by searching for possible hyperparameter values. This paper proposes an alternative approach to the problem of selecting the hyperparameter values of CNN based on the analysis of the processes running in the network. The effectiveness of this approach is demonstrated by solving the problem of semantic segmentation of the underlying surface obtained by remote sensing of the Earth’s surface.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Finn, A. and Scheding, S., Developments and Challenges for Autonomous Unmanned Vehicles, Springer, 2010.
Valavanis, K.P., Ed., Advances in Unmanned Aerial Vehicles: State of the Art and the Road to Autonomy, Springer, 2007.
Shakirov, V., Solovyeva, K., and Dunin-Barkowski, W., Review of state-of-the-art in deep learning artificial intelligence, Opt. Mem. Neural Networks, 2018, vol. 27, no. 2, pp. 65–80.
Neapolitan, R.E. and Jiang, X.P., Artificial Intelligence with an Introduction to Machine Learning, London: CRC Press, 2018.
Favorskaya, M.N. and Jain, L.C., Computer Vision in Control Systems, Vol. 3: Aerial and Satellite Image Processing, Springer, 2018.
Szeliski, R., Computer Vision: Algorithms and Applications, Springer, 2011.
Gonzalez, R.C. and Woods, R.E., Digital Image Processing, 2nd ed., Prentice-Hall, 2002.
Goodfellow, I., Bengio, Y., and Courville, A., Deep Learning, MIT Press, 2016.
Igonin, D.M. and Tiumentsev, Yu.V., Efficiency analysis for various neuroarchitectures for semantic segmentation of images in remote sensing applications, Opt. Mem. Neural Networks, 2019, vol. 28, no. 4, pp. 306–320.
Igonin, D.M. and Tiumentsev, Yu.V., Semantic segmentation of images obtained by remote sensing of the Earth, in Advances in Neural Computation, Machine Learning, and Cognitive Research III. NEUROINFORMATICS 2019, Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V., and Tiumentsev, Y., Eds., Cham: Springer, 2020, pp. 309–318.
Zhao, Z.-Q. et al., Object detection with deep learning: A review, arXiv: 1807.05511v2 [cs.CV], 2019.
Gu, J. et al., Recent advances in convolutional neural networks, arXiv: 1512.07108v6, 2017.
Krishnakumari, K. et al., Hyperparameter tuning in convolutional neural networks for domain adaptation in sentiment classification (HTCNN-DASC), Soft Comput., 2019, vol. 24, pp. 3511–3527.
Neary, P.L., Automatic hyperparameter tuning in deep convolutional neural networks using asynchronous reinforcement learning, in Proc. 2018 IEEE Intern. Conference on Cognitive Computing, 2018, pp. 73–77.
Florea, A. and Andonie, R., Weighted random search for hyperparameter optimization, arXiv: 2004.01628v1, 2004.
Bergstra, J. and Bengio, Y., Random search for hyper-parameter optimization, J. Mach. Learning Res., 2012, vol. 13, pp. 281–305.
Feurer, M. and Hutter, F., Hyperparameter optimization, in Automated Machine Learning, Hutter, F., Ed., Berlin: Springer, 2019, pp. 3–33.
Cardona-Escobar, A.F. et al., Efficient hyperparameter optimization in convolutional neural networks by learning curves prediction, Lect. Notes Comput. Sci., 2018, vol. 10657, pp. 143–151.
Diaz, G.I. et al., An effective algorithm for hyperparameter optimization of neural networks, IBM J. Res. Devel., 2017, vol. 61, nos. 4/5, pp. 9:1–9:11.
Hinz, T. et al., Speeding up the hyperparameter optimization of deep convolutional neural networks, Int. J. Comput. Intell. Appl., 2018, vol. 17, no. 2, p. 15.
var Thokle Hovden, Optimizing Artificial Neural Network Hyperparameters and Architecture, University of Oslo, 2019. https://www.mn.uio.no/fysikk/english/people/aca/ivarth/works/in9400nnhponashovdenr2.pdf.
WorldView-3 Satellite Imagery, DigitalGlobe, Inc., 2017. https://www.digitalglobe.com/products/satellite-imagery.
Qu, J.J. et al., Earth Science Satellite Remote Sensing, Vol. 2: Data, Computational Processing, and Tools, Springer, 2006.
Awesome Semantic Segmentation, 2019. https://github.com/mrgloom/awesome-semantic-segmentation.
Ronneberger, O., Fischer, P., and Brox T., U-Net: Convolutional networks for biomedical image segmentation, arXiv: 1505.04597v1 [cs.CV], 2015.
Badrinarayanan, V., Kendall, A., and Cipolla, R., SegNet: A deep convolutional encoder-decoder architecture for image segmentation, arXiv: 1511.00561v3 [cs.CV], 2016.
The brain from the inside (visualization of the pattern passing through the model of artificial neural network), 2019. https://habr.com/ru/post/438972/F.
Funding
This research is supported by the Ministry of Science and Higher Education of the Russian Federation as Project no. 9.7170.2017/8.9.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The authors declare that they have no conflicts of interest.
About this article
Cite this article
Igonin, D.M., Kolganov, P.A. & Tiumentsev, Y.V. Choosing Hyperparameter Values of the Convolution Neural Network When Solving the Problem of Semantic Segmentation of Images Obtained by Remote Sensing of the Earth’s Surface. Opt. Mem. Neural Networks 29, 317–329 (2020). https://doi.org/10.3103/S1060992X20040086
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3103/S1060992X20040086