Abstract
Image stitching, or known as image mosaic, is the process that combines images with overlapped areas to form an image with wide view and high resolution. Image stitching technique has been quickly developed these years. It has become an important branch in digital image processing and has wide applications. Many image stitching methods have been proposed. This article takes investigation on some image stitching techniques, including image registration, seam removal and quality assessment. Most existing related methods are introduced. Experiments are done to show the result of some main methods. At last, the advantages and disadvantages of some existing methods are discussed and some future potential work are pointed out.
Similar content being viewed by others
References
Bhosle, U., Chaudhuri, S., Roy, S.D.: A fast method for image mosaicing using geometric hashing. Iete J. Res. 48(3–4), 317–324 (2002)
Laraqui, A., et al.: Image mosaicing using voronoi diagram. Multimed. Tools Appl. 76(6), 8803–8829 (2017)
Kekec, T., Yildirim, A., Unel, M.: A new approach to real-time mosaicing of aerial images. Robot. Autonom. Syst. 62(12), 1755–1767 (2014)
Xie, X., et al.: A study on fast SIFT image mosaic algorithm based on compressed sensing and wavelet transform. J. Ambient Intell. Human. Comput. 6(6), 835–843 (2015)
Ye, M.J., et al.: Automatic seamless stitching method for CCD images of Chang’E-1 lunar mission. J. Earth Sci. 22(5), 610–618 (2011)
Hui, F.M., et al.: An improved landsat image mosaic of Antarctica. Sci China-Earth Sci 56(1), 1–12 (2013)
Bhat, A.S., et al.: Template matching technique for panoramic image stitching. In: Modelling symposium (AMS), 7th Asia. IEEE (2013)
Dekel, T., et al.: Best-buddies similarity for robust template matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition (2015)
Pilchak, A.L., et al.: Using cross-correlation for automated stitching of two-dimensional multi-tile electron backscatter diffraction data. J. Microsc. 248(2), 172–186 (2012)
Adwan, S., Alsaleh, I., Majed, R.: A new approach for image stitching technique using dynamic time warping (DTW) algorithm towards scoliosis X-ray diagnosis. Measurement 84, 32–46 (2016)
Liu, Y.-Y., et al.: A study of image stitching algorithm based on ratio matching. Electro-Opt. Technol. Appl. 6, 17 (2007)
Legesse, F.B., et al.: Seamless stitching of tile scan microscope images. J. Microsc. 258(3), 223–232 (2015)
Nasibov, A., H. Nasibov, Hacizade, F.: Seamless image stitching algorithm using radiometric lens calibration for high resolution optical microscopy. In: Soft computing, computing with words and perceptions in system analysis, decision and control, 2009. ICSCCW 2009. Fifth international conference on. IEEE (2009)
Li, W.T.: Mutual information functions versus correlation-functions. J. Stat. Phys. 60(5–6), 823–837 (1990)
Dame, A., Marchand, E.: Second-order optimization of mutual information for real-time image registration. IEEE Trans. Image Process. 21(9), 4190–4203 (2012)
Rivaz, H., Karimaghaloo, Z., Collins, D.L.: Self-similarity weighted mutual information: a new nonrigid image registration metric. Med. Image Anal. 18(2), 343–358 (2014)
Gong, M.G., et al.: A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Trans. Geosci. Remote Sens. 52(7), 4328–4338 (2014)
Gupta, N., et al.: Extracting information from a query image, for content based image retrieval. In: 2015 eighth international conference on advances in pattern recognition, New York, IEEE, p. 225 (2015)
Liu, H., et al.: Feature selection with dynamic mutual information. Pattern Recogn. 42(7), 1330–1339 (2009)
Soleimani, S., et al.: Online wear detection using high-speed imaging. Microsc. Microanal. 22(4), 820–840 (2016)
Lin, Y., Yu, Q., Medioni, G.: Efficient detection and tracking of moving objects in geo-coordinates. Mach. Vis. Appl. 22(3), 505–520 (2011)
Studholme, C., Hill, D.L.G., Hawkes, D.J.: An overlap invariant entropy measure of 3D medical image alignment. Pattern Recogn. 32(1), 71–86 (1999)
Klein, S., et al.: Automatic segmentation of the prostate in 3D MR images by atlas matching using localized mutual information. Med. Phys. 35(4), 1407–1417 (2008)
Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. (TOG) 2(4), 217–236 (1983)
Wang, C.-Y., Zhou, M.-Q.: A localized pyramid decomposition approach for image mosaic. J. Northw. Univ. (Nat. Sci. Edn.) 3, 12 (2006)
Anuta, P.E.: Spatial registration of multispectral and multitemporal digital imagery using fast Fourier transform techniques. IEEE Trans. Geosci. Electron. 8(4), 353–368 (1970)
Hurtós, N., Petillot, Y., Salvi, J.: Fourier-based registrations for two-dimensional forward-looking sonar image mosaicing. In: Intelligent robots and systems (IROS), 2012 IEEE/RSJ international conference on. IEEE (2012)
Vescovi, R.F.C., Cardoso, M.B., Miqueles, E.X.: Radiography registration for mosaic tomography. J. Synch. Radiat. 24, 686–694 (2017)
Ghantous, M., Ghosh, S., Bayoumi, M.: A multi-modal automatic image registration technique based on complex wavelets. In: Image processing (ICIP), 2009 16th IEEE international conference on. IEEE (2009)
Xing, Y.X., et al.: Robust fast corner detector based on filled circle and outer ring mask. IET Image Proc. 10(4), 314–324 (2016)
Trajkovic, M., Hedley, M.: Fast corner detection. Image Vis. Comput. 16(2), 75–87 (1998)
Förstner, W., Gülch, E.: A fast operator for detection and precise location of distinct points, corners and centres of circular features. In: Proc. ISPRS intercommission conference on fast processing of photogrammetric data (1987)
Moravec, H.P.: Obstacle avoidance and navigation in the real world by a seeing robot rover. DTIC Document (1980)
Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey vision conference. Citeseer (1988)
Harris, C.G., Pike, J.: 3D positional integration from image sequences. Image Vis. Comput. 6(2), 87–90 (1988)
Zhu, M.C., et al.: Efficient video panoramic image stitching based on an improved selection of Harris corners and a multiple-constraint corner matching. Plos One 8(12), e81182 (2013)
Zhu, M.C., et al.: A fast image stitching algorithm via multiple-constraint corner matching. Math. Prob. Eng. 2013, 157847 (2013)
Fu, Z.X., Wang, L.M.: Optimized design of automatic image mosaic. Multimed. Tools Appl. 72(1), 503–514 (2014)
Pan, H., et al.: An adaptive Harris corner detection algorithm for image mosaic. In: Chinese Conference on Pattern Recognition. Springer (2014)
Zhou, Z. et al.: Fast image mosaic algorithm based on the improved Harris-SIFT algorithm, In: International Symposium on Photonics and Optoelectronics. SPIE (2015)
Jiang, Z., Liu, M.: Fast image mosaic algorithm based on the improved Harris-SIFT algorithm. In: Zhou, Z. (Ed.) International Symposium on Photonics and Optoelectronics (2015)
Li, B.P., Guo, C.X., Inc, D.E.P.: Application of image stitching in the scene investigation of traffic accident. In: 2016 international conference on information engineering and communications technology, pp. 127–131 (2016)
Lowe, D.G.: Object recognition from local scale-invariant features. In: Computer vision, 1999. The proceedings of the seventh IEEE international conference on. IEEE (1999)
Brown, M., Lowe, D.G.: Invariant features from interest point groups. In BMVC (2002)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Ni, X.L., et al.: A fully automatic registration approach based on contour and SIFT for HJ-1 images. Sci. China-Earth Sci. 55(10), 1679–1687 (2012)
Guo, F., et al.: Automatic retinal image registration using blood vessel segmentation and SIFT feature. Int. J. Pattern Recognit. Artif. Intell. 31(11), 1757006 (2017)
Goncalves, H., Corte-Real, L., Goncalves, J.A.: Automatic image registration through image segmentation and SIFT. IEEE Trans. Geosci. Remote Sens. 49(7), 2589–2600 (2011)
Wang, V.T., Hayes, M.P.: Synthetic aperture sonar track registration using SIFT image correspondences. IEEE J. Ocean. Eng. 42(4), 901–913 (2017)
Min, Z., Jiguo, Z., Xusheng, X.: Panorama stitching based on sift algorithm and Levenberg–Marquardt optimization. Phys. Procedia 33, 811–818 (2012)
Zhang, Y.H., Jin, X., Wang, Z.J.: A new modified panoramic UAV image stitching model based on the GA-SIFT and adaptive threshold method. Memetic Comput. 9(3), 231–244 (2017)
Qu, Z., et al.: The improved algorithm of fast panorama stitching for image sequence and reducing the distortion errors. Math. Probl. Eng. 2015, 428076 (2015)
Zhu, J., Ren, M.W.: Image mosaic method based on SIFT features of line segment. Comput. Math. Methods Med. 2014, 926312 (2014)
Wang, F.-B., et al.: Multi-image mosaic with SIFT and vision measurement for microscale structures processed by femtosecond laser. Opt. Lasers Eng. 100, 124–130 (2018)
Mills, A., Dudek, G.: Image stitching with dynamic elements. Image Vis. Comput. 27(10), 1593–1602 (2009)
Ke, Y., Sukthankar, R.: PCA-SIFT: a more distinctive representation for local image descriptors. In: Computer vision and pattern recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. IEEE (2004)
Jolliffe, I.: Principal Component Analysis. Wiley Online Library, New York (2002)
Kshirsagar, V., Baviskar, M., Gaikwad, M.: Face recognition using Eigenfaces. In: Computer research and development (ICCRD), 2011 3rd international conference on. IEEE (2011)
Li, B.L., Thomas, G., Williams, D.: Detection of ice on power cables based on image texture features. IEEE Trans. Instrum. Meas. 67(3), 497–504 (2018)
Wang, Q.Q., et al.: l(2, p)-norm based PCA for image recognition. IEEE Trans. Image Process. 27(3), 1336–1346 (2018)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. Comp. Vis. ECCV 2006 Pt 1 Proc. 3951, 404–417 (2006)
Juan, L., Gwun, O.: A comparison of sift, PCA-sift and surf. Int. J. Image Process. (IJIP) 3(4), 143–152 (2009)
Lukashevich, P., Zalesky, B., Ablameyko, S.: Medical image registration based on surf detector. Pattern Recognit. Image Anal. 21(3), 519 (2011)
Wang, G., et al.: Robust point matching method for multimodal retinal image registration. Biomed. Signal Process. Control 19, 68–76 (2015)
Yang, F., Deng, Z.S., Fan, Q.H.: A method for fast automated microscope image stitching. Micron 48, 17–25 (2013)
Yang, Z.L., Shen, D.G., Yap, P.T.: Image mosaicking using SURF features of line segments. Plos One 12(3), e0173627 (2017)
Tsai, C.M., Shih, F.Y.: An efficient image stitching method for heterogeneous car videos based on bounding boxes of features. Int. J. Pattern Recognit. Artif. Intell. 31(5), 1755008 (2017)
Wang, J., et al.: A fast mosaic method for airborne images: the new template-convolution speed-up robust features (TSURF) algorithm. Int. J. Remote Sens. 35(16), 5959–5970 (2014)
Alcantarilla, P.F., Bartoli, A., Davison, A.J.: KAZE Features. Springer, Berlin (2012)
Alcantarilla, P.F., Nuevo, J., Bartoli, A.: Fast explicit diffusion for accelerated features in nonlinear scale spaces. IEEE Trans. Patt. Anal. Mach. Intell. 34, 1281–1298 (2013)
Yang, X., Cheng, K.-T.: Local difference binary for ultrafast and distinctive feature description. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 188–194 (2014)
Thanh, T.M., et al.: Robust semi-blind video watermarking based on frame-patch matching. AEU-Int. J. Electron. Commun. 68(10), 1007–1015 (2014)
Liu, Y., et al.: S-AKAZE: An effective point-based method for image matching. Optik-Int. J. Light Electron Opt. 127(14), 5670–5681 (2016)
Flores-Rodrıguez, K.L., Trujillo-Romero, F.: Free form object recognition module using A-KAZE and GCS. Res. Comput. Sci 118, 19–29 (2016)
Mukherjee, P., Lall, B.: Saliency and KAZE features assisted object segmentation. Image Vis. Comput. 61, 82–97 (2017)
Qu, Z., Bu, W., Liu, L.: The algorithm of seamless image mosaic based on A-KAZE features extraction and reducing the inclination of image. IEEJ Trans. Electr. Electron. Eng. 13(1), 134–146 (2018)
Pohl, C., Van Genderen, J.L.: Review article multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19(5), 823–854 (1998)
Li, H.Y., et al.: An adaptive image-stitching algorithm for an underwater monitoring system. Int. J. Adv. Robot. Syst. 11, 166 (2014)
Li, Z.Y., et al.: A novel image mosaicking algorithm for wireless multimedia sensor networks. Int. J. Distrib. Sensor Netw. 9, 719640 (2013)
Zhu, L., et al.: Review on image fusion research. Transduces Microsyst. Technol. 33(2), 14–18 (2014)
Zhou, D.-F., He, M.-Y., Yang, Q.: A robust seamless image stitching algorithm based on feature points. Meas. Control Technol. 6, 10 (2009)
Zhao, G., Lin, L., Tang, Y.D.: A new optimal seam finding method based on tensor analysis for automatic panorama construction. Pattern Recogn. Lett. 34(3), 308–314 (2013)
Abdukholikov, M., Whangbo, T.: Fast image stitching method for handling dynamic object problems in Panoramic Images. Ksii Trans. Internet Inf. Syst. 11(11), 5419–5435 (2017)
Jeong, J., Jun, K.: A novel seam finding method using downscaling and cost for image stitching. J. Sens. 2016, 5258473 (2016)
Li, M., et al.: A stereo dual-channel dynamic programming algorithm for UAV image stitching. Sensors 17(9), 2060 (2017)
Li, L., et al.: A unified framework for street-view panorama stitching. Sensors 17(1), 1 (2017)
Li, L., et al.: Optimal seamline detection for multiple image mosaicking via graph cuts. ISPRS J. Photogram. Remote Sens. 113, 1–16 (2016)
Lee, D., Lee, S.: Seamless image stitching by homography refinement and structure deformation using optimal seam pair detection. J. Electron. Imaging 26(6), 063016 (2017)
Wang, Y., et al.: Microscopic image enhancement based on Fourier ptychography technique. SPIE Defense + Commercial Sensing, vol 10990. SPIE (2019)
Chen, M., et al.: Underwater image stitching based on SIFT and wavelet fusion. Oceans—Genova (2015)
Popovic, V., Leblebici, Y.: FIR filters for hardware-based real-time multi-band image blending. In: Kehtarnavaz, N., Carlsohn, M.F. (Eds) Real-Time Image and Video Processing (2015)
Candes, E.J., Guo, F.: New multiscale transforms, minimum total variation synthesis: applications to edge-preserving image reconstruction. Signal Process. 82(11), 1519–1543 (2002)
Srivastava, R., Prakash, O., Khare, A.: Local energy-based multimodal medical image fusion in curvelet domain. IET Comput. Vis. 10(6), 513–527 (2016)
Yang, H., et al.: A seismic interpolation and denoising method with curvelet transform matching filter. Acta Geophys. 65(5), 1029–1042 (2017)
Gai, S.: Multichannel image denoising using color monogenic curvelet transform. Soft. Comput. 22(2), 635–644 (2018)
Math, S.S.P., Kaliyaperumal, V.: Enhancement of SAR images using fuzzy shrinkage technique in curvelet domain. Sadhana-Acad. Proc. Eng. Sci. 42(9), 1505–1512 (2017)
Ali, F.E., et al.: A curvelet transform approach for the fusion of MR and CT images. J. Mod. Opt. 57(4), 273–286 (2010)
Gattim, N.K., et al.: multimodal image fusion using curvelet and genetic algorithm. J. Sci. Ind. Res. 76(11), 694–696 (2017)
Candes, E., Demanet, L.: Curvelets and fourier integral operators. Comptes Rendus Math. 336(5), 395–398 (2003)
Wu, Q., et al.: A perceptually weighted rank correlation indicator for objective image quality assessment. IEEE Trans. Image Process. 27(5), 2499–2513 (2018)
Huynh-Thu, Q., Ghanbari, M.: Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44(13), 800–U35 (2008)
Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)
Wang, Z., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Thomos, N., Boulgouris, N.V., Strintzis, M.G.: Optimized transmission of JPEG2000 streams over wireless channels. IEEE Trans. Image Process. 15(1), 54–67 (2006)
Hou, W.L., et al.: Blind image quality assessment via deep learning. IEEE Trans. Neural Netw. Learn. Syst. 26(6), 1275–1286 (2015)
Guo-ting, W., et al.: Method for quality assessment of image mosaic. J. Commun. 8, 011 (2013)
Qureshi, H.S., et al.: Quantitative quality assessment of stitched panoramic images. IET Image Proc. 6(9), 1348–1358 (2012)
Acknowledgements
This work was jointly supported by National Natural Science Foundation of China (Grant no. 61201421), China Postdoctoral Science Foundation (Grant no. 2013M532097).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.
Additional information
Communicated by Y. Zhang.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, Z., Yang, Z. Review on image-stitching techniques. Multimedia Systems 26, 413–430 (2020). https://doi.org/10.1007/s00530-020-00651-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00530-020-00651-y