Skip to main content
SpringerLink
Log in

FPGA-based infrared image deblurring using angular position of IR detector

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The motion of the object or the infrared (IR) imaging system during the integration time causes blurring of the IR image. This study covers real-time field programmable gate array (FPGA)-based deblurring for IR detectors, and an inertial measurement unit (IMU) was used to quantify the blur caused by the IR detector movement. Point spread function for each pixel was calculated using the angular position data of the IR detector obtained from IMU. Both spatially invariant and spatially variant blur cases can be modeled for the IR detector motion. After the quantification, the spatially invariant-type blur was eliminated using a Wiener filter-based deblurring algorithm. Deblurring algorithm was implemented in the Xilinx system generator environment directly using FPGA IP cores. The simulation results in the Xilinx system generator environment indicate that the proposed image deblurring method is real-time applicable, and it reduces the processing time of a single frame to 4 ms. For the implementation of 2D-fast Fourier transform design in FPGA using the corner turn matrix method, memory management is the most critical factor influencing the speed. The real-time deblurring solution given herein has the potential to be used in IR cameras on the moving platforms to increase the performance and robustness in systems such as object tracking and visual navigation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Richardson, W.H.: Bayesian-based iterative method of image restoration. JoSA 62(1), 55–59 (1972)

    Article  Google Scholar 

  2. Kundur, D., Hatzinakos, D.: Blind image deconvolution. IEEE Signal Process. Mag. 13(3), 43–64 (1996)

    Article  Google Scholar 

  3. Zhang, X., Sun, F., Liu, G., Ma, Y.: Non-blind deblurring of structured images with geometric deformation. Vis. Comput. 31(2), 131–140 (2015)

    Article  Google Scholar 

  4. Yang, H., Zhang, Z., Guan, Y.: Rolling bilateral filter-based text image deblurring. Vis. Comput. 35(11), 1627–1640 (2019)

    Article  Google Scholar 

  5. Hunt, B.R.: The application of constrained least squares estimation to image restoration by digital computer. IEEE Trans. Comput. 100(9), 805–812 (1973)

    Article  Google Scholar 

  6. Kang, M.G., Katsaggelos, A.K.: General choice of the regularization functional in regularized image restoration. IEEE Trans. Image Process. 4(5), 594–602 (1995)

    Article  Google Scholar 

  7. Feng, Q., Fei, H., Wencheng, W.: Blind image deblurring with reinforced use of edges. Vis. Comput. 35(6–8), 1081–1090 (2019)

    Article  Google Scholar 

  8. Segall, C.A., Katsaggelos, A.K.: Digital Image Restoration—Classical. Encycl. Opt. Eng. Abe-Las 1(1), 411–427 (2003)

    Google Scholar 

  9. Yuan, L., Sun, J., Quan, L., Shum, H. Y.: Image deblurring with blurred/noisy image pairs. In: ACM SIGGRAPH 2007 papers, pp. 1-es (2007)

  10. Raskar, R., Agrawal, A., Tumblin, J.: Coded exposure photography: motion deblurring using fluttered shutter. In: ACM SIGGRAPH 2006 Papers, pp. 795–804 (2006)

  11. Joshi, N., Kang, S.B., Zitnick, C.L., Szeliski, R.: Image deblurring using inertial measurement sensors. ACM Trans. Graph. 29(4), 1–9 (2010)

    Article  Google Scholar 

  12. Oswald-Tranta, B., Sorger, M., O’Leary, P.: Motion deblurring of infrared images from a microbolometer camera. Infrared Phys. Technol. 53(4), 274–279 (2010)

    Article  Google Scholar 

  13. Oswald-Tranta, B.: Motion deblurring of infrared images. In: Proceedings of IRS, pp. 783–787 (2017)

  14. Oswald-Tranta, B.: Temperature reconstruction of infrared images with motion deblurring. J. Sens. Sens. Syst. 7(1), 13–20 (2018)

    Article  Google Scholar 

  15. Wang, N., Wang, J., Zhang, Y., Sun, X.: Restoration of the infrared image blurred by motion. In: Selected Papers of the Chinese Society for Optical Engineering Conferences held October and November 2016, vol. 10255, p. 102554Q. International Society for Optics and Photonics (2017)

  16. Anacona-Mosquera, O., Arias-García, J., Muñoz, D.M., Llanos, C. H.: Efficient hardware implementation of the Richardson–Lucy Algorithm for restoring motion-blurred image on reconfigurable digital system. In: 2016 29th Symposium on Integrated Circuits and Systems Design (SBCCI), pp. 1-6. IEEE (2016)

  17. Chen, M.F., Kung, Y.C., Chou, S.J., Lo, W.S., Wang, C.K.: Real-time image acquisition and deblurring for underwater gravel extraction by smartphone. Int. J. Autom. Smart Technol. 4(1), 5–11 (2014)

    Article  Google Scholar 

  18. Dysart, T.J., Brockman, J.B., Jones, S., Bacon, F.: Embedded real-time HD video deblurring. In: 2014 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1-6. IEEE (2014)

  19. Gupta, A., Joshi, N., Zitnick, C.L., Cohen, M., Curless, B.: Single image deblurring using motion density functions. In: European conference on computer vision, pp. 171–184. Springer, Berlin, Heidelberg (2010)

  20. Hirsch, M., Schuler, C.J., Harmeling, S., Schölkopf, B.: Fast removal of non-uniform camera shake. In: 2011 International Conference on Computer Vision, pp. 463–470. IEEE (2011)

  21. Harmeling, S., Michael, H., Schölkopf, B.: Space-variant single-image blind deconvolution for removing camera shake. In: Advances in neural information processing systems, pp. 829–837 (2010)

  22. Guarnieri, M.: The rise of light—discovering its secrets [scanning our past]. Proc. IEEE 104(2), 467–473 (2016)

    Article  Google Scholar 

  23. Whyte, O., Sivic, J., Zisserman, A., Ponce, J.: Non-uniform deblurring for shaken images. Int. J. Comput. Vis. 98(2), 168–186 (2012)

    Article  MathSciNet  Google Scholar 

  24. Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Pearson Prentice Hall, Upper Saddle River (2008)

    Google Scholar 

  25. Wiener, N.: Extrapolation, Interpolation, and Smoothing of Stationary Time Series. MIT Press, Cambridge (1964)

    Google Scholar 

  26. Free FLIR Thermal Dataset for Algorithm Training. https://www.flir.com/oem/adas/adas-dataset-form/. Accessed 29 Jan 2019

  27. Hee Park, S., Levoy, M.: Gyro-based multi-image deconvolution for removing handshake blur. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3366-3373), 2014

  28. Zhang, D.L., Yang, Y., Song, Y.K., Du, G.M.: Design and implement of large dimension matrix transpose based on DDR3 SDRAM. In: Advanced Materials Research, vol. 760, pp. 1423–1428. Trans Tech Publications Ltd. (2013)

  29. Özsaraç, İ.: FPGA Implementation of real time digital video stabilization. Master Thesis, Middle East Technical University, Ankara (2011)

  30. Xilinx system generator example design: Two Dimensional FFT using Corner Turning Technique for MRI Sagittal Image Reconstruction, Vivado 2018.1: Xilinx (2018)

  31. Scorpio LW Datasheet. http://www.sofradir.com/wp-content/uploads/2013/09/sofradir-fiche-scorpio-lw.pdf. Accessed May 2019

  32. Carrato, S., Ramponi, G., Marsi, S., Jerian, M., Tenze, L.: FPGA implementation of the lucy-richardson algorithm for fast space-variant image deconvolution. In: 2015 9th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 137–142. IEEE (2015)

  33. Sims, O.: Efficient implementation of video processing algorithms on FPGA. EngD thesis, University of Glasgow (2007)

  34. Atoche, A.C., Marrufo, O.P., Castellanos, L.R.: Aggregation of parallel computing and hardware/software co-design techniques for high-performance remote sensing applications. In: 2011 IEEE International Geoscience and Remote Sensing Symposium, pp. 217–220 (2011)

Download references

Author information

Authors and Affiliations

  1. Electrical and Electronics Engineering, Hacettepe University, 06800, Ankara, Turkey

    Tugay Doner & Dincer Gokcen

  2. METU MEMS Research and Application Center, 06531, Ankara, Turkey

    Dincer Gokcen

Authors
  1. Tugay Doner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dincer Gokcen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dincer Gokcen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doner, T., Gokcen, D. FPGA-based infrared image deblurring using angular position of IR detector. Vis Comput 37, 2039–2050 (2021). https://doi.org/10.1007/s00371-020-01961-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01961-y

Keywords

Search

Navigation