Skip to main content
SpringerLink
Log in

Learning from Synthetic Data: Enhancing Refraction Correction Accuracy for Airborne Image-Based Bathymetric Mapping of Shallow Coastal Waters

  • Original Article
  • Published:
PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science Aims and scope Submit manuscript

Abstract

The increasing need for accurate bathymetric mapping is essential for a plethora of offshore activities. Even though aerial image datasets through Structure from Motion (SfM) and Multi-View Stereo (MVS) techniques can provide a low-cost alternative compared to LiDAR and SONAR, offering additionally, important visual information, water refraction poses significant obstacles in delivering accurate bathymetry. In this article, the generation of manned and unmanned airborne synthetic datasets of dry and water covered areas is presented. These data are used to train models for correcting the geometric effects of refraction on real-world image-based point clouds and aerial images. Based on a thorough evaluation, important improvements are presented, indicating the increased accuracy and the reduced noise in the point clouds of the derived bathymetric products, meeting also the International Hydrographic Organization’s (IHO) standards.

Zusammenfassung

Zusammenfassung Lernen aus synthetischen Daten: Verbesserung der Genauigkeit der Refraktionskorrektur für flugzeuggestützte bathymetrische Kartierung von flachen Küstengewässern. Für eine Vielzahl von Offshore-Aktivitäten besteht ein zunehmender Bedarf an genauen bathymetrischen Kartierungen. Obwohl Luftbilddatensätze durch Structure-from-Motion (SfM) und Multi-View-Stereo (MVS)-Techniken eine kostengünstige Alternative zu LiDAR und SONAR darstellen können und zusätzlich wichtige visuelle Informationen bieten, stellt die Refraktion an der Wasseroberfläche ein erhebliches Hindernis für die Bereitstellung präziser Bathymetrie dar. In diesem Artikel wird die Erzeugung von synthetischen Datensätzen von trockenen und wasserbedeckten Gebieten aus bemannter und unbemannter Erfassung vorgestellt. Diese Daten werden verwendet, um Modelle zur Korrektur der Strahlbrechung bei photogrammetrisch bestimmten Punktwolken aus realen Luftbildern zu trainieren. Basierend auf einer gründlichen Auswertung werden maßgebliche Verbesserungen vorgestellt, die eine erhöhte Genauigkeit und ein reduziertes Rauschen der abgeleiteten bathymetrischen Punktwolken belegen, die auch die Genauigkeitsstandards der International Hydrographic Organization (IHO) erfüllen.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Agrafiotis P (2020) Image-based bathymetry mapping for shallow waters. PhD thesis, National Technical University of Athens

  • Agrafiotis P, Georgopoulos A (2015) Camera constant in the case of two media photogrammetry. ISPRS Int Arch Photogramm Remote Sens Spatial Inf Sci XL–5/W5:1–6. https://doi.org/10.5194/isprsarchives-XL-5-W5-1-2015

    Article  Google Scholar 

  • Agrafiotis P, Skarlatos D, Forbes T, Poullis C, Skamantzari M, Georgopoulos A (2018) Underwater photogrammetry in very shallow waters: main challenges and caustics effect removal. ISPRS Int Arch Photogramm Remote Sens Spatial Inf Sci XLII–2:15–22. https://doi.org/10.5194/isprs-archives-XLII-2-15-2018

    Article  Google Scholar 

  • Agrafiotis P, Skarlatos D, Georgopoulos A, Karantzalos K (2019a) Depthlearn: Learning to correct the refraction on point clouds derived from aerial imagery for accurate dense shallow water bathymetry based on SVMs-fusion with LiDAR point clouds. Remote Sens 11(19):2225

    Article  Google Scholar 

  • Agrafiotis P, Skarlatos D, Georgopoulos A, Karantzalos K (2019b) Shallow water bathymetry mapping from UAV imagery based on machine learning. ISPRS Int Arch Photogramm Remote Sens Spatial Inf Sci XLII–2/W10:9–16. https://doi.org/10.5194/isprs-archives-XLII-2-W10-9-2019

    Article  Google Scholar 

  • Agrafiotis P, Karantzalos K, Georgopoulos A, Skarlatos D (2020) Correcting image refraction: Towards accurate aerial image-based bathymetry mapping in shallow waters. Remote Sens 12(2):322

    Article  Google Scholar 

  • Awad M, Khanna R (2015) Support vector regression. Efficient learning machines. Springer, New York, pp 67–80

    Chapter  Google Scholar 

  • Barbosa IB, Cristani M, Caputo B, Rognhaugen A, Theoharis T (2018) Looking beyond appearances: synthetic training data for deep CNNs in re-identification. Comput Vis Image Underst 167:50–62

    Article  Google Scholar 

  • Bianco S, Ciocca G, Marelli D (2018) Evaluating the performance of Structure from Motion pipelines. J Imaging 4(8):98

    Article  Google Scholar 

  • Bishop CM (2006) Pattern recognition and machine learning. Springer, New York

    Google Scholar 

  • Butler J, Lane S, Chandler J, Porfiri E (2002) Through-water close range digital photogrammetry in flume and field environments. Photogramm Rec 17(99):419–439

    Article  Google Scholar 

  • Cao B, Fang Y, Jiang Z, Gao L, Hu H (2019) Shallow water bathymetry from Worldview-2 stereo imagery using two-media photogrammetry. Eur J Remote Sens 52(1):506–521

    Article  Google Scholar 

  • Cao B, Deng R, Zhu S (2020) Universal algorithm for water depth refraction correction in through-water stereo remote sensing. Int J Appl Earth Obs Geoinf 91:102108

    Article  Google Scholar 

  • Chirayath V, Li A (2019) Next-Generation Optical Sensing Technologies for Exploring Ocean Worlds—NASA FluidCam, MiDAR, and NeMO-Net. Front Mar Sci 6:521

    Article  Google Scholar 

  • Dietrich JT (2017) Bathymetric structure-from-motion: extracting shallow stream bathymetry from multi-view stereo photogrammetry. Earth Surf Proc Land 42(2):355–364

    Article  Google Scholar 

  • Fryer JG (1983) Photogrammetry through shallow water. Aust J Geodesy Photogramm Surv 38:25–38

    Google Scholar 

  • Georgopoulos A, Agrafiotis P (2012) Documentation of a submerged monument using improved two media techniques. In: 2012 18th international conference on virtual systems and multimedia, IEEE, pp 173–180

  • Hinterstoisser S, Lepetit V, Wohlhart P, Konolige K (2018) On pre-trained image features and synthetic images for deep learning. In: Proceedings of the European conference on computer vision (ECCV)

  • IHO (2019) Zones of confidence. IHO

    Google Scholar 

  • IHO (2020) Iho s-44 edition 6.0.0. IHO

    Google Scholar 

  • Kahmen O, Rofallski R, Conen N, Luhmann T (2019) On scale definition within calibration of multi-camera systems in multimedia photogrammetry. Int Arch Photogramm Remote Sens Spatial Inf Sci XLII-2/W10, 93–100

  • Kahmen O, Rofallski R, Luhmann T (2020) Impact of stereo camera calibration to object accuracy in multimedia photogrammetry. Remote Sens 12(12):2057

    Article  Google Scholar 

  • Li C, Zhang J, Guo B (2011) New method of landslide monitoring based on close-range photogrammetry. Jisuanji Gongcheng yu Yingyong (Comput Eng Appl) 47(3):6–8

    Google Scholar 

  • Luhmann T (2016) Learning photogrammetry with interactive software tool PhoX. Int Arch Photogramm Remote Sens Spatial Inf Sci 41:39

    Article  Google Scholar 

  • Maas HG (2015) On the accuracy potential in underwater/multimedia photogrammetry. Sensors 15(8):18140–18152

    Article  Google Scholar 

  • Mandlburger G (2018) A case study on through-water dense image matching. ISPRS Int Arch Photogramm Remote Sens Spatial Inf Sci XLII–2:659–666. https://doi.org/10.5194/isprs-archives-XLII-2-659-2018

    Article  Google Scholar 

  • Mandlburger G (2019) Through-water dense image matching for shallow water bathymetry. Photogramm Eng Remote Sens 85(6):445–455

    Article  Google Scholar 

  • Mandlburger G, Pfennigbauer M, Schwarz R, Flöry S, Nussbaumer L (2020) Concept and performance evaluation of a novel UAV-borne topo-bathymetric LiDAR sensor. Remote Sens 12(6):986

    Article  Google Scholar 

  • Menna F, Nocerino E, Remondino F (2018) Photogrammetric modelling of submerged structures: influence of underwater environment and lens ports on three-dimensional (3d) measurements. Latest developments in reality-based 3D surveying and modelling MDPI Basel, Switzerland pp 279–303

  • Mulsow C (2010) A flexible multi-media bundle approach. Int Arch Photogramm Remote Sens Spat Inf Sci 38:472–477

    Google Scholar 

  • Mulsow C, Mandlburger G, Maas HG (2020) Comparison of subaquatic digital elevation models from airborne laser scanning and imagery. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci V–2–2020:671–677. https://doi.org/10.5194/isprs-annals-V-2-2020-671-2020

    Article  Google Scholar 

  • Nesbit PR, Hugenholtz CH (2019) Enhancing UAV-SfM 3D model accuracy in high-relief landscapes by incorporating oblique images. Remote Sens 11(3):239

    Article  Google Scholar 

  • Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    Google Scholar 

  • Peng X, Sun B, Ali K, Saenko K (2015) Learning deep object detectors from 3d models. In: Proceedings of the IEEE international conference on computer vision, pp 1278–1286

  • Richardson E, Sela M, Kimmel R (2016) 3d face reconstruction by learning from synthetic data. In: 2016 fourth international conference on 3D vision (3DV), IEEE, pp 460–469

  • Ros G, Sellart L, Materzynska J, Vazquez D, Lopez AM (2016) The synthia dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3234–3243

  • Saleh FS, Aliakbarian MS, Salzmann M, Petersson L, Alvarez JM (2018) Effective use of synthetic data for urban scene semantic segmentation. European conference on computer vision. Springer, New York, pp 86–103

    Google Scholar 

  • Sankaranarayanan S, Balaji Y, Jain A, Nam Lim S, Chellappa R (2018) Learning from synthetic data: addressing domain shift for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3752–3761

  • Shan J (1994) Relative orientation for two-media photogrammetry. Photogramm Rec 14(84):993–999

    Article  Google Scholar 

  • Shrivastava A, Pfister T, Tuzel O, Susskind J, Wang W, Webb R (2017) Learning from simulated and unsupervised images through adversarial training. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2107–2116

  • Skarlatos D, Agrafiotis P (2018) A novel iterative water refraction correction algorithm for use in structure from motion photogrammetric pipeline. J Mar Sci Eng 6(3):77

    Article  Google Scholar 

  • Skarlatos D, Georgopoulos A (2006) The method of two overlapping orthoimages for checking the produced DTM. Technical report

  • Smith MW, Vericat D (2015) From experimental plots to experimental landscapes: topography, erosion and deposition in sub-humid badlands from structure-from-motion photogrammetry. Earth Surf Proc Land 40(12):1656–1671

    Article  Google Scholar 

  • Sun Y, Sun H, Yan L, Fan S, Chen R (2016) Rba: Reduced bundle adjustment for oblique aerial photogrammetry. ISPRS J Photogramm Remote Sens 121:128–142

    Article  Google Scholar 

  • Vapnik VN, Chervonenkis AY (2015) On the uniform convergence of relative frequencies of events to their probabilities. Measures of complexity. Springer, New York, pp 11–30

    Chapter  Google Scholar 

  • Wang Z (1990) Principles of photogrammetry: (with remote sensing). Press of Wuhan Technical University of Surveying and Mapping, Wuhan

    Google Scholar 

  • Wimmer M (2016) Comparison of active and passive optical methods for mapping river bathymetry. TU Wien, Vienna

    Google Scholar 

  • Woodget A, Carbonneau P, Visser F, Maddock IP (2015) Quantifying submerged fluvial topography using hyperspatial resolution uas imagery and structure from motion photogrammetry. Earth Surf Proc Land 40(1):47–64

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to the reviewers for their valuable comments and suggestions. Also, we would like to acknowledge the Dep. of Land and Surveys of Cyprus for providing the LiDAR reference data, the Cyprus Dep. of Antiquities for permitting the flight over the Amathouda site and commissioning the flight over Agia Napa, and HFF for partially funding the acquisition of Cyprus’ datasets.

Author information

Authors and Affiliations

  1. School or Rural and Surveying Engineering, National Technical University of Athens, 15780, Athens, Greece

    Panagiotis Agrafiotis, Konstantinos Karantzalos & Andreas Georgopoulos

  2. Civil Engineering and Geomatics Department, Cyprus University of Technology, 3036, Limassol, Cyprus

    Dimitrios Skarlatos

Authors
  1. Panagiotis Agrafiotis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantinos Karantzalos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Georgopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dimitrios Skarlatos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Panagiotis Agrafiotis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agrafiotis, P., Karantzalos, K., Georgopoulos, A. et al. Learning from Synthetic Data: Enhancing Refraction Correction Accuracy for Airborne Image-Based Bathymetric Mapping of Shallow Coastal Waters. PFG 89, 91–109 (2021). https://doi.org/10.1007/s41064-021-00144-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41064-021-00144-1

Keywords

Search

Navigation