SeaGrassDetect: A Novel Method for the Detection of Seagrass from Unlabelled Underwater Videos

https://doi.org/10.1016/j.ecoinf.2020.101083Get rights and content

Highlights

  • Seagrass can be detected automatically from video transects without domain expert.

  • Can detect and rectify rare mistakes made by domain experts.

  • Presence/Absence status can be extended further for robust coverage estimation.

  • Our method does not require pixel-level annotation of seagrass images.

  • Easy to use and works very well in noisy environment.

Abstract

Benthic vegetation is arguably one of the most important indicators of the state of marine environment. Assessment of the status of eelgrass (Zostera marina) is commonly done using various remote sensing methods such as aerial photography or satellite images. These methods often fail to capture the true scenario beneath the surface of the water if the water is turbid or the satellite image is masked by cloud cover which makes it impossible to see beneath them. As a second line of defense, researchers have used under water videos (obtained either with scuba,snorkel or other visual observations) to assess the ground truth. Lots of man-hours are spent browsing through many hours of video data manually by an expert and assessing the status (presence/absence) which is a very common practice. Here we propose two methods for detection of eelgrass (presence/absence) from under water videos obtained from Roskilde Fjord in Denmark. We extend these methods to show that it can be used as a proxy to estimate coverage of eelgrass in a given area which match well with an expert's estimation. The benefit of using this method is that it is objective, less biased, cost efficient, robust to noisy environment, does not require pixel-level annotated ground truth images and can be used on existing video transects. This method can also detect rare errors from domain expert's visual estimation.

Introduction

Any study on assessment of marine ecosystems includes measuring the status of various ecological indicators which determines the state of the environment. The seagrass are important habitats in coastal ecosystems. It helps in carbon sequestration (Fourqurean et al., 2012). It also provides protection and nurseries for juvenile fishes and contributes to sediment stabilisation. Eelgrass (Zostera marina), a kind of seagrass that grows in the sea bed has been considered one of the most important ecological indicators (Barrell, 2009). The role of seagrass is often compared to that of a forest (Boudouresque et al., 2012). Seagrass among other benthic vegetation in the coastal marine ecosystem are under tremendous human pressure (Brown et al., 2011) and according to The International Union for Conservation of Nature (IUCN) Red list of Threatened species (www.iucnredlist.org), eelgrass is decreasing. If no precautions are taken, fish and sea products are estimated to reduce dramatically by the middle of 21st century (Worm et al., 2006) and all the world's oceans are said to be affected (Halpern et al., 2008). In northern temperate ecosystems, eelgrass is the dominant seagrass and in many areas constitutes the only rooted vegetation in soft sediments and constitute a key organism both in terms of ecosystem functioning and as indicator for environmental assessment and management of marine ecosystems. According to studies only 5–10% of the world's seafloor is mapped (Wright and Heyman, 2008). Due to it's vulnerability to anthropogeneic pressures, seagrass is used as an indicator of ecosystem health and state in many environmental regulations such as the European WFD (Borum et al., 2004). There is a need to map the benthic vegetation in a easy and robust way to be able to assess the full impact of human exploitation of nature as well as continuous monitoring of the seabed to evaluate the positive steps taken to replenish the seagrass.

Monitoring of seagrass is inherently difficult due to large patchiness. Traditional methods are to assess the depth limit (Krause-Jensen et al., 2005), biomass cover (Carstensen et al., 2016), etc. But these methods are time consuming, subjective and associated with large uncertainty due to diver-specific variations (Balsby et al., 2013). Other approaches include remote sensing methods like satellite images or aerial photography (Frederiksen et al., 2004). But remote sensing methods are also prone to errors when satellite images are masked by cloud cover and the turbidity of water prevent visibility of vegetation cover beneath the surface of the water. Due to this researchers have also traditionally relied on under water video transects and scuba divers as ground truth to assess the status of seagrass. Many studies such as (Chamberlain et al., 2009; Collier and Humber, 2007; Ierodiaconou et al., 2007; Legrand et al., 2010; Malthus and Karpouzli, 2009; Micallef et al., 2012; Munday et al., 2013; Riegl and Purkis, 2005; Ryan et al., 2007; Stevens et al., 2008), has used either videos, scuba divers or remote operating vehicles(ROV) to assess ground truth for different species of seagrass. (Gumusay et al., 2019) has reviewed 91 seagrass related studies which comprise 58 journal articles, 20 conference proceedings, 7 technical notes, 2 books, 3 Ph.D theses of which 52 studies have used scuba divers, video transects as method for ground truthing. Ground truthing is usually done by visual inspection of the videos by an expert, who keeps a track of the presence/absence or coverage of eelgrass along with the location(gps coordinates), time, height from the ground and other measured quantities, which is a very tedious task. Lot of person hours are spent on this exercise. It would save lot of time, if this whole process of detecting grass from videos be automatized. Also this process includes an element of expert judgement and can never be fully unbiased. Multiple experts may not agree on the same precision of estimated coverage from the video transects as no scientific standards are established. Therefore, our aim was to investigate if the assessment of traditional diver transects could be improved through automated image analysis, allowing larger areas to be covered without the time-consuming and error-prone post-analysis of video inspection by domain experts.

With the increasing popularity of machine learning in the past few years, this kind of problem, which falls in the domain of Computer Vision can now be efficiently tackled saving both time and commercial resources for the ecological researchers. Training a machine learning model requires labelled training data. (Gonzalez-Cid et al., 2017) manually labelled a small dataset(less than 200 images) to train a Support Vector Machine(SVM) and Artificial Neural Networks(ANN) to predict presence/absence of posidonia meadows from Autonomous underwater vehicles(AUV) and ROV. (Rende et al., 2015) has also used underwater vehicles to evaluate bottom coverage descriptors for assessing the good ecological status of seagrass meadows. However, to the author's knowledge, an autonomous detection and mapping of seagrass has not yet been achieved. (Massot-Campos et al., 2013) divide the images in small patches and classify each patch as seagrass or background. (Bonin-Font et al., 2017; Burguera et al., 2016; Gonzalez-Cid et al., 2017) also follow a patch classification approach to segment images. (Bonin-Font et al., 2017) also use a pixel refinement method as post processing step to get better results than pure patch based classification. Due to the lack of availability of standard labelled seagrass images in public domain, there have been limited use of machine learning models on seagrass detection tasks. (Reus et al., 2018) in 2018 published the first publicly available pixel annotated seagrass dataset captured from an AUV at different depths and trained a Convolutional Neural Network(CNN) model to segment seagrass images. Using the dataset of (Reus et al., 2018; Weidmann et al., 2019) trained different deep learning architectures, including the DeepLabv3Plus network and compared their results. In 2018 (Martin-Abadal et al., 2018) came out with a method which use Visual Geometric Group network(VGGNet) as the encoder and a Fully Convolutional Network(FCN-8) as decoder plus skip connections to segment seagrass images collected from an Autonomous underwater vehicles(AUV). There exists concern regarding the annotation of ground-truth data of (Reus et al., 2018), which was done relatively roughly with certain inaccuracies using polygons (Weidmann et al., 2019). In reality, the seagrass areas are defined by the fine leaves of the plants. This also raises doubts and uncertainity about the correctness of the labelling process as inaccurate labelling have significant negative effect on the metrics.

In this paper, we propose a novel method using an edge detection based approach as a feature extractor to discriminate between regions with eelgrass and no eelgrass without any need of labelled seagrass images. This will bypass the need of domain expert's requirement to estimate coverage or status of seagrass manually. Line detection is a classic technique that has been widely used in many areas like agriculture for task related to detection of crop rows (Ji and Qi, 2011), Optical Character Recognition(OCR) for detecting characters (Chaudhuri and Pal, 1998) and Biometrics (Miura et al., 2004). We propose two methods, one which involves fusion of predictions obtained from two individual features to distinguish eelgrass frames from no eelgrass frames and another using a Gaussian mixture model to cluster video frames of eelgrass from no eelgrass. The methods also give a way to quantify the subjectivity involved in eelgrass estimation through the threshold. We extended this model further from presence/absence modelling to a proxy for coverage estimation in a given area and show that it follows the domain expert's estimation very well. It even detects and rectify rare human error from the domain expert, which makes it truly a robust tool for eelgrass detection. This method can also be extended to detect other species of seagrass or vegetation that has sharp edges (Posidonia oceania meadows).

Section snippets

Field site

The data used for this research is obtained from Roskilde Fjord in Denmark. Roskilde Fjord is a 40-km long and narrow micro-tidal estuary forming four major basins. It is 10 km wide and has a surface area of 122 km2, a volume of (360 × 106 m3) and a mean depth of 3 m (Flindt et al., 1997). The water temperature vary from 0 °C in the winter to 22 °C in the summer. There is a strong hydraulic effect from the outer boundary where strong western wind events are able to create water levels of up to

Comparison of feature fusion and probabilistic clustering approaches

Fig. 6 shows the contours of different decision boundaries from the two different approaches on a scatter plot. Three linear decision boundaries (corresponding to the feature fusion approach) and one non-linear decision boundary (corresponding to GMM) can be seen. It is interesting to see that the decision boundaries from both these approaches are not so far away from each other.

The GMM has slightly lower threshold (using the 50% cut point) than the combination strategy due to which it will

Previous studies and necessity of automation

There exist many methods of collecting ground truth of eelgrass coverage. These include but not limited to videos, scuba diver, grab, aerial image, buoy and vegetation sampler. (Gumusay et al., 2019) showed that underwater video and scuba diver are the most popular choice of collecting ground truth. Many previous studies that has used underwater videos and scuba diver for ground truth estimation of seagrass has been done manually by a domain expert. We did not find many papers where the ground

Future scope

The present/absent status obtained with this method can be used as a ground truth and extrapolated on unknown regions of satellite images where no video transects are available. On a satellite image, the transect location would appear as the few labelled pixels which would provide information to extrapolate or propagate the label information to other areas. Semi-supervised learning methods are useful tools to solve this problem and will be the main part of our future work.

Another interesting

Conclusion

Here we presented two methods which detect eelgrass from underwater video transects. We showed how our methods are robust to noise introduced during the recording of the transects and how these methods can be extended nicely to be used as a proxy to find the percentage coverage in a given area which matches very well with a domain expert's estimation of coverage. A big advantage of our methods is that, it reduces the cost and complexity of conducting this kind of study anywhere as it does not

Funding

This research is part of the SeaStatus Project (http://seastatus.dhigroup.com/) which is funded by the Innovation Fund Denmark (IFD), grant number: 6154-00005B. The data was provided by DHI (Danish Hydraulics Institute) and the research was carried out at the Department of Applied Mathematics and Computer Science, Technical University of Denmark.

Acknowledgement

We are thankful to Jacob Carstensen (Professor, Department of Bioscience, Aarhus University) and Karen Timmermann (Senior scientist, Department of Bioscience, Aarhus University) for their valuable input and ideas which shaped this paper into its current form. We are also thankful to Lars Boye Hansen (Head of Projects, DHI GRAS) and Mikkel Lydholm Rasmussen (Remote sensing specialist, DHI GRAS) for the insightful discussions about the data and Emil Guddal Larsen for telling us about the way

References (85)

  • A. Leriche et al.

    Spatial, temporal and structural variations of a posidonia oceanica seagrass meadow facing human activities

    Aquat. Bot.

    (2006)
  • A. Micallef et al.

    A multi-method approach for benthic habitat mapping of shallow coastal areas with high-resolution multibeam data

    Cont. Shelf Res.

    (2012)
  • V. Pasqualini et al.

    Mapping ofposidonia oceanicausing aerial photographs and side scan sonar: application off the island of Corsica (France)

    Estuar. Coast. Shelf Sci.

    (1998)
  • M. Paul et al.

    An acoustic method for the remote measurement of seagrass metrics

    Estuar. Coast. Shelf Sci.

    (2011)
  • L. PIAZZI et al.

    Mapping of posidonia oceanica beds around elba island (western mediterranean) with integration of direct and indirect methods

    Oceanol. Acta

    (2000)
  • B. Rasmussen et al.

    Consistent estimates for the residence time of micro-tidal estuaries

    Estuar. Coast. Shelf Sci.

    (2002)
  • L. Reshitnyk et al.

    Evaluation of worldview-2 and acoustic remote sensing for mapping benthic habitats in temperate coastal pacific waters

    Remote Sens. Environ.

    (2014)
  • D.A. Reynolds et al.

    Speaker verification using adapted gaussian mixture models

    Digital Signal Process.

    (2000)
  • B.M. Riegl et al.

    Detection of shallow subtidal corals from ikonos satellite and qtc view (50, 200 khz) single-beam sonar data (arabian gulf; Dubai, Uae)

    Remote Sens. Environ.

    (2005)
  • D.A. Ryan et al.

    The influence of geomorphology and sedimentary processes on shallow-water benthic habitat distribution: Esperance bay, Western Australia

    Estuar. Coast. Shelf Sci.

    (2007)
  • N. Sánchez-Carnero et al.

    Using vertical sidescan sonar as a tool for seagrass cartography

    Estuar. Coast. Shelf Sci.

    (2012)
  • H. Van Rein et al.

    An evaluation of acoustic seabed classification techniques for marine biotope monitoring over broad-scales (> 1 km2) and meso-scales (10 m2–1 km2)

    Estuar. Coast. Shelf Sci.

    (2011)
  • J.D. Warren et al.

    Use of a 600-khz acoustic doppler current profiler to measure estuarine bottom type, relative abundance of submerged aquatic vegetation, and eelgrass canopy height

    Estuar. Coast. Shelf Sci.

    (2007)
  • G. Ardizzone et al.

    Long-term change in the structure of a posidonia oceanica landscape and its reference for a monitoring plan

    Mar. Ecol.

    (2006)
  • T.L. Bailey et al.

    Fitting a Mixture Model by Expectation Maximization to Discover Motifs in Bipolymers

    (1994)
  • T.J. Balsby et al.

    Sources of uncertainty in estimation of eelgrass depth limits

    Hydrobiologia

    (2013)
  • J. Barrell

    Does Eelgrass (Zostera marina) Meet the Criteria as an Ecologically Significant Species?

    (2009)
  • J.P. Barrell

    Quantification and Spatial Analysis of Seagrass Landscape Structure Through the Application of Aerial and Acoustic Remote Sensing

    (2016)
  • J. Barrell et al.

    Detecting hot and cold spots in a seagrass landscape using local indicators of spatial association

    Landscape Ecology

    (2013)
  • M. Basso et al.

    A uav guidance system using crop row detection and line follower algorithms

    J. Intell. Robot. Syst.

    (2019)
  • F. Bonin-Font et al.

    Visual discrimination and large area mapping of posidonia oceanica using a lightweight auv

    IEEE Access

    (2017)
  • J. Borum et al.

    European Seagrasses: An Introduction to Monitoring and Management

    (2004)
  • C.-F. Boudouresque et al.

    Protection and Conservation of Posidonia oceanica Meadows

    (2012)
  • A. Burguera et al.

    Towards automatic visual sea grass detection in underwater areas of ecological interest

  • J. Carstensen et al.

    Biomass-cover relationship for eelgrass meadows

    Estuar. Coasts

    (2016)
  • R.H. Chamberlain et al.

    Comparison of manual and hydroacoustic measurement of seagrass distribution in the caloosahatchee estuary, Florida

    Florida Sci.

    (2009)
  • P. Colantoni et al.

    Patterns of posidonia oceanica (l.) delile beds around the island of ischia (gulf of Naples) and in adjacent waters

    Mar. Ecol.

    (1982)
  • K. Collins et al.

    The impacts of anchoring and mooring in seagrass, studland bay, dorset, Uk

    Underw. Technol.

    (2010)
  • A.P. Dempster et al.

    Maximum likelihood from incomplete data via the em algorithm

    J. R. Stat. Soc. Ser. B Methodol.

    (1977)
  • G. Di Maida et al.

    Discriminating between posidonia oceanica meadows and sand substratum using multibeam sonar

    ICES J. Mar. Sci.

    (2010)
  • T. Domico et al.

    Assessment of digital sonar technology to map eelgrass (zostera marina) in the San Juan islands

    Puget Sound BioSurvey

    (2001)
  • R. Ferretti et al.

    Machine learning methods for acoustic-based automatic posidonia meadows detection by means of unmanned marine vehicles

  • Cited by (0)

    View full text