SeaGrassDetect: A Novel Method for the Detection of Seagrass from Unlabelled Underwater Videos
Introduction
Any study on assessment of marine ecosystems includes measuring the status of various ecological indicators which determines the state of the environment. The seagrass are important habitats in coastal ecosystems. It helps in carbon sequestration (Fourqurean et al., 2012). It also provides protection and nurseries for juvenile fishes and contributes to sediment stabilisation. Eelgrass (Zostera marina), a kind of seagrass that grows in the sea bed has been considered one of the most important ecological indicators (Barrell, 2009). The role of seagrass is often compared to that of a forest (Boudouresque et al., 2012). Seagrass among other benthic vegetation in the coastal marine ecosystem are under tremendous human pressure (Brown et al., 2011) and according to The International Union for Conservation of Nature (IUCN) Red list of Threatened species (www.iucnredlist.org), eelgrass is decreasing. If no precautions are taken, fish and sea products are estimated to reduce dramatically by the middle of 21st century (Worm et al., 2006) and all the world's oceans are said to be affected (Halpern et al., 2008). In northern temperate ecosystems, eelgrass is the dominant seagrass and in many areas constitutes the only rooted vegetation in soft sediments and constitute a key organism both in terms of ecosystem functioning and as indicator for environmental assessment and management of marine ecosystems. According to studies only 5–10% of the world's seafloor is mapped (Wright and Heyman, 2008). Due to it's vulnerability to anthropogeneic pressures, seagrass is used as an indicator of ecosystem health and state in many environmental regulations such as the European WFD (Borum et al., 2004). There is a need to map the benthic vegetation in a easy and robust way to be able to assess the full impact of human exploitation of nature as well as continuous monitoring of the seabed to evaluate the positive steps taken to replenish the seagrass.
Monitoring of seagrass is inherently difficult due to large patchiness. Traditional methods are to assess the depth limit (Krause-Jensen et al., 2005), biomass cover (Carstensen et al., 2016), etc. But these methods are time consuming, subjective and associated with large uncertainty due to diver-specific variations (Balsby et al., 2013). Other approaches include remote sensing methods like satellite images or aerial photography (Frederiksen et al., 2004). But remote sensing methods are also prone to errors when satellite images are masked by cloud cover and the turbidity of water prevent visibility of vegetation cover beneath the surface of the water. Due to this researchers have also traditionally relied on under water video transects and scuba divers as ground truth to assess the status of seagrass. Many studies such as (Chamberlain et al., 2009; Collier and Humber, 2007; Ierodiaconou et al., 2007; Legrand et al., 2010; Malthus and Karpouzli, 2009; Micallef et al., 2012; Munday et al., 2013; Riegl and Purkis, 2005; Ryan et al., 2007; Stevens et al., 2008), has used either videos, scuba divers or remote operating vehicles(ROV) to assess ground truth for different species of seagrass. (Gumusay et al., 2019) has reviewed 91 seagrass related studies which comprise 58 journal articles, 20 conference proceedings, 7 technical notes, 2 books, 3 Ph.D theses of which 52 studies have used scuba divers, video transects as method for ground truthing. Ground truthing is usually done by visual inspection of the videos by an expert, who keeps a track of the presence/absence or coverage of eelgrass along with the location(gps coordinates), time, height from the ground and other measured quantities, which is a very tedious task. Lot of person hours are spent on this exercise. It would save lot of time, if this whole process of detecting grass from videos be automatized. Also this process includes an element of expert judgement and can never be fully unbiased. Multiple experts may not agree on the same precision of estimated coverage from the video transects as no scientific standards are established. Therefore, our aim was to investigate if the assessment of traditional diver transects could be improved through automated image analysis, allowing larger areas to be covered without the time-consuming and error-prone post-analysis of video inspection by domain experts.
With the increasing popularity of machine learning in the past few years, this kind of problem, which falls in the domain of Computer Vision can now be efficiently tackled saving both time and commercial resources for the ecological researchers. Training a machine learning model requires labelled training data. (Gonzalez-Cid et al., 2017) manually labelled a small dataset(less than 200 images) to train a Support Vector Machine(SVM) and Artificial Neural Networks(ANN) to predict presence/absence of posidonia meadows from Autonomous underwater vehicles(AUV) and ROV. (Rende et al., 2015) has also used underwater vehicles to evaluate bottom coverage descriptors for assessing the good ecological status of seagrass meadows. However, to the author's knowledge, an autonomous detection and mapping of seagrass has not yet been achieved. (Massot-Campos et al., 2013) divide the images in small patches and classify each patch as seagrass or background. (Bonin-Font et al., 2017; Burguera et al., 2016; Gonzalez-Cid et al., 2017) also follow a patch classification approach to segment images. (Bonin-Font et al., 2017) also use a pixel refinement method as post processing step to get better results than pure patch based classification. Due to the lack of availability of standard labelled seagrass images in public domain, there have been limited use of machine learning models on seagrass detection tasks. (Reus et al., 2018) in 2018 published the first publicly available pixel annotated seagrass dataset captured from an AUV at different depths and trained a Convolutional Neural Network(CNN) model to segment seagrass images. Using the dataset of (Reus et al., 2018; Weidmann et al., 2019) trained different deep learning architectures, including the DeepLabv3Plus network and compared their results. In 2018 (Martin-Abadal et al., 2018) came out with a method which use Visual Geometric Group network(VGGNet) as the encoder and a Fully Convolutional Network(FCN-8) as decoder plus skip connections to segment seagrass images collected from an Autonomous underwater vehicles(AUV). There exists concern regarding the annotation of ground-truth data of (Reus et al., 2018), which was done relatively roughly with certain inaccuracies using polygons (Weidmann et al., 2019). In reality, the seagrass areas are defined by the fine leaves of the plants. This also raises doubts and uncertainity about the correctness of the labelling process as inaccurate labelling have significant negative effect on the metrics.
In this paper, we propose a novel method using an edge detection based approach as a feature extractor to discriminate between regions with eelgrass and no eelgrass without any need of labelled seagrass images. This will bypass the need of domain expert's requirement to estimate coverage or status of seagrass manually. Line detection is a classic technique that has been widely used in many areas like agriculture for task related to detection of crop rows (Ji and Qi, 2011), Optical Character Recognition(OCR) for detecting characters (Chaudhuri and Pal, 1998) and Biometrics (Miura et al., 2004). We propose two methods, one which involves fusion of predictions obtained from two individual features to distinguish eelgrass frames from no eelgrass frames and another using a Gaussian mixture model to cluster video frames of eelgrass from no eelgrass. The methods also give a way to quantify the subjectivity involved in eelgrass estimation through the threshold. We extended this model further from presence/absence modelling to a proxy for coverage estimation in a given area and show that it follows the domain expert's estimation very well. It even detects and rectify rare human error from the domain expert, which makes it truly a robust tool for eelgrass detection. This method can also be extended to detect other species of seagrass or vegetation that has sharp edges (Posidonia oceania meadows).
Section snippets
Field site
The data used for this research is obtained from Roskilde Fjord in Denmark. Roskilde Fjord is a 40-km long and narrow micro-tidal estuary forming four major basins. It is 10 km wide and has a surface area of 122 km2, a volume of (360 × 106 m3) and a mean depth of 3 m (Flindt et al., 1997). The water temperature vary from 0 °C in the winter to 22 °C in the summer. There is a strong hydraulic effect from the outer boundary where strong western wind events are able to create water levels of up to
Comparison of feature fusion and probabilistic clustering approaches
Fig. 6 shows the contours of different decision boundaries from the two different approaches on a scatter plot. Three linear decision boundaries (corresponding to the feature fusion approach) and one non-linear decision boundary (corresponding to GMM) can be seen. It is interesting to see that the decision boundaries from both these approaches are not so far away from each other.
The GMM has slightly lower threshold (using the 50% cut point) than the combination strategy due to which it will
Previous studies and necessity of automation
There exist many methods of collecting ground truth of eelgrass coverage. These include but not limited to videos, scuba diver, grab, aerial image, buoy and vegetation sampler. (Gumusay et al., 2019) showed that underwater video and scuba diver are the most popular choice of collecting ground truth. Many previous studies that has used underwater videos and scuba diver for ground truth estimation of seagrass has been done manually by a domain expert. We did not find many papers where the ground
Future scope
The present/absent status obtained with this method can be used as a ground truth and extrapolated on unknown regions of satellite images where no video transects are available. On a satellite image, the transect location would appear as the few labelled pixels which would provide information to extrapolate or propagate the label information to other areas. Semi-supervised learning methods are useful tools to solve this problem and will be the main part of our future work.
Another interesting
Conclusion
Here we presented two methods which detect eelgrass from underwater video transects. We showed how our methods are robust to noise introduced during the recording of the transects and how these methods can be extended nicely to be used as a proxy to find the percentage coverage in a given area which matches very well with a domain expert's estimation of coverage. A big advantage of our methods is that, it reduces the cost and complexity of conducting this kind of study anywhere as it does not
Funding
This research is part of the SeaStatus Project (http://seastatus.dhigroup.com/) which is funded by the Innovation Fund Denmark (IFD), grant number: 6154-00005B. The data was provided by DHI (Danish Hydraulics Institute) and the research was carried out at the Department of Applied Mathematics and Computer Science, Technical University of Denmark.
Acknowledgement
We are thankful to Jacob Carstensen (Professor, Department of Bioscience, Aarhus University) and Karen Timmermann (Senior scientist, Department of Bioscience, Aarhus University) for their valuable input and ideas which shaped this paper into its current form. We are also thankful to Lars Boye Hansen (Head of Projects, DHI GRAS) and Mikkel Lydholm Rasmussen (Remote sensing specialist, DHI GRAS) for the insightful discussions about the data and Emil Guddal Larsen for telling us about the way
References (85)
- et al.
Benthic habitat mapping: a review of progress towards improved understanding of the spatial ecology of the seafloor using acoustic techniques
Estuar. Coast. Shelf Sci.
(2011) - et al.
A complete printed bangla ocr system
Pattern Recogn.
(1998) - et al.
Time-lapse side-scan sonar imaging of bleached coral reefs: a case study from the Seychelles
Remote Sens. Environ.
(2007) - et al.
Relationships between multibeam backscatter, sediment grain size and posidonia oceanica seagrass distribution
Cont. Shelf Res.
(2010) - et al.
Fast and easy method for seagrass monitoring: application of acoustic telemetry to precision mapping of posidonia oceanica beds
Mar. Pollut. Bull.
(2011) - et al.
Description of the three shallow estuaries: Mondego river (Portugal), Roskilde fjord (Denmark) and the lagoon of Venice (Italy)
Ecol. Model.
(1997) - et al.
Spatial and temporal variation in eelgrass (zostera marina) landscapes: influence of physical setting
Aquat. Bot.
(2004) - et al.
Crop-row detection algorithm based on random hough transformation
Math. Comput. Model.
(2011) - et al.
Use of multi-beam sonar to map seagrass beds in otsuchi bay on the sanriku coast of Japan
Aquat. Living Resour.
(2003) - et al.
Use of a high-resolution profiling sonar and a towed video camera to map a zostera marina bed, solent, Uk
Estuar. Coast. Shelf Sci.
(2009)