Object and behavior differentiation for improved automated counts of migrating river fish using imaging sonar data
Introduction
Multibeam imaging sonars, such as the Adaptive Resolution Imaging Sonar (ARIS, Sound Metrics Corp.; www.soundmetrics.com) and the Dual-frequency IDentification SONar (DIDSON; Sound Metrics Corp.), are commonly used for monitoring migrating fish in rivers (Baumgartner et al., 2006; Lagasse et al., 2017; Martignac et al., 2014). They produce high-resolution underwater sonar video output without any underwater light (i.e. can be used overnight and in turbid waters), and allow counting and measuring of fish directly from the footage (Martignac et al., 2014; Moursund et al., 2003).
While sonars facilitate rapid monitoring of migrating fish populations in rivers, analysing the sonar footage is still a laborious task (Boswell et al., 2008; Martignac et al., 2014). The footage is typically analysed manually by users, either by using tally counters while watching the video (Faulkner and Maxwell, 2015; Hateley and Gregory, 2006), or by using a software to count and measure by drawing a line along the body of each identified fish (Cronkite et al., 2006; Lilja et al., 2010; Martignac et al., 2014). Different data sampling methods are also used to reduce the amount of data analysed (Lilja et al., 2008; Petreman et al., 2014).
Automated and semi-automated data processing tools have been used to process the sonar files, and they have the potential to reduce costs by generating fish counts with significantly fewer user interactions than manual counting (Boswell et al., 2008; Eggleston et al., 2020; Han et al., 2009; Handegard and Williams, 2008; Kang, 2011; Kupilik and Petersen, 2014; Petreman et al., 2014). Kang (2011) presents a template for semiautomated analysis using Echoview software that can be applied to new research, and cautions that parameter settings must be tailored to the characteristics of the data and research aim. The basic outline of the process is subtraction of the stationary background, tracking moving objects, filtering, and calculation of properties from the echoes (Boswell et al., 2008; Kang, 2011). Building the basic workflow in Echoview is possible without programming experience, but the same operations can be implemented in other software (Boswell et al., 2008). Similar algorithms have been used in other studies, but different approaches are used in some of the necessary steps due to differences in research aims and software (Han et al., 2009; Handegard and Williams, 2008; Kang, 2011). As an example, Handegard and Williams (2008) used MatLab (The MathWorks Inc.; www.mathworks.com) to track fish in trawls and Han et al. (2009) developed their software for counting and sizing farmed Yellowtail (Seriola quinqueradiata). The same methodology can be applied to other imaging sonars, such as the Teledyne BlueView imaging sonar (Teledyne Technologies; www.teledyne.com; Kang, 2011).
The Echoview approach has since been utilized and tested in fish migration studies (Faulkner and Maxwell, 2015; Jones and Petreman, 2015; Petreman et al., 2014). The correlation between manual and semi-automated approach varied between datasets and was highest in the footage where short range was used and the image quality was high with no visible surface noise while the auto-processing performed very poorly in the dataset with longest (40 m) range (Faulkner and Maxwell, 2015). Fish density also has an effect, and high density of large fish observed at close range was found to cause the greatest disagreement between manual and automated tracking (Handegard and Williams, 2008). Compared to different sampling methods, automation-assisted subsampling was found to be the most cost-effective means to estimate the number of migrating fish in rivers (Petreman et al., 2014). However, Peterman et al. (2014) report that the automatically produced numbers of downstream moving fish were highly inaccurate, especially because other downstream-moving objects, such as detritus and leaf-litter, were counted. Sources of error for the automated counts also include milling fish (i.e. fish remaining in the sonar field for extended periods of time), gravel bars, and mobile sediment that can cause the software to count the same fish multiple times (Petreman et al., 2014). A similar problem can arise when a fish turns directly toward or away from the imaging sonar: a change in aspect angle reduces the amount of reflected energy and produces a poor image, and the fish may be detected only intermittently (i.e. fish sometimes being lost from view), which makes it more difficult for the tracking algorithm to keep track of the fish, and subsequently resulting in the software to count the fish multiple times (Belcher et al., 2002; Kang, 2011). These issues do not occur exclusively on automated counts, and the same factors can also influence fish counts produced by a human operator. For example, excessive milling behavior can make human-generated fish counts unreliable (Pipal et al., 2010).
In this study, the automated data analysis method in Echoview (Kang, 2011) was adopted for analysing 1.1 MHz ARIS Explorer 1800 sonar data collected using the whole river width of approximately 30 m in the Little Southwest Miramichi River, New Brunswick, Canada. The aim of the study was to produce an accurate count of the number of fish passes through the sonar beam with fewer human interactions (i.e. more efficiently) than manual counting. To achieve this, the objectives were to predict if a fish was moving up- or downstream and to remove erroneous fish tracks using the Echoview summary tables; predict if a downstream moving object was a fish; and finally, to produce timely fish counts of similar accuracy and processing time to fish counts produced manually from sonar footage. We hypothesized that the computer-generated counts are similar to human-generated counts of up- and downstream moving fish; that the computer-generated and multiple human-generated fish counts are in agreement with each other; and computer-generated counts can be produced faster than human-generated counts.
Section snippets
Data collection and datasets
The data were collected as a part of an Atlantic salmon (Salmo salar) monitoring project using an imaging sonar (ARIS Explorer 1800 with a standard 14° lens), Sound Metrics Corp.; www.soundmetrics.com) at a fixed location near the mouth of the Little Southwest Miramichi River (known by local Mi’kmaq as Tuadook), New Brunswick, Canada (46.957673, -65.860730). The sonar was set aiming across the river using 1.1. MHz frequency, and the range was set to cover the whole river width of approximately
Direction calculations and thresholding
The 5th percentile of the MAD value of the real objects group was 2.43 (Fig. 3). When calculated separately by direction, the 5th percentile of the MAD was 3.98 for upstream and 1.40 for downstream fish tracks in the real objects group. The 95th percentile of the dynamic noise group was 3.98, and the 95th percentile of milling fish group was 6.42 (Fig. 3).
Because the purpose of thresholding was to remove most of the dynamic noise and milling fish tracks without removing the true fish tracks,
Upstream direction and filtering milling fish and noise
Using the MAD-value proved to be a rapid and effective way of recording the direction of the fish and filtering fish tracks caused by milling fish and dynamic noise from the data. The upstream direction calculation using the MAD-value was correct in 96 % of the fish tracks. The direction was incorrect in the instances when another object entered the field and the tracker had followed it through the field to other direction. Such instances are difficult to avoid and represent a source of error
CRediT authorship contribution statement
Jani Helminen: Conceptualization, Methodology, Resources, Formal analysis, Visualization, Writing - original draft. Tommi Linnansaari: Conceptualization, Funding acquisition, Supervision, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
We would like to thank all the technicians who helped either in the data collection or analysing process (B. Andrews, C. Cusak, C. DeCoste, C. Donovan, C. MacIntyre, D. Hanscom, H. Ralph, J. Giraudet, J. Pruneau, K. Patles, L. MacNeil L. Spencer, N. Rondeau, M. McGrath, late O. Linnansaari, T. Outrequin, T. Robichaud, T. Trask, and I. Watters). Thank you to T. Jarvis (Echoview Software Pty Ltd) who gave us a kick-start with Echoview and A.-M. Mueller (Aquacoustics) who further helped with fish
References (44)
- et al.
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
J. Chiropr. Med.
(2016) - et al.
Optimizing sampling effort within a systematic design for estimating abundant escapement of sockeye salmon (Oncorhynchus nerka) in their natal river
Fish. Res.
(2008) - et al.
Observer bias and subsampling efficiencies for estimating the number of migrating fish in rivers using Dual-frequency IDentification SONar (DIDSON)
Fish. Res.
(2014) - et al.
Detecting a nearshore fish parade using the adaptive resolution imaging sonar (ARIS): an automated procedure for data analysis
Fish. Res.
(2017) - et al.
Development of active numerating side-scan for a high-density overwintering location for endemic shortnose sturgeon (Acipenser brevirostrum) in the Saint John River, New Brunswick
Diversity
(2020) - et al.
Assessment of a Dual-frequency Identification Sonar (DIDSON) for application in fish migration studies
NSW Dep. Prim. Ind. - Fish
(2006) - et al.
Dual-frequency identification sonar (DIDSON)
Multitarget Tracking With Radar Applications
(1986)- et al.
Statistical methods for assessing agreement between two methods of clinical measurement
Lancet
(1986) - et al.
A Semiautomated approach to estimating fish size, abundance, and behavior from Dual-Frequency Identification Sonar (DIDSON) data
North Am. J. Fish. Manag.
(2008)
Realtime classification of fish in underwater sonar videos
J. R. Stat. Soc. Ser. C Appl. Stat.
Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach
Glmulti: an R package for easy automated model selection with (generalized) linear models
J. Stat. Softw.
R: a Language and Environment for Statistical Computing
Use of high-frequency imaging sonar to estimate adult sockeye salmon escapement in the Horsefly River, British Columbia
Can. Tech. Rep. Fish. Aquat. Sci.
Application of the Bland–Altman plot for interpretation of method-comparison studies: a critical investigation of its practice
Clin. Chem.
Detection and characterisation of deep-sea benthopelagic animals from an autonomous underwater vehicle with a multibeam echosounder: a proof of concept and description of data-processing methods
Deep. Res. Part I Oceanogr. Res. Pap.
Echoview Help File 9.0.19 for Echoview 9.0.322
Improved fish counting method accurately quantifies high‐density fish movement in dual‐frequency identification sonar data files from a coastal wetland environment
North Am. J. Fish. Manag.
Application of DIDSON imaging sonar at Qualark Creek on the Fraser River for enumeration of adult pacific salmon: an operational manual
Cananadian Tech. Rep. Fish. Aquat. Sci.
The feasibility of using sonar to estimate adult sockeye salmon passage in the Lower Kvichak River
Tornionjoen Nousulohien (Salmo Salar) Pituuden Mittaaminen DIDSON-luotaimella [in Finnish]
Cited by (14)
Tracking the real-time behavior of Hemimysis anomala's winter swarms using acoustic camera
2024, Journal of Great Lakes ResearchOut of the shadows: automatic fish detection from acoustic cameras
2023, Aquatic EcologyThe capacity of imaging sonar for quantifying the abundance, species richness, and size of reef fish assemblages
2023, Marine Ecology Progress SeriesCombining Imaging Sonar Counting and Underwater Camera Species Apportioning to Estimate the Number of Atlantic Salmon and Striped Bass in the Miramichi River, New Brunswick, Canada
2023, North American Journal of Fisheries Management