Collating stock assessment packages to improve stock assessments
Introduction
Quantitative stock assessments based on fitting population dynamics models to monitoring data are integral to modern fisheries management. They are used to assess stock status relative to biological reference points, assess whether stocks are subject to overfishing or are in an overfished state, form the basis for application of harvest control rules, and act as operating models to evaluate candidate management strategies using management strategy evaluation (Punt et al., 2020).
The most appropriate method of assessment depends on the data available, whether the model on which the assessment method is based adequately captures the population and fishery dynamics, and the methods for estimating the parameters of the model and quantifying uncertainty. ‘Integrated approaches’ (e.g., Fournier and Archibald, 1982; Maunder and Punt, 2013) are the preferred assessment paradigm when data are available on removals, there is an index of relative abundance, and data are available on size- and/or age-composition from fishery-dependent and -independent monitoring, thereby allowing exploration of many aspects of the populations dynamics and how the data pertain to the modelled population. However, there are often insufficient data to apply integrated approaches, resulting in use of alternative methods such as surplus production models, catch curve methods, and catch- or length-only methods.
Historically, assessments were conducted using bespoke methods implemented using user-developed software that were designed for specific fisheries and tailored to their particular needs. However, there is now a trend towards the use of software packages that implement assessment methods (Dichmont et al., 2016a). Assessment packages are intended to be user-friendly applications that enable users to apply established assessment methods using their own data, without the need to code a bespoke model. Where existing methods are applicable to a fishery’s circumstances, the use of the associated packages avoids redundancy, reduces the time and resource cost of conducting the assessment, and eliminates the potential for programming errors, all of which are associated with the development of bespoke methods (Wilson et al., 2014). Bespoke models have their place, but the features of packages are important in the decision of whether to use a package or develop a bespoke model.
Dichmont et al. (2016a) identified the benefits of using packages rather than user-developed software as: (a) substantially increased flexibility to explore alternative assessment configurations, (b) ease of peer-reviewing, (c) increased confidence that the assessment is correctly coded and tested, (d) the availability of tools to explore uncertainty and summarize model fits to data, (e) increased collaboration amongst assessment analysts on generic questions related to stock assessment practice given a common software platform is being used, (f) faster development time for an assessment, (g) increased ability for a new analyst to take over a stock assessment because they are familiar with the package, and (h) a large user base to facilitate further development and improvement (and to detect errors).
Dichmont et al. (2016b) noted that 76 model-based stocks assessments were conducted in Australia at time of their review (reflecting 34 % of the total catch in 2013), and that all but 18 could have been conducted using stock assessment packages developed in New Zealand and the United States. Apart from legitimate technical reasons, a common reason identified for not using packages was that there is no central repository for stock assessment packages, to allow analysts to identify the package most appropriate to their stock.
There are now many stock assessment packages, some of which (a) implement methods now considered not to be state-of-the-art, (b) are rarely updated and maintained, (c) do not have reliable or comprehensive documentation, or (d) have been subject only to limited testing. Some packages are not well known, while others are widely used e.g. Stock Synthesis (Methot and Wetzell, 2013). The packages can be available from online repositories or platforms, or can be limited to individual countries or institutions. Moreover, the same method may be implemented in more than one package; for example, a statistical catch-at-age approach using empirical weight-at-age data is implemented in both the ASAP (NOAA Toolbox, 2016) and AMAK (NOAA Toolbox, 2016) packages. Consequently, analysts wishing to transition from existing bespoke assessments, or undertake a new assessment, using a package, may be confronted with an overwhelmingly large number of methods and packages that are spread over a wide range of web and/or version control sites, with little or no guidance on which package is most appropriate for their stock and is most up to date. The FishPath decision support tool (Dowling et al., 2016) provides information about which method is appropriate given an assessor’s specific fishery circumstances. However, at present FishPath concentrates on data-limited assessment methods.
Even when users are confident of their choice of method, a package may be difficult to download or implement given the available documentation or support provided. Data formatting and post-assessment analyses can be challenging for many of the more complex packages without good documentation, and pre- and post-processing tools such as r4ss (https://github.com/r4ss/r4ss) and ASAPplots (https://github.com/cmlegault/ASAPplots). Moreover, in the absence of a centralised, global repository, analysts may be unaware of the options for methods and packages, or may have cultural preferences for packages and methods that have less to do with scientific rigor and more to do with the constraint of what is known or familiar (Privitera-Johnson and Punt, 2020).
We collated the available assessment packages and provided links to them using a web-site http://toolbox.frdc.com.au/. A multi-prong approach was used to identify packages, starting with web searches using Google, Google Scholar and library abstract search engines such as Aquatic Sciences and Fisheries Abstracts; search terms included “stock assessment packages” and combinations thereof. The resources provided by the Center for ADvancement of Population Assessment Methods (CAPAM) workshops (Maunder et al., 2014, 2016, 2017; Sharma et al., 2019; Cadrin et al., 2020) were also used to identify data-rich approaches. Assessment reports from organisations such as NOAA and ICES were investigated for package use. Finally, resources in the latest version of NOAA Toolbox (https://noaa-fisheries-integrated-toolbox.github.io/) and FishPath (tool.fishpath.org) were considered.
Although we have tended to be as inclusive as possible, we have not included packages that (a) were not an assessment by our definition (i.e., a method that estimates biomass or fishing mortality), (b) involved software that is not freely available, and (c) had not been supported for more than two years. The aim of the web-site is to provide analysts with the range of possible packages within a standardised, curated platform, enabling them to directly compare package attributes, broadly identify those packages most likely to lead to the most robust results given available data and knowledge, and generally increase the use of state-of-the-art packages as well as the number of assessments that an assessor can undertake compared to the (usually) more time-consuming bespoke models. For some data-poor and data-moderate methods, the web-site includes installation details and instructions of how to apply these, using a common test data set. FishPath would be a valuable resource to be used alongside the web-site to guide an assessor to an assessment method and package, and then this web-site will provide the package’s specification details and location.
This short communication provides an overview of the types of assessment methods included on the web-site and the criteria for identifying the most appropriate methods. The availability of a broad range of stock assessment packages should facilitate the number of robust stock assessments and stock assessment science generally, and the web-site should enhance achievement of that goal. More broadly, the set of packages provides developers of the next-generation stock assessment methods with a sense of the features that assessment analysts desire.
Section snippets
Type of assessment methods
Stock assessments involve many forms of analysis, ranging from assembly of catch, compositional and catch-rate time series, plotting of data to identify changes in fisher behaviour, to applying population dynamics models to determine stock status and the implications of management actions. We focus on the methods used to conduct model-based stock assessments, which we define as estimating biomass and/or fishing mortality, either in absolute terms and/or relative to reference points. Thus,
What makes a good stock assessment package?
The web-site identifies which packages appear to be supported (because there is evidence that the package is being maintained and is occasionally being updated), and indicates which packages are not being maintained, or are still under development (see Table 2 for the categorizations used). We also identify packages that are available online but have been superseded, either by other methods or other packages which we assess in favour. We reviewed methods that have been published in the
Discussion
There has been much work to evaluate alternative stock assessment methods in both data-rich and data-limited contexts. Our web-site fulfils a practical need to collate, characterise, and compare the packages (from data-limited to -rich) that may be employed to undertake any given method. It thus provides a standardised platform so that analysts may make informed choices as to the most effective and efficient way to undertake their stock assessment. The web-site also contributes to an
Conclusion
Our web-site created to collate existing stock assessment packages is not the first nor only. The NOAA Toolbox (https://noaa-fisheries-integrated-toolbox.github.io/) has been available in various forms for decades, highlighting NOAA-developed packages, while several R packages (e.g. SS-DL or MSETool) have been developed that include multiple assessment methods. However, ours is the first that attempts to include all of the packages used globally (with English documentation) to conduct
CRediT authorship contribution statement
Catherine M. Dichmont: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing - review & editing. Roy A. Deng: Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing - review & editing. Natalie Dowling: Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration,
Declaration of Competing Interest
The authors report no declarations of interest.
Acknowledgements
We would like to thank all of the reviewers who provided feedback on initial versions of the web-site, as well as the participants at the 2019 CAPAM workshop on Next Generation Stock Assessments, and the FishPath team for access to their information, as well as to the two anonymous reviewers. This work was funded by the Fisheries Research and Development Corporation (project no. 2018-148), Australia with additional in-kind contributions from CSIRO and Cathy Dichmont Consulting.
References (46)
- et al.
Spatial Structure: theory, estimation and application in stock assessment models
Fish. Res.
(2020) - et al.
Ecopath with Ecosim: methods, capabilities and limitations
Ecol. Model.
(2004) Implementing a statistical catch-at-age model (Stock Synthesis) as a tool for deriving overfishing limits in data-limited situations
Fish. Res.
(2013)- et al.
How many of Australia’s stock assessments can be conducted using stock assessment packages?
Mar.Pol.
(2016) - et al.
Depletion-based stock reduction analysis: a catch-based method for determining sustainable yields for data-poor fish stocks
Fish. Res.
(2011) - et al.
Blood from a stone: performance of catch-only methods in estimating stock biomass status
Fish. Res.
(2020) - et al.
Best practice in Ecopath with Ecosim food-web models for ecosystem-based management
Ecol. Model.
(2016) - et al.
A review of integrated analysis in fisheries stock assessment
Fish. Res.
(2013) - et al.
Selectivity: theory, estimation, and application in fishery stock assessment models
Fish. Res.
(2014) - et al.
Growth: theory, estimation, and application in fishery stock assessment models
Fish. Res.
(2016)
Data conflict and weighting, likelihood functions and process error
Fish. Res.
Stock Synthesis: a biological and statistical framework for fish stock assessment and fishery management
Fish. Res.
Estimation of time-varying selectivity in stock assessments using state-space models
Fish. Res.
A review of approaches to quantifying uncertainty in fisheries stock Assessments
Fish. Res.
Which assessment configurations perform best in the face of spatial heterogeneity in fishing mortality, growth and recruitment? A case study based on pink ling in Australia
Fish. Res.
Essential features of the next-gen Integrated approach: a perspective
Fish. Res.
Recruitment: theory, estimation, and application in fishery stock assessment models
Fish. Res.
Evaluating the performance of data-moderate and catch-only assessment methods for U.S. West coast groundfish
Fish. Res.
JABBA: just another bayesian biomass assessment
Fish. Res.
Gadget User Guide
On the dynamics of exploited fish populations
Fish. Invest. Ser. 2 Mar. Fish. G.B. Minist. Agric. Fish
CASAL (C++ Algorithmic Stock Assessment Laboratory) User Manual v2.30-2012/03/21. NIWA Technical Report 135
The data- limited methods toolkit (DLMtool): an R package for informing management of data- limited populations
Meth. Ecol. Evol.
Cited by (16)
Stock assessment of rock lobster stocks: Past, present and future
2024, Fisheries ResearchSpecies portfolio schemes buffering the risk of overexploitation in mixed fisheries management
2024, Fisheries ResearchA review of estimation methods for natural mortality and their performance in the context of fishery stock assessment
2023, Fisheries ResearchCitation Excerpt :Each of the major stock assessment packages have different features, and hence advantages and disadvantages. It is beyond the scope of the current paper to contrast these packages and provide information on when they are appropriate to use, but this information is available in summary form in Punt et al. (2020) and at the web-site (http://toolbox.frdc.com.au/; Dichmont et al., 2021). An advantage of integrated methods is that it is possible to include the methods outlined above into an analysis with multiple data types.
Preface: Developing the next generation of stock assessment software
2022, Fisheries Research