The impact of alternative age-length sampling schemes on the performance of stock assessment methods
Introduction
Stock assessments form a key component of the basis for management advice for fish and invertebrate stocks worldwide. Fournier and Archibald (1982) introduced the first assessment method that integrated multiple sources of data and assumed that fishing mortality was separable into age and annual components. This ‘integrated’ approach to stock assessment separates the development of the population dynamics model from that of a model that relates population dynamics to observations. The parameters of the model are often estimated using a state-space formulation (albeit implemented as penalized likelihood rather than as a true state-space model; Punt et al., 2020). The integrated approach to stock assessment is now considered state-of-the-art and there are applications of the approach in virtually every fisheries management jurisdiction (Maunder and Punt, 2013).
Composition data (length-composition, age-composition, and conditional age-at-length data1) are integral to integrated assessments based on approaches such as Stock Synthesis (Methot and Wetzel, 2013), CASAL/Casal2 (Bull et al., 2005; Doonan et al., 2016), GADGET (Begley, 2005), and MULTIFAN-CL (Fournier et al., 1998). In general, the basic data are collected in the form of length-composition samples and age-length pairs (from which both age-composition and conditional age-at-length data can be developed). For the purposes of this paper the focus is on length-composition and conditional age-at-length data as this is the form in which composition data are included in most Stock Synthesis assessments2 . These data inform several of the parameters of the model on which a stock assessment is based, but in general their use is as follows:
- 3
Length-composition data. These data broadly inform growth (e.g., maximum length), are primarily informative about length-specific selectivity, and can inform estimation of year-class strength. The ability to estimate year-class strength from length-composition data can be weak unless cohorts are evident. Fig. 1 shows true (operating model) and estimated year-class strengths for an assessment method that uses only length-composition data (and an abundance index). The sample sizes are high in this case. The model is able to follow broad trends in year-class strength, but is unable to identify the strengths of (particularly) the strong and weak year-classes (see also, Mohn (1994) and Restrepo (1995)). Length-composition data are generally used when constructing age-compositions by multiplying annual length-compositions by age-length keys given the number of aged animals is often very small3 .
- 4
Conditional age-at-length data. These data provide the key information to estimate the growth curve (the expectation of length-at-age and its variation), selectivity as a function of length, natural mortality (if the age data reflect a period before intense fishing), total mortality, and year-class strength.
Composition data can be collected from ‘fisheries’ and ‘surveys’. Fisheries and surveys are distinguished in stock assessments not just by who operates them and the size of their removals from the population, but also by their selection patterns and how and when they operate. Specifically, a ‘survey’ should be conducted so the assumption of time invariance in selectivity and catchability is not violated (unfortunately this can occur even for very well-designed surveys owing, for example, to the environmental effects on catchability; e.g., for Eastern Bering Sea pollock [Kotwicki et al., 2013]). Moreover, surveys should be designed so that all parts of the survey area have an equal probability of being sampled or, alternatively, such that the analysis approach can account for the survey design. In general, surveys are conducted to monitor younger fish than are captured in fisheries, thereby improving the ability to characterize the growth curve and estimate incoming recruitment.
The value of composition data in providing information on population size and trends depends on how representative of the population / fishery these data are and their ‘effective’ sample size.
The weight assigned to the likelihood component for the composition data in stock assessments depends on the sample size. However, the ‘raw’ sample size (also referred to as a ‘stage-1 sample size’) is seldom used for data weighting because of (a) the pseudo-replication that arises when multiple fish are selected from the same haul, which means that their lengths and ages will not be independent, (b) non-random spatial and temporal distribution of fish, and (c) violations of model assumptions such as temporally and spatially static selection patterns. The ‘ideal’ of 1 fish per haul from several hundred hauls is rarely feasible. Composition data are weighted using ‘effective’ (or ‘stage-2’) sample sizes, which can be thought of as the number of independent samples corresponding to the data set for a given year/area, etc. Consequently, the ‘stage-2 sample sizes’ are based on adjusting the stage-1 sample sizes (which themselves may be based on the number of hauls or landings) using methods such as those of Francis (2011) and Punt (2017) and/or allowance is made for time-varying selection. The latter is not a characteristic of all stock assessments owing to typically low stage-1 sample sizes, and is not considered in this paper.
Past practice has been to impose minimum (stage-1) sample sizes for including composition data in assessments (e.g., 100 lengths). There are several reasons for this: (a) small sample sizes provide effectively no information on abundance, selection, year-class strength etc., (b) the computational time to conduct the assessment increases with the number of age- and length-compositions, (c) reviewers may focus attention on fits to what amounts to noise, (d) having many years of data may give a misleading impression of the amount of information available on which to base assessments, and (e) prior to version 3.30, Stock Synthesis imposed a minimum ‘effective’ sample size of 1, which meant that samples with ‘medium’ effective sample sizes could be given the same weight as ‘low’ effective sample sizes4 . This could cause the model to ‘follow noise’.
Ideally, surveys should be conducted (a) at the same time each year, and (b) with a consistent survey design. Age and length samples (and ideally data on biological parameters such as maturity status) should be collected randomly during the survey. It is often necessary to stratify the collection of composition data to ensure that the resulting data are representative of the population being surveyed. Survey composition data are used to estimate survey selectivity, along with the other quantities in the model.
Some composition data should be collected from all fleets to enable selectivity to be estimated for each fleet (to remove the catches correctly), but it is not necessary for fleets to be equally well sampled. Ideally, samples should be collected proportionally to catch (both spatially and temporally), but this is often not possible, with spatial coverage being of greatest concern for many fisheries, including Australia’s Southern and Eastern Scalefish and Shark Fishery (SESSF). If the model is not spatial, length-composition data should be catch-weighted to obtain data that are as representative of the whole fishery as possible.
For stocks that are assessed using integrated assessments, annual sampling of lengths and ages in the SESSF has been undertaken by onboard and port based observers since the mid-1990s, with some sampling of stocks occurring before this time. The initial sampling design aimed to monitor only the trawl sector of the fishery (Smith et al., 1997; Knuckey and Gason, 2001) and was modified in 2008. Onboard observer effort is currently allocated to strata in proportion to the average of the previous five years of fishing effort, with strata based on location, season and gear type (Bergh et al., 2009). Since 2008, observer effort per annum has declined from over 500 sea days to around 300 sea days.
The specific questions addressed in this paper are:
- •
How does the ability to estimate quantities of management interest change with average annual (effective) sample sizes for length-composition and conditional age-length data?
- •
Does including data sources with small samples in an assessment lead to bias?
- •
What are the implications of collecting length-composition and age data at intervals of greater than one year, but with larger sample sizes?
- •
How does assessment performance change if sampling is unbalanced over time and space?
The focus of this paper is on fishery-dependent composition data to provide advice on the design of sampling schemes (the combination of the effective sample size, the frequency of data collection and the allocation of sampling time spatially) for a fishery (the SESSF) where the primary data source is from fisheries.
The evaluation is based on a simulation evaluation of the performance of a Stock Synthesis assessment in which the assumptions of both the model used to generate the data (the operating model) and that used for parameter estimation (the estimation method) are adjusted to reflect different assumptions regarding how past data were collected and how assessments are conducted. The simulations are based on three fish life histories (short, medium and long) that are characteristic of the SESSF.
Section snippets
General structure of the simulation study
The simulation study involved using three operating models to generate pseudo data sets (200 for each of the scenarios) and applying an estimation method to estimate quantities of management interest. The estimation method is mis-specified for two of three operating models (the operating models are spatially-structured, but the estimation method is not).
Population dynamics
The operating model is a single-sex, age- and length-structured population dynamics model, implemented using Stock Synthesis. The operating
The operating models
The time-trajectories of operating model depletion are qualitatively similar among species and stock structure hypotheses (see Supplementary Fig. 3 for results for a 2018 depletion of 0.5 and the ‘increasing-then-decreasing’ catch series). The among-simulation variation is greater for blue grenadier than for the other two species, which can be attributed to the greater extent of recruitment variation, σR. Although σR for school whiting is lower than for tiger flathead (0.35 vs 0.6), the fewer
Discussion
The ability to estimate quantities of management interest depends on the quality of the available data on abundance and length/age composition as well as other factors such as the nature of the catch time-series and the current status of the population relative to the unfished level. Moreover, which of these factors is most influential in terms of estimation performance varies among quantities, although the nature of the catch series, the current depletion and the quality of length-composition
Authors contributions
AP and GT developed the conceptual framework for the paper; AP created the experimental design and ran the simulations; AP, GT, JD, PB, RB, and PBB wrote the MS, interpreted the results in the manuscript.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
L. Richard Little and two anonymous reviewers are thanked for their comments on an earlier version of this paper. This publication was funded by CSIRO and the Cooperative Institute for Climate, Ocean and Ecosystem Studies (CICOES) under NOAA Cooperative Agreement. This is JISAO publication contribution 2020-1124.
References (43)
- et al.
Accounting for spatial population structure at scales relevant to life history improves stock assessment: the case for Lake Erie walleye Sander vitreus
Fish. Res.
(2012) - et al.
Reconciling stock assessment and management scales under conditions of spatially varying catch histories
Fish. Res.
(2011) - et al.
Use of multiple selectivity patterns as a proxy for spatial structure
Fish. Res.
(2014) - et al.
Can steepness of the stock-recruitment relationship be estimated in fishery stock assessment models?
Fish. Res.
(2012) - et al.
A review of integrated analysis in fisheries stock assessment
Fish. Res.
(2013) - et al.
Stock Synthesis: a biological and statistical framework for fish stock assessment and fishery management
Fish. Res.
(2013) Some insights into data weighting in integrated stock assessments
Fish. Res.
(2017)- et al.
Which assessment configurations perform best in the face of spatial heterogeneity in fishing mortality, growth and recruitment? A case study based on pink ling in Australia
Fish. Res.
(2015) - et al.
Essential features of the next-gen integrated assessment: a perspective
Fish. Res.
(2020) - et al.
Model-based estimates of effective sample size in stock assessment models using the Dirichlet-multinomial distribution
Fish. Res.
(2017)
Using areas-as-fleets selectivity to model spatial fishing: asymptotic curves are unlikely under equilibrium conditions
Fish. Res.
Gadget User Guide. Marine Research Institute, Reykjavik, Iceland) 90 Marine Research Institute Report 120
A Revised Sampling Regime for the Southern and Eastern Scalefish and Shark Fishery
CASAL (C++ Algorithmic Stock Assessment Laboratory): CASAL User manualv2.30-2012/03/21. NIWA Technical Report 127
Blue grenadier (Macruronus novaezelandiae) stock assessment based on data up to 2017 base case. Pp. 314-341
When can we reliably estimate the productivity of fish stocks?
Can. J. Fish. Aquat. Sci.
Tiger flathead (Neoplatycephalus richardsoni) stock assessment using data to 2015. Pp. 443-512
School whiting (Sillago flindersi) stock assessment based on data up to 2016. Pp. 588-663
Simulation testing the robustness of stock assessment models to error: some results from the ICES strategic initiative on stock assessment methods
ICES J. Mar. Sci.
Casal2: New Zealand’s integrated population modelling tool
Fish. Res.
A general theory for analyzing catch at age data
Can. J. Fish. Aquat. Sci.
Cited by (2)
Towards best practice for specifying selectivity in age-structured integrated stock assessments
2022, Fisheries ResearchCitation Excerpt :The contribution of each composition datum, relative to its various sources and other types of data, relies on a weighting factor characterized as the effective sample size (Punt et al., 2021). Punt et al. (2021) demonstrated that, while higher effective sample sizes for length-composition and conditional age-at-length data resulted in smaller errors when estimating key management quantities (i.e., spawning biomass, depletion, recruitment, population model parameters, catch limit recommendations), effective sample size is often not the main factor that determines the magnitude of estimation errors. Overly simple selectivity formulations are a gross approximation to the many factors influencing a fishery’s time-varying interaction with a population of fish (Maunder et al., 2014), and violations can lead to bias and imprecision when estimating quantities of management interest (Maunder and Punt, 2013).