Optimal exploitation for hybrid systems of renewable resources under partial observation☆
Introduction
This work focuses on the management of renewable resources that are represented by controlled hybrid stochastic differential equations. Although controlled hybrid diffusions have been well studied, the work on controlled stochastic processes for exploitation of resources seems to be relatively limited. Mathematically, the class of stochastic control problems we are interested in have been studied in various settings for different domain of applications. To mention just a few, we refer to [1], [2] for optimal exploitation of a single resource with stochastic population dynamics; [3], [4], [5], [6] for optimal harvesting problems of single species ecosystems in random environments and [7], [8], [9], [10] for interacting populations. The reader can also find related works on optimal dividend strategies in [11], [12]. Motivated by recent developments in renewable resource exploitation and related areas, in contrast to the available results in the literature, we propose a new model to treat a generalized situation with incomplete information.
Consider a two component process given by where , are suitable real-valued functions, is a real-valued Brownian motion, and is a finite-state Markov chain, and is the control component. Because the example is for motivation only, we defer the discussion of the precise formulation and conditions needed to the next section. Suppose is the size of a renewable resource at time . Typical examples of renewable resources are animals, water, and forests, food; where food and fiber are renewable agricultural resources. Harvesting–renewing policies are introduced to derive financial benefit as well as to ensure a certain sustainability of the resource. The goal is to find a policy that maximizes the expected reward. From the biological point of view, is the population size of a species at time . The manager can either harvest partially and sell it or seed part of the species to maintain the ecosystem. Related examples are fishery [13] and forest replanting [14]. Thus, the problem under consideration is important for the establishment of ecologically, environmentally, and economically reasonable wildlife management; see also [1], [15] and references therein.
Treating renewable resource exploitation, most of aforementioned papers devote to a single resource evolving according to a logistic stochastic differential equation and the analysis employs the structure of the equation; see [1], [3], [5]. We are interested in a general hybrid system, which is a generalization of Eq. (1.1), of interacting resources. The numerical approach allows us to approximate the optimal policy and the value function for a broad class of models. It should be noted that in the popular optimal harvesting formulations [3], [5], [6], [8], [9], the control cost is ignored; that is, the manager pays nothing for the actions they do. As a result, in some cases it is optimal for the manager to harvest immediately all the available resources regardless of the fact that the manager might need to pay a lot for such a policy; see [3]. The analysis and numerical examples in [8] pointed out that when the manager decides to harvest (resp. renew), she should do that with the maximal possible harvesting rates (resp. renewing rates). In this work, we go a further step to consider the control costs associated with harvesting and renewing activities. Whenever the manager harvests or renews part of the resource, she has to pay a certain cost and the cost is considered as a real multivariate function of the harvesting–renewing rates, the current sizes of resources, and the regime of the environment. Under this consideration, we observe in Section 5 that in certain cases the maximal possible rates are no longer optimal for harvesting and renewing. Another interesting point in our work is that the environment (Markov chain ) can only be observed with noise; that is, at any given instance, the exact state of residency of is unknown. We can only have noise-corrupted observation of plus noise. The Wonham filter appears to be a promising tool to convert such a partially observed system to completed observed one; see [16], [17], [18]. Focusing on a two-state hidden Markov chain, we will develop an effective way to approximate the system of resources with partial observation.
We emphasize that the results on exploitation for such complex considerations seem to be scared to date to the best of our knowledge. Because of the complexity, a closed-form solution is virtually impossible to obtain, we develop numerical algorithms to approximate the value function and the optimal control. We adopt the Markov chain approximation methodology developed by Kushner and Dupuis [19], [20]. In contrast to the existing results, our new contributions in this paper are as follows. (i) We formulate a generalized exploitation model with a hidden Markov chain for interacting renewable resources, which is a new angle in the study of optimal exploitation problems. (ii) We develop numerical approximation schemes based on the Markov chain approximation method to treat the proposed model. (iii) We explore new phenomena arising from new features in our model by applying the numerical schemes to several stochastic models.
The rest of the work is organized as follows. Section 2 begins with the problem formulation. Section 3 presents the numerical algorithm based on the Markov chain approximation method. Section 4 focuses on the case that the harvesting–renewing rate of each resource is proportional to the resource size. In Section 5, we present several examples. Finally, the paper is concluded with the conclusion section. To facilitate the reading, all proofs are placed in an appendix at the end of the paper.
Section snippets
Formulation
For , let be the size of the th resource at time and denote (with denoting the transpose of with ). We assume that the growth of the resources is subject to random fluctuations and abrupt changes within a finite number of configurations of the environment and we model it by a continuous-time Markov chain taking values in . In this work, we focus on two-state case . Such a Markov chain is also known as a telegraph
Numerical algorithm
Following the Markov chain approximation method in [19], [20], we will construct a controlled Markov chain in discrete time to approximate the controlled diffusions. A careful treatment is required due to the presence of a combination of the control and Wonham filter .
The harvesting–renewing per-unit rates
In this section, we assume that the rate at which each resource can be harvested or renewed is proportional and positively related to the current size. Such a consideration is motivated by the observations in [3, Section 3] or [1]. The evolution of the combined process is given by where and for . We assume that
Examples
Throughout this section, we suppose the discounting factor is . Let be an upper bound introduced for computational purpose. We will compute for and . In particular, we take . In our examples, since is much higher than the carrying capacity of the environment (which can be thought as the maximum population size of a species that can be sustained in that specific environment) the population size will rarely grow outside . Thus, we implicitly
Conclusion
This paper focused on modeling and numerical methods for optimal harvesting–renewing policies in random environments. The novelties of our work include that (1) we formulated an optimal exploitation problem for hybrid stochastic systems of renewable resources with a hidden Markov chain and new features arising in practice. (2) we built the numerical approximation based on Markov chain approximation techniques to solve the optimal control problem under partial observation. The convergence of the
CRediT authorship contribution statement
Ky Tran: Conceptualization, Methodology, Formal analysis, Investigation, Writing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
The author is indebted to Professor George Yin for discussions on numerical methods for optimal control problems. The author thanks the two anonymous reviewers for their careful reading of the manuscript and their insightful comments and suggestions leading to much improvement.
References (29)
- et al.
Optimal exploitation of a resource with stochastic population dynamics and delayed renewal
J. Math. Anal. Appl.
(2019) - et al.
Optimal harvesting from a population in a stochastic crowded environment
Math. Biosci.
(1997) - et al.
Numerical methods for optimal harvesting strategies in random environments under partial observations
Automatica J. IFAC
(2016) - et al.
Numerical methods for optimal dividend payment and investment strategies of regime-switching jump diffusion models with capital injections
Automatica J. IFAC
(2013) - et al.
Risk aversion and optimal forest replanting: A stochastic efficiency study
European J. Oper. Res.
(2007) - et al.
Stochastic competitive Lotka–Volterra ecosystems under partial observation: feedback controls for permanence and extinction
J. Franklin Inst.
(2014) Singular stochastic control in the presence of a state-dependent yield structure
Stochastic Process. Appl.
(2000)- et al.
Stochastic population dynamics under regime switching II
J. Math. Anal. Appl.
(2009) - et al.
Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions
Automatica J. IFAC
(2006) - et al.
Regulation of renewable resource exploitation
SIAM J. Control Optim.
(2020)
Optimal harvesting of stochastically fluctuating populations
J. Math. Biol.
Asymptotic harvesting of populations in random environments
J. Math. Biol.
On optimal harvesting problems in random environments
SIAM J. Control Optim.
Harvesting of interacting stochastic populations
J. Math. Biol.
Cited by (5)
Exponential contraction of switching jump diffusions with a hidden Markov chain
2021, Statistics and Probability LettersApplications of the Linear Quadratic Regulator Optimal Control With Completely and Partially Observed Markovian Switching
2023, Journal of Dynamic Systems, Measurement and Control, Transactions of the ASMEHarvesting of a Stochastic Population Under a Mixed Regular-Singular Control Formulation
2022, Journal of Optimization Theory and Applications
- ☆
This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ICT Consilience Creative, Republic of Korea program (program (IITP-2020-2011-1-00783) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation, Republic of Korea .