Abstract Predator–prey interaction is considered as a natural phenomenon in ecological system. An obvious question arises in ecosystem is that how predator affect the density of prey population. In most of the ecological models, predators reduce prey densities by direct killing which is a minor part in population dynamics. Experimental studies on vertebrate has shown that fear of predators can influence the behavioural pattern of prey population and reduce their reproduction. Mathematical models of predator–prey system with variety of predator functional responses with fear effect are studied recently. These studies ignore the fear effect on the death rate of prey species. Based on the above observation, we propose a mathematical model of predator–prey system with the cost of fear that affect reproduction and death rate of prey population. We also include a term modelling intraspecific competition within the predator population. This additional term limits the predator population growth for large predator to prey density ratios. Basic results on the positivity and boundedness of the solutions, feasibility of equilibria, local and global stability and uniform persistence (ensuring the long term survival of the species) and limit cycles and its nature are investigated. Numerical experiments are performed to test the theoretical results obtained for this model.

中文翻译：

---

恐惧在具有种内竞争的捕食者-猎物系统中的作用

摘要 捕食者-猎物相互作用被认为是生态系统中的一种自然现象。生态系统中出现的一个明显问题是捕食者如何影响猎物种群的密度。在大多数生态模型中，捕食者通过直接杀戮来降低猎物密度，这在种群动态中只占很小的一部分。对脊椎动物的实验研究表明，对捕食者的恐惧会影响猎物种群的行为模式并减少它们的繁殖。最近研究了具有各种捕食者功能反应和恐惧效应的捕食者-猎物系统的数学模型。这些研究忽略了恐惧对猎物物种死亡率的影响。基于上述观察，我们提出了一个捕食者-猎物系统的数学模型，其中的恐惧成本会影响猎物种群的繁殖和死亡率。我们还包括一个术语，对捕食者种群内的种内竞争进行建模。这个额外的术语限制了大型捕食者与猎物密度比的捕食者种群增长。研究了关于解的正性和有界性、平衡的可行性、局部和全局稳定性和均匀持久性（确保物种的长期生存）和极限循环及其性质的基本结果。进行了数值实验以测试为该模型获得的理论结果。研究了局部和全局稳定性和统一持久性（确保物种的长期生存）和极限循环及其性质。进行了数值实验以测试为该模型获得的理论结果。研究了局部和全局稳定性和统一持久性（确保物种的长期生存）和极限循环及其性质。进行了数值实验以测试为该模型获得的理论结果。