Abstract
Queues can be seen as a service facility where quality of service (QoS) is an important measure for the performance of the system. In many cases, the queue implements the optimal admission control (either discounted or average) policy in the presence of holding/congestion cost and revenue collected from admitted customers. In this paper, users offer an arrival rate at stationarity that depends on the QoS they experience. We study the interaction between arriving customers and such a queue under two different QoS measures—the asymptotic rate of the customers lost and the fraction of customers lost in the long run. In particular, we investigate the behaviour of equilibrium points and equilibrium sets associated with this interaction and their interpretations in terms of business cycles. We provide sufficient conditions for existence of equilibrium sets for M/M/1 queue. These conditions further help us to identify the relationship among system parameters for which equilibrium sets exist. Next, we consider \(\textit{GI}/M/1\) queues and provide a sufficient condition for existence of multiple optimal revenue policies. We then specialize these results to study the equilibrium sets of (i) a D/M/1 queue and (ii) a queue where the arrival rate is locally continuous. The equilibrium behaviour in the latter case is more interesting as there may be multiple equilibrium points or sets. Motivated by such queues, we introduce a weaker version of monotonicity and investigate the existence of generalized equilibrium sets.
Similar content being viewed by others
Notes
We assume an affine demand model for ease of computation and concreteness. Note that our results hold for any \(f(\cdot )\) which satisfies the monotonicity Assumption A1 mentioned in Section 3.1.
Throughout this paper, we use user-set, market or customer-base interchangeably. Similarly, we use queue, service-provider or firm interchangeably.
This length \(|{\mathcal {I}}|\) can also be obtained by using QoS L for \(\rho = 1\). In particular, the interval for \(\rho = 1\) is \( \bigg (m - \frac{e}{{R^*_\lambda }+1},\, m - \frac{e}{{R^*_\lambda }+2}\bigg )\).
Taking the limit \(\rho \rightarrow 1\) allows us to simplify the first condition as, \(\frac{r}{h} = \frac{(R^*_\lambda +1)\, (R^*_\lambda +2)}{2\, \lambda }\).
References
Hemachandra, N., Naga, R.K.S., Qavi, M.A.: A model for equilibrium in some service-provider user-set interactions. Ann. Oper. Res. 243, 95–115 (2016)
Ariel, R.: Comments on the interpretation of game theory. Econom. J. Econom. Soc. 59(4), 909–924 (1991)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (2005)
Stidham Jr., S.: Socially and Individually optimal control of arrivals to a \(GI/M/1\) queue. Manag. Sci. 24, 1598–1610 (1978)
Osborne, M.J., Rubinstein, A.: A Course in Game Theory. MIT Press, Cambridge (1994)
Fudenberg, D., Tirole, J.: Game Theory. MIT Press, Cambridge (1991)
Narahari, Y.: Game Theory and Mechanism Design (IISc Lecture Notes Series) (2014)
Van Nunen, J.A.E.E., Puterman, M.L.: Computing optimal control limits for \(GI/M/s\) queuing systems with controlled arrivals. Manag. Sci. 29(6), 725–734 (1983)
Puranam, K.S., Hemachandra, N.: Sensitivity Analysis of the Control Limit with Respect to the Cost Parameters in an M/M/1 Queue, Vision 2020. The Strategic Role of Operations Research, ORSI, Ahmedabad, pp. 433–439 (2005)
Wolff, R.W.: Stochastic Modeling and the Theory of Queues. Prentice-Hall, Englewood Cliffs (1989)
Cil, E.B., Ormeci, E.L., Karaesmen, F.: Effects of system parameters on the optimal policy structure in a class of queueing control problems. Queue. Syst. 61, 273–304 (2009)
Rue, R.C., Rosenshine, M.: Some properties of optimal control policies for entry to an \(M/M/1\) queues. Nav. Res. Logist. Q. 28, 525–532 (1981)
Haviv, M., Puterman, M.L.: Bias optimality in controlled queueing systems. J. Appl. Prob. 35(1), 136–150 (1998)
Puterman, M.L., Thomas, L.C.: A note on computing optimal control limits for \(GI/M/1\) queueing systems. Manag. Sci. 33(7), 939–943 (1987)
Hemachandra, N., Tripathi, S., Patil, K.: Equilibrium sets of some \(GI/M/1\) queues, Technical Report. http://www.ieor.iitb.ac.in/files/EqSetsTechReport.pdf
Roughgarden, T.: Introduction to Inefficiency in Equilibria, Chapter 17 in Algorithmic Game Theory, vol. 441–457. Cambridge University Press, Cambridge (2007)
Stidham, S., Jr.: The price of anarchy for a network of queues in heavy traffic. In: Essays in Production, Project Planning and Scheduling: A Festschrift in Honor of Salah Elmaghraby, pp. 91–121 (2014)
Naor, P.: The regulation of queue size by levying tolls. Econometrica 37(1), 15–24 (1969)
Stidham Jr., S.: Optimal control of admission to a queueing system. IEEE Trans. Autom. Control 30(8), 705–713 (1985)
Stidham Jr., S., Weber, R.R.: A survey of Markov decision models for control of networks of queues. Queue. Syst. 13, 291–314 (1993)
Sennott, L.I.: Stochastic Dynamic Programming and the Control of Queueing Systems. Wiley, New York (1998)
Stidham Jr., S.: Optimal Design of Queueing Systems. Chapman & Hall CRC, London (2009)
Hassin, R., Haviv, M.: To Queue or Not to Queue, International Series in Operations Research and Management Science. Springer, Berlin (2003)
Hassin, R.: Rational Queueing, Chapman & Hall/ CRC Series in Operations Research. CRC Press, Boca Raton (2016)
Kim, W.S.: Price-based quality-of-service control framework for two-class network services. J. Commun. Netw. 9(3), 319–329 (2007)
Fleming, W.H.: Functions of Several Variables. Springer, Berlin (1987)
James, G., James, R.C.: The Mathematics Dictionary. Springer, Berlin (1992)
Acknowledgements
We would like to thank both the anonymous Reviewers for their invaluable comments which helped us to improve the content and presentation of the manuscript. We would also like to thank both the Guest Editors for their support and suggestions for revising the manuscript. Lastly, most of this work was done when Dr. KP and Dr. ST were Masters students at IEOR@IIT Bombay, and during that period, they were partially supported by Teaching Assistantships offered by Government of India.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Existence of control limits and properties of QoS
1.1 A.1 Proof of Lemma 2
Proof
Let \(R_\lambda \) be the control limit when the arrival rate is \(\lambda \) of an admission-controlled M/M/1 queue. Let \(\varLambda _{R_\lambda }\) denote the rate at which customers are admitted into the system in the long run, \(H_{R_\lambda }\) denote the average queue length and \(g^{R_\lambda } = r\, \varLambda _{R_\lambda } - hH_{R_\lambda }\) denote the average utility (difference of reward and holding cost). Using the stationary distribution for a \(M/M/1/R_\lambda \) queue,
when \( \rho = \frac{\lambda }{\mu } < 1\). If there exists \(R_\lambda \) such that, \(g^{R_\lambda -1}\) is greater than \(g^{R_\lambda }\), then we know that a finite control limit exists which is \(R_\lambda \). Consider
After straightforward algebraic manipulations,
Note that \(\rho < 1\), and from the above equation, one can find a sufficiently large \(R_\lambda \) for which \(g^{R_\lambda }-g^{R_\lambda -1} < 0 \). \(\square \)
1.2 A.2 Proof of Theorem 1
Proof
We already know that if multiple optimal control limit exist, they are consecutive integers. Let us assume that \(R^*_\lambda \) and \(R^*_\lambda +1 \) are those optimal values. The QoS is given by \(L(R^*_\lambda ) = \frac{(1-\rho )\rho ^{R^*_\lambda }}{1-\rho ^{{R^*_\lambda }+1}}\). For the first condition, let us solve the expected average reward of the queue as
where \(\varLambda _{R_\lambda }\) is the rate at which customers are admitted into the system in the long run and \(H_{R_\lambda }\) denotes average queue length. The above equation is nothing but the difference between reward and holding cost. Using the stationary distribution for an \(M/M/1/R_\lambda \) queue it is easy to see that
The firm’s profit remains the same for these multiple optimal control limits \(R_\lambda ^*\) and \(R_\lambda ^* + 1\), i.e.
Using (13) in the above, straightforward algebraic manipulations yield the desired result.
The second condition is straightforward. Using the value of the QoS L in the affine utility function \(f(L(\lambda ))\) we obtain
This completes the proof. \(\square \)
1.3 A.3 Proof of Theorem 2
Proof
Let \(\{\tau _i\}_{i\ge 1}\) be the sequence of inter-arrival times and y be the expected inter-arrival time. Also, let \(\{\sigma _i\}_{i\ge 0}\) be the sequence of times which denotes successive arrivals and \(W_t\) denote the state of the natural process (number in the system) at time t.
Let \(X_i\), \(Y_i\) be the random variables denoting the state and action at ith decision epoch, \(k(X_i,Y_i)\) be the lump sum portion of the reward and \(c(W_t,X_i,Y_i)\) be the rate at which the continuous portion of the reward is received between the decision epoch i and \(i+1\) for \(i \in \{0,1,2,\ldots \}.\) From [3, Chapter 11], the average reward is as follows:
Substitute \(r=1\) and \(h=0\) and note that this is a finite state Markov chain. We have
Thus,
One can compute \(g^{{R_\lambda }}\) by substituting \(r=1\) and \(h=0\) in [14, Equation 13] after making the changes given in Sect. 5.2. This is same as the system of equations in (8). Let the solution of this system of equations be \({\tilde{g}}^{R_\lambda }\). Then, the fraction of customers lost in long run is
and the rate of customers lost is
\(\square \)
1.4 A.4 Proof of Theorem 3
Proof
Let \(R^*_{\lambda _0}\) be the unique control limit at \(\lambda _0\). Then,
For the linear system of equations in (6), the coefficient matrix w.r.t. the variables \(v_\alpha ^{R_\lambda }(0),\ldots ,v_\alpha ^{R_{\lambda }}(R_{\lambda })\) is same as the matrix \((I-M_{R^*_{\lambda _{0}}})\) from [3, Theorem 11.3.1], where \(M_{R^*_{\lambda _{0}}}\) is the transition probability matrix corresponding the Markov Deterministic policy \(R^*_{\lambda _0}\). The solution of this system of equations is the value vector corresponding to the policy \(R^*_{\lambda _0}\) which exists and is unique. This implies that the matrix \((I-M_{R^*_{\lambda _0}})\) is invertible. Due to the continuity of matrix inverse, the system of equations in (6) will have a solution \(\phi _{i}(\lambda ), i=0,\ldots ,R_\lambda \), which is continuous in a small enough neighbourhood (a, b) of \(\lambda _0\) (which can also be obtained using the implicit function theorem [26, 27] on the above linear system), given below:
So, for a small enough interval (a, b) around \(\lambda _0\) we have
Therefore, in the interval (a, b) the optimal control limit will be \(R^*_{\lambda _0}\). \(\square \)
1.5 A.5 Proof of Theorem 4
Proof
From Theorem 3, the optimal control limit at \(\lambda _2\) cannot be unique as it will give rise to an open interval around \(\lambda _2\) overlapping with \((\lambda _1,\lambda _2)\) on the left (and \((\lambda _2,\lambda _3)\) on the right) leading to a contradiction on uniqueness of \(R^*_\lambda \) (and \({\tilde{R}}^{*}_{\lambda }\)). So, multiple optima exists at the common point \(\lambda _2\) of these open intervals. From Corollary 3 in [14], the value \(v_\alpha ^{R^*_\lambda }(s)\) is unimodal with respect to \(R^*_\lambda \) with possible ties at multiple optimal control limits which necessarily occur consecutively. Also,
Taking the limit \(\lambda \uparrow \lambda _2\) on both sides and using the fact that continuous and unique functions have a unique limit, we get
Also,
Taking \(\lambda \downarrow \lambda _2\), we get
Suppose \({\hat{R}}^{*}_{\lambda _2}\) is optimal at \(\lambda _2\) but \(R^{*}_{\lambda _2}\) is not optimal at \(\lambda _2\). Then,
We also know that \(v_\alpha ^{{R}^*_{\lambda _2}}(s)\) is a continuous function of \(\lambda _2\). Therefore, we can always find an \(\epsilon \) small enough such that
However, (17) contradicts the definition of \(R^{*}_{\lambda }\) as the unique optimal control limit in \((\lambda _1,\lambda _2)\). Hence,
Since \({\hat{R}}^*_{\lambda }\) is the optimal control limit and (18) holds, it can be concluded that \(R^*_\lambda \), \({\tilde{R}}^{*}_{\lambda }\) and \({\hat{R}}^*_{\lambda }\) are optimal at \(\lambda _2\), and others also if they exist. \(\square \)
1.6 A.6 Proof of Theorem 5
Proof
If there exist multiple optima for a given \(\lambda \), then
The value vector can be uniquely determined by the first \(R_{\lambda }+1\) states, as the remaining \(v_{\alpha }^{R_{\lambda }}(s)\)’s for \(s>R_{\lambda }\) can be evaluated recursively using the first \(R_{\lambda }+1\) components of the value vector. Therefore, it is enough to check that the above equation holds for all \(s\le R_{\lambda }\).
As given by Van Nunen and Puterman in [8],
Now, \(v_{\alpha }^{R_{\lambda }}(s)=v_{\alpha }^{R_{\lambda }+1}(s)\) gives \(v_{\alpha }^{R_{\lambda }}(s-1)=v_{\alpha }^{R_{\lambda }+1}(s-1)\) and doing this recursively ultimately leads to \(v_{\alpha }^{R_{\lambda }}(0)=v_{\alpha }^{R_{\lambda }+1}(0)\). So, this condition is a necessary one. We can see that it is also a sufficient condition. \(\square \)
Appendix B: Methods for calculating \(\lambda \) when \(R^*_\lambda = 1,2\) in a discounted cost criteria M/M/1 queue
After showing the existence of multiple control limits, we want to identify the point \(\lambda \) where multiple optimal control limits exist. The following three methods involve finding \(\varDelta \lambda \), if we start from some arbitrary \(\lambda _0\). We only consider the case when \(R_{\lambda }=1,2\). For higher value of control limits, these methods becomes very cumbersome.
1.1 B.1 Method 1
The first method involves equating \(v_\alpha ^{R_\lambda }(0)\) and \(v_\alpha ^{R_{\lambda }+1}(0)\) as a function of \(\varDelta \lambda \). In particular, for \(R_{\lambda }=1\) and \(R_{\lambda }=2\), the value at state 0 is given as follows:
After adding each of the roots obtained from solving \(v_{\alpha }^{1}(0) = v_{\alpha }^{2}(0)\) to \(\lambda _0\), one of these value is the \(\lambda \) where the multiple optimal control exists. However, we need to check and find which one is the correct root by finding the control limit at the each of them. Also, some roots might be invalid. The following example illustrate the method for a M/M/1 queue with given set of costs and parameters \(r,h,\mu ,\alpha \). The computations were done in Mathematica 9 on Paaspoli server (OS: Debian, AMD OPTERON 6212, 2.6 GHz processors and 32GB RAM).
-
1.
\({h = 1}\), \({r=0.4}\), \({{\mu } = 5.5}\), \({{\alpha }=0.54}\) and \({{\lambda _0} = 10}\)
The simplified version of the final equation to be solved is \(1.45185\varDelta \lambda ^4+61.9742\varDelta \lambda ^3+955.654\varDelta \lambda ^2+6309.78\varDelta \lambda +14988.1\). Roots for this equation are
-
(a)
\(\varDelta \lambda _1 =-10.000000\)
-
(b)
\(\varDelta \lambda _2 =-6.106282\)
-
(c)
\(\varDelta \lambda _3 =-10.540051\)
-
(d)
\(\varDelta \lambda _4 =-16.040032\)
Root (b) is the point which after adding to \(\lambda _0\) gives the point of multiple optima, i.e. \(\lambda = 3.893718\).
-
(a)
1.2 B.2 Method 2
This method involves solving an optimization problem which minimizes the square of the difference between the value corresponding to control limits \(R_\lambda =1,2\) at state zero. The optimization problem is formulated as follows:
The above constrained optimization problem is a non convex minimization problem. It can be solved easily using solvers like SNOPT, KNITRO, etc. This approach has an edge over the earlier one as it gives an if and only if condition for obtaining the \(\lambda \) where multiple optimal control limits exist.
Theorem 7
For an M/M/1 queue with discounted criteria, multiple optimal control limits 1 and 2 will exist if and only if M2 attains its global minimum.
Proof
Suppose we have multiple optimal control limits 1 and 2 for a given \(\lambda _m\) (say \(\lambda _0 + \varDelta \lambda _m\)). Then, at \(\lambda _0 + \varDelta \lambda _m\), the value vectors for control limit 1 and 2 are equal, i.e.
In particular, we have \(v_\alpha ^{1,0}(\lambda _0 + \varDelta \lambda _m) = v_\alpha ^{2,0}(\lambda _0 + \varDelta \lambda _m)\). Hence, we conclude that at the point of multiple optima \(\lambda _m\), the objective value of M2 is 0, and it attains its global minima. Let us consider the converse case where we are given that the global optimal solution of M2 with objective value 0 is attained at \(\varDelta \lambda _m\). Then, we have that for \(\varDelta \lambda _m\),
This is a sufficient condition for multiple optimal control limit to exist at \(\lambda _0 + \varDelta \lambda _m\) as given in Theorem 5.
Hence, the solution set of the above minimization problem characterizes the set of points at which multiple optimal control limit exists. \(\square \)
The following example illustrate this constrained optimization method. The optimization problem was modelled in AMPL and solved using the SNOPT solver on OPTIMUS server (OS Linux, Intel Quad core Xeon E5506 2.13 GHz and 64GB RAM).
-
1.
Costs and parameters \({h = 1}\), \({r=0.4}\), \({{\mu } = 5.5}\), \({{\alpha }=0.54}\) and \({{\lambda _0} = 5}\) of an M/M/1 queue when used in M2 gives objective value \(= 1.349202e^{-12}\) and \(\varDelta \lambda = -1.10629\), giving \(\lambda = 3.89371\).
1.3 B.3 Method 3
If we substitute the value of \(v_\alpha ^{1,0}(\lambda _0 + \varDelta \lambda )\) and \(v_\alpha ^{2,0}(\lambda _0 + \varDelta \lambda )\) in the objective function of (19), then we get the unconstrained version of M2 given as follows:
This can be solved by equating the gradient of the objective function in M3 to 0 as shown below:
The solution of the unconstrained optimization problem can be root of either of the terms in (21). However, we are only interested in the root of the first term which gives global minima. The roots of second term might lead to some unwanted solution with non-zero objective value.
The following example illustrate this method using AMPL as modelling language. The optimization problem was solved using the SNOPT solver on OPTIMUS server (OS Linux, Intel Quad core Xeon E5506 2.13 GHz and 64GB RAM).
-
Costs and parameters \({h = 1}\), \({r=0.4}\), \({{\mu } = 5.5}\), \({{\alpha }=0.54}\) and \({{\lambda _0} = 10}\) of an M/M/1 queue when used in M3 gives objective value \(=3.722129e^{-10}\) and \(\varDelta \lambda = -6.10629\), giving \(\lambda = 3.89371\).
Appendix C: Generation of the arrival rates from a discrete distribution
Consider the following discrete distribution whose mean is required to be locally continuous:
Mean arrival rate is \(\lambda =u_{_{1,k}}+k\times u_{_{k}}+(k+1)\times u_{_{k+1}}\). We will fix \(u_{_{1,k}}\) in order to generate mean arrival rate for a suitable interval. Let us take the support as \(\{1,\, k,\, k+1\}\); the arrival rate is \(\lambda \in (\underline{m}_{k},{\overline{m}}_{k}]\), where \({\overline{m}}_{k}\) and \(\underline{m}_{k}\) can be derived as follows:
Now, \(u_{_{k}}\in [0,\,1-u_{_{1,k}}]\). Thus, \({\overline{m}}_{k}=(k+1)-k\times u_{_{1,k}}\) and \(\underline{m}_{k}=(k+1)-k\times u_{_{1,k}}-(1-u_{_{1,k}}).\).
To generate arrival rate whose mean is \(\lambda \in (\underline{m}_{k+1},{\overline{m}}_{k+1}]\), we change the support of the distribution to \(\{1,\, k+1,\, k+2\}\) and so on. While changing the support we are fixing \(u_{_{1,k+1}} \ge u_{_{1,k}}\), where \({\overline{m}}_{k+1}=(k+2)-(k+1)\times u_{_{1,k+1}}\) and \(\underline{m}_{k+1}=(k+2)-(k+1)\times u_{_{1,k+1}}-(1-u_{_{1,k+1}})\).
Example 11
In this example, we will illustrate the above method to generate arrival rates whose mean lies in \((3.7,\,6.25]\).
-
1.
We take the support as \(\{1,\,4,\,5\}\) and fix \(u_{_{1,4}}=0.1\). Then, \(\lambda =0.1+4u_{_{4}}+5(0.9-u_{_{4}})\), i.e. \(\lambda =4.6-u_{_{4}}\). Thus, with support \(\{1,\,4,\,5\}\), \( u_k \in [0, 0.9)\) and \(u_{5} = 1 -u_4 \) we can generate a set of arrival rates with above mass functions such that the arrival rate \(\lambda \) is continuous over the interval \((3.7,\,4.6]\).
-
2.
The support now is \(\{1,\,5,\,6\}\) and we fix \(u_{_{1,5}}=0.15\). So, \(\lambda =0.15+4u_{_{5}}+5(0.85-u_{_{4}})\), i.e. \(\lambda =5.25-u_{_{5}}\). Thus, with support \(\{1,\,5,\,6\}\) we can generate \(\lambda \in (4.4,\,5.25]\) but we have already generated \(\lambda \) until 4.6 from the previous support; so, we generate a family of arrivals whose rates continuously change over \(\lambda \in (4.6,\,5.25]\).
-
3.
We take the support as \(\{1,\,6,\,7\}\) and we take \(u_{_{1,6}}=0.2\). Then, \(\lambda =0.2+6u_{_{6}}+7(0.8-u_{_{6}})\), i.e. \(\lambda =5.8-u_{_{6}}\). Thus, with support \(\{1,\,6,\,7\}\) we can generate \(\lambda \in (5,\,5.8]\) but we have already generated \(\lambda \) until 5.25 from the previous support; so, we now have arrivals whose rate is locally continuous over \(\lambda \in (5.25,\,5.8]\).
-
4.
We take \(\{1,\,7,\,8\}\) as support and \(u_{_{1,7}}=0.25\) so that \(\lambda =0.25+7u_{_{7}}+8(0.75-u_{_{7}})\), i.e. \(\lambda =6.25-u_{_{7}}\). Thus, with support \(\{1,\,7,\,8\}\) we can generate \(\lambda \in (5.5,\,6.25]\) but we have already generated \(\lambda \) until 5.25 from the previous support; so, we have the arrivals with the desired property over \(\lambda \in (5.8,\,6.25]\).
Rights and permissions
About this article
Cite this article
Hemachandra, N., Patil, K. & Tripathi, S. Equilibrium points and equilibrium sets of some \(\textit{GI}/M/1\) queues. Queueing Syst 96, 245–284 (2020). https://doi.org/10.1007/s11134-020-09677-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11134-020-09677-5
Keywords
- Admission control of queues
- Quality of service
- Multiple optimal control limits
- Fixed points
- Threshold policies
- Inter-arrival time distribution
- Business cycles