Abstract
Measurement error models are studied extensively in the literature. In these models, when the measurement error variance \(\varvec{\Sigma _{\delta \delta }}\) is known, the estimating techniques require positive definiteness of the matrix \(\varvec{S_{xx}-\Sigma _{\delta \delta }}\), even when this is positive definite, it might be near singular if the number of observations is small. There are alternative estimators discussed in literature when this matrix is not positive definite. In this paper, estimators when the matrix \(\varvec{S_{xx}-\Sigma _{\delta \delta }}\) is near singular are proposed and it is shown that these estimators are consistent and have the same asymptotic properties as the earlier ones. In addition, we show that our estimators work far better than the earlier estimators in case of small samples and equally good for large samples.
Similar content being viewed by others
References
Beran, R., Srivastava, M.S.: Bootstrap tests and confidence regions for functions of a covariance matrix. The Annals of Statistics 13: pp. 95–115 (1985)
Cheng, C., Kukush, A.: Non-existence of the first moment of the adjusted least squares estimator in multivariate errors-in-variables model. Metrika 64(1), 41–46 (2006)
Dolby, G.R.: The ultrastructural relation: a synthesis of the functional and structural relations. Biometrika 63(1), 39–50 (1976)
Fuller, W.A.: Measurement Error Models. Wiley (1987)
Gleser, J.L.: Estimation in a multivariate ’errors-in-variables’ regression model: large sample results. Ann. Statist. 9, 24–44 (1981)
Haff, L.: Empirical bayes estimation of the multivariate normal covariance matrix. The Annals of Statistics, pp. 586–597 (1980)
Hoerl, A.E., Kennard, R.W.: Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)
Ledoit, O., Wolf, M.: A well-conditioned estimator for large-dimensional covariance matrices. Journal of multivariate analysis 88(2), 365–411 (2004)
Nakamura, T.: Corrected score function for errors-in-variables models: Methodology and application to generalized linear models. Biometrika 77(1), 127–137 (1990)
Rao C. R.and Rao, M.B.: Matrix algebra and its applications to statistics and econometrics. World Scientific, Singapore (1998)
Sen, S.K.: On numerical measures of singularity of a matrix. Journal of the Indian Institute of Science 49(2), 37 (2013)
Theil, H.: The entropy of the maximum entropy distribution. Economics Letters 5(2), 145–148 (1980)
Vinod, H.: Maximum entropy measurement error estimates of singular covariance matrices in undersized samples. Journal of Econometrics 20(2), 163–174 (1982)
Yuan, K.H., Chan, W.: Structural equation modeling with near singular covariance matrices. Computational Statistics & Data Analysis 52(10), 4842–4858 (2008)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Ravindra B Bapat, Prof.
Appendix
Appendix
Some definitions and results that are needed for the discussions in the paper are presented here.
Definition 1
A sequence of numbers \(\{a_N: N = 1, 2,\dots \}\) converges to a if, for all \(\epsilon > 0\), there exists an \(N_\epsilon\) such that if \(N > N_\epsilon\), then \(\left| a_N-a\right| < \epsilon .\) This is written as \(a_N \rightarrow a\) as \(N \rightarrow \infty\), or as \(\lim _{N\rightarrow \infty }a_N=a\).
Definition 2
A sequence of numbers \(\{a_N: N = 1, 2,\dots \}\) is bounded if and only if there is some \(b < \infty\) such that \(|a_N| < b\) for all \(N = 1, 2,\dots\).If this condition does not hold, then we say that the sequence \(\{a_N\}\) is unbounded.
Definition 3
A sequence \(\{a_N\}\) of numbers is at most of order \(N^\lambda\), denoted by \(O(N^\lambda ),\) if the sequence \(\frac{a_N}{N^\lambda }\) is bounded. In other words, if \(\{a_N\}\) is \(O(N^\lambda )\), then \(a_N\) cannot be infinitely larger than \(N^\lambda\) as \(N\rightarrow \infty\). Also note that when \(\lambda =0\), the sequence \(\{a_N\}\) is at most of order 1 which means that the sequence \(\{a_n\}\) is bounded.
Definition 4
A sequence \(\{a_N\}\) is of order smaller than \(N^\lambda\), denoted by \(o(N^\lambda )\), if \(\lim _{N \rightarrow \infty }\frac{a_N}{N^\lambda }=0\). In other words, If \(\{a_N\}\) is \(o(N^\lambda ),\) then \(a_N\) is infinitely smaller than \(N^\lambda\) as \(N \rightarrow \infty\). Also note that when \(\lambda =0\), the sequence \(\{a_N\}\) is of order smaller than 1 which means that the sequence \(\{a_N\}\) converges to zero.
Lemma 5
If any sequence \(\{a_N\}\) is \(o(N^\lambda )\) then it is also an \(O(N^\lambda )\).
Proof
If \(\{a_N\}\) is \(o(N^\lambda )\), then by definition, \(\lim _{N\rightarrow \infty }\frac{a_N}{N^\lambda }=0\) which implies that \({\frac{a_N}{N^\lambda }}\) is bounded. That is the sequence\(\{a_N\}\) is \(O(N^\lambda )\). \(\square\)
For the following definitions, consider \((\Omega ,{\mathbb {F}},P)\) be the probability space on which the random variables are defined.
Definition 5
A sequence of random variables \(\{X_N\}\) converges in probability to X if, for all \(\epsilon>0,P(|X_N - X| > \epsilon ) \rightarrow 0.\) This is denoted by \(X_N \mathop \rightarrow\limits^{P} X\) and also we write that \(X=plim X_N\).
Definition 6
A sequence of random variables \(\{X_N\}\) is said to converge in distribution to a random variable X if \(P(X_N \le x) \mathop \rightarrow\limits^{P} (X \le x)\) at every point x at which the limit distribution function \(P(X \le x)\) is continuous. This is denoted by \(X_N \mathop \rightarrow\limits^{d} X\).
Note that these definitions can be extended for vectors as well as matrices provided that each element of the vector or matrix satisfies these definitions. Here we state some results without proofs.
Proposition 1
-
(1)
Weak Law of Large Numbers (WLLN): Let \(X_1,X_2,\dots\) be a sequence of identically and independently distributed random variables such that \(E[|X_1|] < \infty\). Then \({\bar{X}} \equiv \frac{1}{n}X_n \mathop \rightarrow\limits^{P} E[X_1].\)
-
(2)
Multivariate Central Limit Theorem (CLT): Let \(\varvec{X_1,X_2,\ldots }\) be identically and independently distributed random vectors in \(\varvec{{\mathbb {R}}^k}\) with mean \(\varvec{\mu = E[X_1]}\) and covariance matrix \(\varvec{\Sigma } = \varvec{E[(X_1 - \mu )(X_1 - \mu )^T]}\). Then
$$\begin{aligned} \sqrt{n}\varvec{({\bar{X}}-\mu )}\mathop \rightarrow\limits^{d}N(\varvec{0},\varvec{\Sigma }) \end{aligned}$$.
-
(3)
Continuous Mapping Theorem (CMT): Let g be a function from \(\varvec{{\mathbb {R}}^k}\) to \(\varvec{{\mathbb {R}}^m}\), and suppose g is continuous at every point in a set C such that \(P(\varvec{X} \in C)=1.\) Then
-
i.
If \(\varvec{X_N} \mathop \rightarrow\limits^{P} \varvec{X}\), then \(\varvec{g(X_N)}\mathop \rightarrow\limits^{P} \varvec{g(X)}\).
-
ii.
If \(\varvec{X_N} \mathop \rightarrow\limits^{d} \varvec{X}\), then \(\varvec{g(X_N)} \mathop \rightarrow\limits^{d} \varvec{g(X)}\).
-
i.
-
(4)
\(X_N \mathop \rightarrow\limits^{P} X \Rightarrow X_N \mathop \rightarrow\limits^{d} X\).
-
(5)
\(X_N \mathop \rightarrow\limits^{d} a\) (a constant) \(\Rightarrow X_N \mathop \rightarrow\limits^{P} a\).
-
(6)
If \(X_N \mathop \rightarrow\limits^{d} X\) and \(|X_N-Y_N| \mathop \rightarrow\limits^{P} 0\), then \(Y_N \mathop \rightarrow\limits^{d} X\).
-
(7)
If \(X_N \mathop \rightarrow\limits^{d} X\) and \(Y_N \mathop \rightarrow\limits^{P} a\) where a is a constant, then the vector \((X_N,Y_N) \mathop \rightarrow\limits^{d} (X,a)\).
-
(8)
If \(X_N \mathop \rightarrow\limits^{P} X\) and \(Y_N \mathop \rightarrow\limits^{P} Y\), then \((X_N,Y_N) \mathop \rightarrow\limits^{P} (X,Y)\).
-
(9)
Slutsky’s Lemma: Let \(\varvec{X_n, X}\) and \(\varvec{Y_n}\) be random matrices. If \(\varvec{X_N} \mathop \rightarrow\limits^{d} \varvec{X}\) and \(\varvec{Y_N} \mathop \rightarrow\limits^{d} \varvec{C}\) where \(\varvec{C}\) is a constant matrix, then
-
(i)
\(\varvec{X_N+Y_N} \mathop \rightarrow\limits^{d} \varvec{X+C}\)
-
(ii)
\(\varvec{Y_NX_N} \mathop \rightarrow\limits^{d} \varvec{CX}\)
-
(iii)
\(\varvec{Y_N^{-1}X_N} \mathop \rightarrow\limits^{d} \varvec{C^{-1}X}\) provided C is invertible
whenever they are well defined.
-
(i)
-
(10)
Lebesgue’s Dominated Convergence Theorem:Let \(\{f_n\}\) be a sequence of real-valued measurable functions on a measure space \((S, {\mathcal {B}}, \mu )\). Suppose that the sequence converges point wise to a function f and is dominated by some integrable function g in the sense that \(|f_{n}(x)|\le g(x)\) for all numbers n in the index set of the sequence and all points \(x \in S\). Then f is integrable and \(\lim _{n \rightarrow \infty }\int _{S}f_{n}\,d\mu = \int _S f\,d\mu\).
Definition 7
Let \(\{X_N\}\) be a sequence of random variables. Suppose that for any \(\epsilon > 0\), there exists a constant \(M (<\infty )\) such that \(P(|X_N| \ge M)\le \epsilon\) for all N, then we say that \(X_N = O_P(1)\). If \(\{\varvec{X_N}\}\) is a sequence of random matrices then we say that \(\varvec{X_N}=O_P(1)\) if every element of \(\varvec{X_N}\) is \(O_P(1)\).
Definition 8
Let \(X_N\) denote a sequence of random variables. If \(X_N \mathop \rightarrow\limits^{P} 0\), then we write \(X_N=o_P(1).\) If \(\varvec{X_N}\) is a sequence of random matrices then we say \(\varvec{X_N}=o_P(1)\) if every element of \(\varvec{X_N}\) is \(o_P(1)\).
Here we state some results without proofs.
Proposition 2
-
(i)
For any random variable X whose first moment exists, we have \(X = O_P (1).\)
-
(ii)
If \(E[|X_N|^k ]\) is bounded for some \(k \ge 1\), then \(X_N = O_P (1).\)
-
(iii)
If \(X_N \mathop \rightarrow\limits^{d} X\), then \(X_N = O_P (1)\).
-
(iv)
If \(X_N = o_P(1)\), then \(X_N = O_P(1)\).
-
(v)
If \(X_N = O_P(1)\) and \(W_N = O_P(1)\), then \(X_N +W_N = O_P(1)\) and \(X_NW_N = O_P(1)\). We write this as \(O_P(1) + O_P(1) = O_P(1)\), and \(O_P(1)\Delta O_P(1) = O_P(1).\)
-
(vi)
\(O_P(1)\Delta o_P(1) = o_P(1)\) and \(O_P(1) + o_P(1) = O_P(1).\)
-
(vii)
\((1+o_P(1))^{-1}=O_P(1)\) and \(o_P(O_P(1))=o_P(1).\)
We conclude this section with the following Lemma.
Lemma 6
Let \(\varvec{C}=(c_{ij})\) be a \(m \times m\) matrix and let \(\Vert \varvec{C}\Vert _{1}\) and \(\Vert \varvec{C}\Vert _{2}\) be the maximum column sum and maximum row sum matrix norms respectively. If atleast one of \(\Vert \varvec{C}\Vert _{1}\) and \(\Vert \varvec{C}\Vert _{2}\) is less than 1, then \(\varvec{(I_m-C)}\) is invertible and
where \(\varvec{C^0}=\varvec{I_m}\)
Proof
Proof is detailed in [10]. \(\square\)
Rights and permissions
About this article
Cite this article
Sarma, B.B., Shoba, B. Consistent estimation in measurement error models with near singular covariance. Indian J Pure Appl Math 53, 32–48 (2022). https://doi.org/10.1007/s13226-021-00024-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13226-021-00024-9