Skip to main content
Log in

Consistent estimation in measurement error models with near singular covariance

  • Original Research
  • Published:
Indian Journal of Pure and Applied Mathematics Aims and scope Submit manuscript

Abstract

Measurement error models are studied extensively in the literature. In these models, when the measurement error variance \(\varvec{\Sigma _{\delta \delta }}\) is known, the estimating techniques require positive definiteness of the matrix \(\varvec{S_{xx}-\Sigma _{\delta \delta }}\), even when this is positive definite, it might be near singular if the number of observations is small. There are alternative estimators discussed in literature when this matrix is not positive definite. In this paper, estimators when the matrix \(\varvec{S_{xx}-\Sigma _{\delta \delta }}\) is near singular are proposed and it is shown that these estimators are consistent and have the same asymptotic properties as the earlier ones. In addition, we show that our estimators work far better than the earlier estimators in case of small samples and equally good for large samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Beran, R., Srivastava, M.S.: Bootstrap tests and confidence regions for functions of a covariance matrix. The Annals of Statistics 13: pp. 95–115 (1985)

    Article  MathSciNet  Google Scholar 

  2. Cheng, C., Kukush, A.: Non-existence of the first moment of the adjusted least squares estimator in multivariate errors-in-variables model. Metrika 64(1), 41–46 (2006)

    Article  MathSciNet  Google Scholar 

  3. Dolby, G.R.: The ultrastructural relation: a synthesis of the functional and structural relations. Biometrika 63(1), 39–50 (1976)

    Article  MathSciNet  Google Scholar 

  4. Fuller, W.A.: Measurement Error Models. Wiley (1987)

    Book  Google Scholar 

  5. Gleser, J.L.: Estimation in a multivariate ’errors-in-variables’ regression model: large sample results. Ann. Statist. 9, 24–44 (1981)

    Article  MathSciNet  Google Scholar 

  6. Haff, L.: Empirical bayes estimation of the multivariate normal covariance matrix. The Annals of Statistics, pp. 586–597 (1980)

  7. Hoerl, A.E., Kennard, R.W.: Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)

    Article  Google Scholar 

  8. Ledoit, O., Wolf, M.: A well-conditioned estimator for large-dimensional covariance matrices. Journal of multivariate analysis 88(2), 365–411 (2004)

    Article  MathSciNet  Google Scholar 

  9. Nakamura, T.: Corrected score function for errors-in-variables models: Methodology and application to generalized linear models. Biometrika 77(1), 127–137 (1990)

    Article  MathSciNet  Google Scholar 

  10. Rao C. R.and Rao, M.B.: Matrix algebra and its applications to statistics and econometrics. World Scientific, Singapore (1998)

    Book  Google Scholar 

  11. Sen, S.K.: On numerical measures of singularity of a matrix. Journal of the Indian Institute of Science 49(2), 37 (2013)

    Google Scholar 

  12. Theil, H.: The entropy of the maximum entropy distribution. Economics Letters 5(2), 145–148 (1980)

    Article  MathSciNet  Google Scholar 

  13. Vinod, H.: Maximum entropy measurement error estimates of singular covariance matrices in undersized samples. Journal of Econometrics 20(2), 163–174 (1982)

    Article  MathSciNet  Google Scholar 

  14. Yuan, K.H., Chan, W.: Structural equation modeling with near singular covariance matrices. Computational Statistics & Data Analysis 52(10), 4842–4858 (2008)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to B. Bhargavarama Sarma.

Additional information

Communicated by Ravindra B Bapat, Prof.

Appendix

Appendix

Some definitions and results that are needed for the discussions in the paper are presented here.

Definition 1

A sequence of numbers \(\{a_N: N = 1, 2,\dots \}\) converges to a if, for all \(\epsilon > 0\), there exists an \(N_\epsilon\) such that if \(N > N_\epsilon\), then \(\left| a_N-a\right| < \epsilon .\) This is written as \(a_N \rightarrow a\) as \(N \rightarrow \infty\), or as \(\lim _{N\rightarrow \infty }a_N=a\).

Definition 2

A sequence of numbers \(\{a_N: N = 1, 2,\dots \}\) is bounded if and only if there is some \(b < \infty\) such that \(|a_N| < b\) for all \(N = 1, 2,\dots\).If this condition does not hold, then we say that the sequence \(\{a_N\}\) is unbounded.

Definition 3

A sequence \(\{a_N\}\) of numbers is at most of order \(N^\lambda\), denoted by \(O(N^\lambda ),\) if the sequence \(\frac{a_N}{N^\lambda }\) is bounded. In other words, if \(\{a_N\}\) is \(O(N^\lambda )\), then \(a_N\) cannot be infinitely larger than \(N^\lambda\) as \(N\rightarrow \infty\). Also note that when \(\lambda =0\), the sequence \(\{a_N\}\) is at most of order 1 which means that the sequence \(\{a_n\}\) is bounded.

Definition 4

A sequence \(\{a_N\}\) is of order smaller than \(N^\lambda\), denoted by \(o(N^\lambda )\), if \(\lim _{N \rightarrow \infty }\frac{a_N}{N^\lambda }=0\). In other words, If \(\{a_N\}\) is \(o(N^\lambda ),\) then \(a_N\) is infinitely smaller than \(N^\lambda\) as \(N \rightarrow \infty\). Also note that when \(\lambda =0\), the sequence \(\{a_N\}\) is of order smaller than 1 which means that the sequence \(\{a_N\}\) converges to zero.

Lemma 5

If any sequence \(\{a_N\}\) is \(o(N^\lambda )\) then it is also an \(O(N^\lambda )\).

Proof

If \(\{a_N\}\) is \(o(N^\lambda )\), then by definition, \(\lim _{N\rightarrow \infty }\frac{a_N}{N^\lambda }=0\) which implies that \({\frac{a_N}{N^\lambda }}\) is bounded. That is the sequence\(\{a_N\}\) is \(O(N^\lambda )\). \(\square\)

For the following definitions, consider \((\Omega ,{\mathbb {F}},P)\) be the probability space on which the random variables are defined.

Definition 5

A sequence of random variables \(\{X_N\}\) converges in probability to X if, for all \(\epsilon>0,P(|X_N - X| > \epsilon ) \rightarrow 0.\) This is denoted by \(X_N \mathop \rightarrow\limits^{P} X\) and also we write that \(X=plim X_N\).

Definition 6

A sequence of random variables \(\{X_N\}\) is said to converge in distribution to a random variable X if \(P(X_N \le x) \mathop \rightarrow\limits^{P} (X \le x)\) at every point x at which the limit distribution function \(P(X \le x)\) is continuous. This is denoted by \(X_N \mathop \rightarrow\limits^{d} X\).

Note that these definitions can be extended for vectors as well as matrices provided that each element of the vector or matrix satisfies these definitions. Here we state some results without proofs.

Proposition 1

  1. (1)

    Weak Law of Large Numbers (WLLN): Let \(X_1,X_2,\dots\) be a sequence of identically and independently distributed random variables such that \(E[|X_1|] < \infty\). Then \({\bar{X}} \equiv \frac{1}{n}X_n \mathop \rightarrow\limits^{P} E[X_1].\)

  2. (2)

    Multivariate Central Limit Theorem (CLT): Let \(\varvec{X_1,X_2,\ldots }\) be identically and independently distributed random vectors in \(\varvec{{\mathbb {R}}^k}\) with mean \(\varvec{\mu = E[X_1]}\) and covariance matrix \(\varvec{\Sigma } = \varvec{E[(X_1 - \mu )(X_1 - \mu )^T]}\). Then

    $$\begin{aligned} \sqrt{n}\varvec{({\bar{X}}-\mu )}\mathop \rightarrow\limits^{d}N(\varvec{0},\varvec{\Sigma }) \end{aligned}$$

    .

  3. (3)

    Continuous Mapping Theorem (CMT): Let g be a function from \(\varvec{{\mathbb {R}}^k}\) to \(\varvec{{\mathbb {R}}^m}\), and suppose g is continuous at every point in a set C such that \(P(\varvec{X} \in C)=1.\) Then

    1. i.

      If \(\varvec{X_N} \mathop \rightarrow\limits^{P} \varvec{X}\), then \(\varvec{g(X_N)}\mathop \rightarrow\limits^{P} \varvec{g(X)}\).

    2. ii.

      If \(\varvec{X_N} \mathop \rightarrow\limits^{d} \varvec{X}\), then \(\varvec{g(X_N)} \mathop \rightarrow\limits^{d} \varvec{g(X)}\).

  4. (4)

    \(X_N \mathop \rightarrow\limits^{P} X \Rightarrow X_N \mathop \rightarrow\limits^{d} X\).

  5. (5)

    \(X_N \mathop \rightarrow\limits^{d} a\) (a constant) \(\Rightarrow X_N \mathop \rightarrow\limits^{P} a\).

  6. (6)

    If \(X_N \mathop \rightarrow\limits^{d} X\) and \(|X_N-Y_N| \mathop \rightarrow\limits^{P} 0\), then \(Y_N \mathop \rightarrow\limits^{d} X\).

  7. (7)

    If \(X_N \mathop \rightarrow\limits^{d} X\) and \(Y_N \mathop \rightarrow\limits^{P} a\) where a is a constant, then the vector \((X_N,Y_N) \mathop \rightarrow\limits^{d} (X,a)\).

  8. (8)

    If \(X_N \mathop \rightarrow\limits^{P} X\) and \(Y_N \mathop \rightarrow\limits^{P} Y\), then \((X_N,Y_N) \mathop \rightarrow\limits^{P} (X,Y)\).

  9. (9)

    Slutsky’s Lemma: Let \(\varvec{X_n, X}\) and \(\varvec{Y_n}\) be random matrices. If \(\varvec{X_N} \mathop \rightarrow\limits^{d} \varvec{X}\) and \(\varvec{Y_N} \mathop \rightarrow\limits^{d} \varvec{C}\) where \(\varvec{C}\) is a constant matrix, then

    1. (i)

      \(\varvec{X_N+Y_N} \mathop \rightarrow\limits^{d} \varvec{X+C}\)

    2. (ii)

      \(\varvec{Y_NX_N} \mathop \rightarrow\limits^{d} \varvec{CX}\)

    3. (iii)

      \(\varvec{Y_N^{-1}X_N} \mathop \rightarrow\limits^{d} \varvec{C^{-1}X}\) provided C is invertible

    whenever they are well defined.

  10. (10)

    Lebesgue’s Dominated Convergence Theorem:Let \(\{f_n\}\) be a sequence of real-valued measurable functions on a measure space \((S, {\mathcal {B}}, \mu )\). Suppose that the sequence converges point wise to a function f and is dominated by some integrable function g in the sense that \(|f_{n}(x)|\le g(x)\) for all numbers n in the index set of the sequence and all points \(x \in S\). Then f is integrable and \(\lim _{n \rightarrow \infty }\int _{S}f_{n}\,d\mu = \int _S f\,d\mu\).

Definition 7

Let \(\{X_N\}\) be a sequence of random variables. Suppose that for any \(\epsilon > 0\), there exists a constant \(M (<\infty )\) such that \(P(|X_N| \ge M)\le \epsilon\) for all N, then we say that \(X_N = O_P(1)\). If \(\{\varvec{X_N}\}\) is a sequence of random matrices then we say that \(\varvec{X_N}=O_P(1)\) if every element of \(\varvec{X_N}\) is \(O_P(1)\).

Definition 8

Let \(X_N\) denote a sequence of random variables. If \(X_N \mathop \rightarrow\limits^{P} 0\), then we write \(X_N=o_P(1).\) If \(\varvec{X_N}\) is a sequence of random matrices then we say \(\varvec{X_N}=o_P(1)\) if every element of \(\varvec{X_N}\) is \(o_P(1)\).

Here we state some results without proofs.

Proposition 2

  1. (i)

    For any random variable X whose first moment exists, we have \(X = O_P (1).\)

  2. (ii)

    If \(E[|X_N|^k ]\) is bounded for some \(k \ge 1\), then \(X_N = O_P (1).\)

  3. (iii)

    If \(X_N \mathop \rightarrow\limits^{d} X\), then \(X_N = O_P (1)\).

  4. (iv)

    If \(X_N = o_P(1)\), then \(X_N = O_P(1)\).

  5. (v)

    If \(X_N = O_P(1)\) and \(W_N = O_P(1)\), then \(X_N +W_N = O_P(1)\) and \(X_NW_N = O_P(1)\). We write this as \(O_P(1) + O_P(1) = O_P(1)\), and \(O_P(1)\Delta O_P(1) = O_P(1).\)

  6. (vi)

    \(O_P(1)\Delta o_P(1) = o_P(1)\) and \(O_P(1) + o_P(1) = O_P(1).\)

  7. (vii)

    \((1+o_P(1))^{-1}=O_P(1)\) and \(o_P(O_P(1))=o_P(1).\)

We conclude this section with the following Lemma.

Lemma 6

Let \(\varvec{C}=(c_{ij})\) be a \(m \times m\) matrix and let \(\Vert \varvec{C}\Vert _{1}\) and \(\Vert \varvec{C}\Vert _{2}\) be the maximum column sum and maximum row sum matrix norms respectively. If atleast one of \(\Vert \varvec{C}\Vert _{1}\) and \(\Vert \varvec{C}\Vert _{2}\) is less than 1, then \(\varvec{(I_m-C)}\) is invertible and

$$\begin{aligned} \varvec{(I_m-C)^{-1}}=\sum \limits _{i=0}^{\infty } \varvec{C^i} \end{aligned}$$

where \(\varvec{C^0}=\varvec{I_m}\)

Proof

Proof is detailed in [10]. \(\square\)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sarma, B.B., Shoba, B. Consistent estimation in measurement error models with near singular covariance. Indian J Pure Appl Math 53, 32–48 (2022). https://doi.org/10.1007/s13226-021-00024-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13226-021-00024-9

Keywords

Mathematics Subject Classification

Navigation