Some information inequalities for statistical inference

Abstract

In this paper, we first describe the generalized notion of Cramer–Rao lower bound obtained by Naudts (J Inequal Pure Appl Math 5(4), Article 102, 2004) using two families of probability density functions: the original model and an escort model. We reinterpret the results in Naudts (2004) from a statistical point of view and obtain some interesting examples in which this bound is attained. Further, we obtain information inequalities which generalize the classical Bhattacharyya bounds in both regular and non-regular cases.

Appendix

Lemma 1

Let X be a random variable with pdf

$$\begin{aligned} f(x,\theta )=\mathcal {N}(0,\theta ^{2})=\frac{1}{\sqrt{2 \pi } \theta } e^{\frac{-x^{2}}{2\theta ^{2}}} , \quad x \in \mathbb {R} \quad \mathrm {and } \quad \theta >0. \end{aligned}$$

(105)

Consider a pdf

$$\begin{aligned} g(x,\theta )=\frac{1}{\sqrt{2 \pi } \theta } \left( \frac{3}{4} + \frac{x^{2}}{4 \theta ^{2}}\right) e^{\frac{-x^{2}}{2\theta ^{2}}}, \quad x \in \mathbb {R}. \end{aligned}$$

(106)

Then the family $\{ g_{\theta } \mid \theta >0\}$ satisfies Assumptions (18) and (19).

Proof

Since $\mathrm {supp}(g_{\theta }) = \mathrm {supp}(f_{\theta })$, $P_{g_{\theta }}$ is absolutely continuous with respect to $P_{f_{\theta }}$ for all $\theta \in \varTheta$. Hence, $\{ g_{\theta } \mid \theta >0\}$ satisfies Assumption (18).

To show $\mathcal {U}_{f} \subseteq \mathcal {U}_{g}$, let $U(X) \in \mathcal {U}_{f}$. Then we have for all $\theta \in \varTheta$,

$$\begin{aligned} E_{f_{\theta }}(U) = \int _{x \in \mathbb {R}} U(t) \; \exp (-t^{2}/2\theta ^{2}) \; \hbox {d}t=0. \end{aligned}$$

(107)

By differentiating (107) both sides with respect to $\theta$, we get

$$\begin{aligned} \int _{x \in \mathbb {R}} U(t) \; t^{2} \; \exp (-t^{2}/2\theta ^{2}) \; \hbox {d}t=0. \end{aligned}$$

(108)

Now from Eqs. (107) and (108), it follows that

$$\begin{aligned} E_{g_{\theta }}(U)= & {} \frac{3}{4\sqrt{2 \pi } \theta } \int _{x \in \mathbb {R}} U(t) \; \exp (-t^{2}/2\theta ^{2}) \; \hbox {d}t \end{aligned}$$

(109)

$$\begin{aligned}&+\,\frac{3}{4\sqrt{2 \pi } \theta ^{3}} \int _{x \in \mathbb {R}} U(t) \; t^{2} \; \exp (-t^{2}/2\theta ^{2}) \; \hbox {d}t \end{aligned}$$

(110)

$$\begin{aligned}= & {} 0. \end{aligned}$$

(111)

Hence, $\{ g_{\theta } \mid \theta >0\}$ satisfies Assumption (19). $\square$