Minimax Linear Estimation with the Probability Criterion under Unimodal Noise and Bounded Parameters

Abstract

We consider a linear regression model with a vector of bounded parameters and a centered noise vector that has an uncertain unimodal distribution but known covariance matrix. We pose the minimax estimation problem for a linear combination of unknown parameters with the use of the probability criterion. The minimax estimate is determined as a result of minimizing a probability bound over the region of possible values of the variance and squared bias for all possible linear estimates. We establish that the resulting robust solution is less conservative in comparison with wider classes of distributions.

Appendix

Proof of Theorem 1. Consider the linear estimate $\tilde{X}$ defined by the coefficient vector $f\in {{\mathbb{R}}}^{n}$ and the bias $c\in {\mathbb{R}}$ according to (5).

If the estimated value X and the observation vector Y satisfy the equations of the regression model (1) with parameter vector θ ∈ Θ and noise vector $\eta \sim {\mathcal{H}}(K)$, then error $\tilde{X}-X$ admits the representation ε + r, where $\varepsilon \sim {\mathcal{H}}(d)$, and d, r satisfy the relations (9). Therefore, Eq. (11) has the inequality sign “⩽.”

To prove the opposite inequality, it suffices for a given vector of parameters θ ∈ Θ and a random variable $\varepsilon \sim {\mathcal{H}}(d)$, where d and r have the form (9), to choose a random vector $\eta \sim {\mathcal{H}}(K)$ satisfying equality $\varepsilon +r=\tilde{X}-X$ with probability 1. By virtue of (1) and (9) the required equality is equivalent to the following:

$$\varepsilon =\langle f,\eta \rangle .$$

Acting in the same way as in [26], we define the desired vector by the rule

$$\eta ={K}^{1/2}\left\{\varepsilon | g{| }^{-2}g+P\zeta \right\},$$

where P = I_n − ∣g∣⁻²gg^*, g = K^1/2f, I_n is the identity matrix of size n × n, and ζ is the standard n-dimensional Gaussian random vector independent of ε. Checking conditions ε = ⟨f, η⟩, Mη = 0 and cov{η, η} = K is identical to the calculations from [26].

In case when ${\mathcal{H}}={\mathcal{P}}$, the proof ends here.

With ${\mathcal{H}}={\mathcal{U}}$ it remains to verify that the distribution of the vector η is symmetric and linearly unimodal. According to [12], this condition means that for any choice of the coefficient vector $b\in {{\mathbb{R}}}^{n}$ the distribution of the linear combination

$$\langle b,\eta \rangle =\varepsilon | g{| }^{-2}\langle b,Tg\rangle +\langle b,TP\zeta \rangle $$

is symmetric and unimodal. This fact follows from the unimodality of the convolution of two symmetric unimodal one-dimensional distributions that are the distributions of both terms due to $\varepsilon \sim {\mathcal{U}}(d)$ and $\zeta \sim {\mathcal{N}}(0,{I}_{n})$ (see Theorem 1.6 from the same source).

Proof of Theorem 2. The convexity of the region Q follows directly from the convexity in $(f,c)\in {{\mathbb{R}}}^{n}\times {\mathbb{R}}$ of two functions

$$\langle Kf,f\rangle \quad \,\text{and}\,\quad \quad \mathop{\max }\limits_{\theta \in \Theta }{(c+\langle {B}^{* }f-a,\theta \rangle )}^{2}.$$

Therefore, the function ρ(d) as the lower boundary of the convex set Q will be convex as well (see Theorem I.5.3 from [27]). And since it is finite everywhere, it will be continuous.

The second statement follows from the definition of a support line for a convex set. With a fixed λ ⩾0, the straight line ρ + λd = γ_λ is a support line to the set Q at the point (d_λ, ρ_λ) if the linear form ρ + λd reaches a minimum (or maximum) over Q at the specified point. The case of a maximum can be discarded since this linear form on the region Q is not bounded from above. Thus, we obtain the required facts (19) and (20).

Proof of Theorem 3. Theorem 1 due to the monotonic dependence of ${\pi }_{h}^{{\mathcal{H}}}(d,\rho )$ on ρ implies that

$$\mathop{\sup }\limits_{\theta \in \Theta }\mathop{\sup }\limits_{\eta \sim {\mathcal{H}}(K)}{\mathtt{P}}\left\{| \tilde{X}-X| \geqslant h\right\}={\pi }_{h}^{{\mathcal{H}}}(d,\rho ),$$

(A.1)

where d = ⟨Kf, f⟩ and $\rho =\mathop{\sup }\limits_{\theta \in \Theta }{\left(c+\langle {B}^{* }f-a,\theta \rangle \right)}^{2}$.

Let $(\hat{d},\hat{\rho })$ be the minimum point of ${\pi }_{h}^{{\mathcal{H}}}(d,\rho )$ over (d, ρ) ∈ Q, and let $(\hat{f},\hat{c})$ be the solution to the minimax problem (19) with parameter λ equal to the slope of the support line for the set Q at point $(\hat{d},\hat{\rho })$. Then, according to Theorem 2, it holds that

$$\hat{d}=\langle K\hat{f},\hat{f}\rangle ,\qquad \hat{\rho }=\mathop{\max }\limits_{\theta \in \Theta }{\left(\hat{c}+\langle {B}^{* }\hat{f}-a,\theta \rangle \right)}^{2}.$$

Therefore, the estimate $\hat{X}=\langle \hat{f},Y\rangle +\hat{c}$ realizes the equality

$$\mathop{\sup }\limits_{\theta \in \Theta }\mathop{\sup }\limits_{\eta \sim {\mathcal{H}}(K)}{\mathtt{P}}\left\{| \hat{X}-X| \geqslant h\right\}={\pi }_{h}^{{\mathcal{H}}}(\hat{d},\hat{\rho }).$$

(A.2)

Now, since ${\pi }_{h}^{{\mathcal{H}}}(\hat{d},\hat{\rho })\leqslant {\pi }_{h}^{{\mathcal{H}}}(d,\rho )$, we see that the left-hand side of (A.2) does not exceed the left-hand side of (A.1). Therefore, $\hat{X}$ is a minimax estimate by the probability criterion on the class ${\mathcal{H}}(K)$, as needed.