The method of envelopes to concisely calculate semiparametric efficient scores under parametric restrictions

1 The problem

Semiparametric problems are useful when their “restrictions” (i.e., assumptions about the distribution) reflect existing knowledge or a simplified expression of a complex reality. When addressing semiparametric problems, the efficient score (ES) of a parameter is often important for generating useful estimators (e.g., [1], [2], [3], [4]).

One important example is the “restricted conditional moment” problem, where we measure a random sample of predictors and a response, (X_i,Y_i), i = 1, … , n. In such a problem, we model the regression E(Yi|Xi)=g(Xi,β0) and we wish to estimate β₀; and the rest of the distribution is unspecified and indexed by λ₀ (e.g., [3]). Another important example is the “treatment effect regression” problem, where a random sample of patients with covariates X_i is assigned one of two treatments, T_i = 0 or 1, based on X_i, and outcome Y_i is measured. In such a problem, we often model the difference E(Yi|Ti=1,Xi)−E(Yi|Ti=0,Xi)=g(Xi,β0) and wish to estimate β₀; and the rest of the distribution is, again, unspecified and indexed by λ₀ (e.g., [5], [, 6]).

The above examples are cases of a more general problem in which we measure a random sample of data D_i, i = 1, … , n from an unknown true distribution pr₀(D_i); we assume pr₀(D_i) is characterized by a finite dimensional vector β₀ that we wish to estimate (say β0∈Setβ) ; and leave the rest of the distribution unspecified and indexed by λ₀ (say λ0∈Setλ). Such a semiparametric model is summarized (e.g., [1]) as:

Derivation of the ES, say Sβ eff(Di,β,λ), is useful because it can lead to efficient estimation of β. Specifically, under some regularity conditions, the estimator for β₀ that zeros out the empirical sum ∑iSβ eff(Di,β,λ) over individuals is consistent and efficient if λ, when needed, is replaced by a consistent estimator with convergence rates larger than n1/4 [4].

In what follows, by notation such as Sβ,i eff we mean the random variable Sβ eff(Di,β,λ), with the understanding that it arises from a deterministic function taking a data value and parameters β,  λ as arguments. The usual derivation of ES is summarized in the following steps (e.g., [2]):

1. consider a parametric submodel, say, Mssem indexed by β and a finite dimensional parameter λ_s;

2. find the scores Sλs;

3. find, across all submodels s, the set of all linear combinations of scores Sλs, and the closure of this set, known as the nuisance tangent space and labeled as Λ;

4. find the efficient score Sβ eff for β as the residual from the score S_β after projecting it on the space Λ.

2 The method of envelopes

The key idea here is that, instead of constructing the score for β with the “bottom up” argument (using smaller parametric submodels Mssem), it can be easier to construct it with a “top down” argument, using a model larger than Msem. To proceed, we assume the support of data D_i is discrete, even though the dimension can be large. This is often reasonable on its own, e.g., for arguments that take sample size to a limit but keep measurement instruments with fixed, finite number of values. Here, discreteness is used mostly as a device to uncover the form of the efficient score when this is understood to exist.

Consider the larger model, say Menv, that is as Msem except that it poses no restrictions using β. Consider then all those features of the distribution pr₀(D_i) in Menv that do get constrained by β in the semiparametric model Msem. Those features, which we denote by “env”, can be seen as envelopes, in the larger model Menv, of the consequences of placing the restrictions β in the semiparametric model Msem. Denote the larger model as:

We then have the following useful observations:

1. the meaning of the parameters λ and the set Setλ are the same in model Msem as the parameters λenv and the set Setλenv in model Menv; therefore, the efficient score for the envelope in Menv is

1. from the semiparametric model we obtain that Sβ=∂env∂β×Senv; here, ∂env∂β is a deterministic matrix of partial derivatives of the dim(env) envelopes with respect to the dim(β) parameters, evaluated at the true values.

2. substituting, using the last expression, ∂env∂β×Senv for Sβ in (2), we obtain the following connection between the two efficient scores:

Result. The efficient score Sβ eff in the smaller model can be obtained from the efficient score Senv eff in the larger model as

This last expression is important because it gives the efficient score Sβ eff as a simple function of the score for the envelopes Senv eff in the larger model, and bypasses long calculations within model Msem. The scores for Senv eff are often known either directly or through their relation to the efficient influence functions (EIF) ϕenv eff of the envelopes:

where the latter are easily derived as Gateaux derivatives [7]. Once the efficient scores are derived, the efficient influence function for β, say ϕβ eff can be derived in the usual way (e.g., [3]; p. 66) as

3 Examples

4 Comments

A conceptual comparison of the usual method to the envelope method to derive a semiparametric efficient score is the following. The usual method relates the semiparametric restricted model to smaller, submodels with more restrictions. The envelope method relates the restricted model to a larger model that relaxes those restrictions. Here, we showed how this latter relation can provide a much shorter derivation and understanding of the efficient scores for teaching and research.

Restrictions in semiparametric models can take forms other than through a finite dimensional parameter, for example through conditional independence or symmetry. If the functional of interest in the smaller model is only a function of easily estimable features of the larger, non-restricted model, then the envelope method can still be useful.

An alternative approach to calculating efficient scores is through Target Maximum Likelihood Estimation (TMLE, e.g., [9]). Ways to use TMLE while avoiding knowledge of the efficient influence curve have been explored in some cases, although their generality is still under investigation.