About the Equivalence of the Latent D-Scoring Model and the Two-Parameter Logistic Item Response Model

Abstract

This article shows that the recently proposed latent D-scoring model of Dimitrov is statistically equivalent to the two-parameter logistic item response model. An analytical derivation and a numerical illustration are employed for demonstrating this finding. Hence, estimation techniques for the two-parameter logistic model can be used for estimating the latent D-scoring model. In an empirical example using PISA data, differences of country ranks are investigated when using different metrics for the latent trait. In the example, the choice of the latent trait metric matters for the ranking of countries. Finally, it is argued that an item response model with bounded latent trait values like the latent D-scoring model might have advantages for reporting results in terms of interpretation.

1. Introduction

Item response theory (IRT; [1]) is the statistical analysis of test items in education, psychology, and other fields of social sciences. Typically, a number of test items are administered to test takers, and the interest is to infer the ability (performance or trait) of them. IRT models relate observed item responses to unobserved latent traits. Because the latent trait is unobserved, there are many plausible choices for modeling these relationships. The most popular class of IRT models comprises logistic IRT models [2]. Recently, in a series of papers, Dimitrov proposed an alternative IRT model, the so-called latent D-scoring model [3]. The main goal of this paper is to demonstrate that the newly proposed IRT model is statistically equivalent to the well-established two-parameter logistic IRT model.

The paper is structured as follows. In Section 2, IRT models are introduced in their general form. Afterward, the logistic IRT model and the latent D-scoring model are discussed. In Section 3, we show the statistical equivalence of the latent D-scoring model and the logistic IRT model utilizing an analytical derivation and a numerical illustration. Furthermore, we study the properties of the two models. Section 4 presents an empirical example that compares outcomes of the two different modeling strategies and compares them with two alternative parameterizations of the latent trait. Finally, the article closes with a discussion.

2. Item Response Modeling

In Section 2.1, we discuss the indeterminacy of the latent trait in IRT models. In Section 2.2, we focus on the logistic IRT model and its estimation. As an alternative IRT model, the latent D-scoring model is introduced in Section 2.3.

2.1. Indeterminacy of the Latent Trait in IRT Models

A unidimensional IRT model for dichotomous item responses $X_i\in \{0,1\}$ is a statistical model [2]

$$\mathrm{P}(\mathit{X}=\mathit{x})={\int }_{-\infty }^{\infty }\prod _{i=1}^I\left[P_i{\left(\theta \right)}^{x_i}{\left(1-P_i\left(\theta \right)\right)}^{1-x_i}\right]f\left(\theta \right)\mathrm{d}\theta ,\theta \sim F,$$

(1)

where f denotes the density function of the latent variable $\theta$ (also denoted as the latent trait), and $P_i(x,\theta )=\mathrm{P}(X_i=x|\theta )$ denotes the item response function of item i. Note that items $i=1,\dots ,I$ are conditionally independent given the latent trait $\theta$. The model parameters in Equation (1) are typically not uniquely defined. Assume that one utilizes a monotone function $m:\mathbb{R}\to (0,1)$ for defining a transformed latent trait $\delta$ by $\delta =m\left(\theta \right)$. For example, m could be the logistic function $\Psi \left(x\right)={\left[1+exp(-x)\right]}^{-1}$ that maps the real line onto the unit interval $(0,1)$. Define $P_i^{*}\left(\delta \right)=P_i\left(m^{-1}\left(\delta \right)\right)$, where $m^{-1}$ denotes the inverse function of m. Furthermore, denote by g the density function of the transformed latent trait $\delta$. The IRT model in Equation (1) can be equivalently written as

$$\mathrm{P}(\mathit{X}=\mathit{x})={\int }_0^1\prod _{i=1}^I\left[P_i^{*}{\left(\delta \right)}^{x_i}{\left(1-P_i^{*}\left(\delta \right)\right)}^{1-x_i}\right]g\left(\delta \right)\mathrm{d}\delta .$$

(2)

The density g can be obtained from the density f by applying the density transformation theorem

$$g\left(\delta \right)=\frac{f\left(m^{-1}\left(\delta \right)\right)}{m^{\prime }\left(m^{-1}\left(\delta \right)\right)},$$

(3)

where $m^{\prime }=\frac{\mathrm{d}m}{\mathrm{d}x}$ is the derivative of m with respect to $\theta$.

It could be argued that only ordinal information can be extracted from the latent trait $\theta$ because the general IRT model (1) is only identified up to monotone transformations [4,5,6,7]. The indeterminacy of the latent trait metric implies that a researcher can seek a transformation $m\left(\theta \right)$ for the sake of enhancing interpretations of the results. One possible transformation is the true score metric $\tau =\tau \left(\theta \right)$ [2] that maps the $\theta$ metric from the real line to the bounded interval $(0,1)$ by defining

$$\tau \left(\theta \right)=\frac{1}{I}\sum _{i=1}^I{P}_i\left(\theta \right).$$

(4)

For a fixed value of $\theta$, $\tau =\tau \left(\theta \right)$ is the expected value of the proportion of correctly solved items. Another alternative is the rank score metric $\rho =\rho \left(\theta \right)$ [7] that is defined by

$$\rho \left(\theta \right)=F\left(\theta \right),$$

(5)

where F is the distribution function of $\theta$. One can show that $\rho$ follows a uniform distribution (hence, the label “rank score”):

$$\mathrm{P}(\rho \le u)=\mathrm{P}(F\left(\theta \right)\le u)=\mathrm{P}(\theta \le F^{-1}\left(u\right))=F\left(F^{-1}\left(u\right)\right)=u,0<u<1.$$

(6)

2.2. Logistic Item Response Model

An important class of IRT models is the class of logistic IRT models. Logistic IRT models employ the logistic link function for parameterizing IRFs. The IRFs in the two-parameter logistic (2PL) model [8] are given by

$$\mathrm{P}(X_i=1|\theta )=P_i\left(\theta \right)=\frac{1}{1+exp(-a_i(\theta -b_i))}=\Psi \left(a_i(\theta -b_i)\right),\theta \sim F,$$

(7)

where $a_i$ are item discriminations, and $b_i$ are item difficulties. The one-parameter logistic (1PL) model (Rasch model; [9]) is obtained by setting all item discriminations equal to one (i.e., $a_i=1$ for $i=1,\dots ,I$).

In Figure 1, IRFs of seven items of the 2PL model are displayed (see the figure legend for item parameters $a_i$ and $b_i$). It can be seen that items with higher item discriminations $a_i$ have steeper slopes. Additionally, items with larger item difficulties $b_i$ are shifted to the right. A fundamental property of IRFs in the 2PL model is a lower asymptote of zero and an upper asymptote of one. Hence, persons with very low abilities ($\theta \to -\infty$) have almost zero probability of correctly solving any item in the test, while highly able persons ($\theta \to \infty$) correctly solve items with a probability of one. Alternative IRT models allow lower and upper asymptotes different from 0 or 1, respectively [10].

In many applications, a normal distribution $\mathrm{N}(\mu ,{\sigma }^2)$ for the latent trait $\theta$ is assumed [7]. However, more flexible distributions or semiparametric specifications are possible [11,12]. Identification constraints are required in the 1PL and 2PL models for the estimation of model parameters. In the 1PL model, one can identify the model by setting $\mu =0$ or fixing an item difficulty of a reference item to 0 (or to a prespecified value). Alternatively, one can constrain the sum of the item difficulties equal to zero. In the 2PL model, identification can be ensured by posing a standard normal distribution $\mathrm{N}(0,1)$ (i.e., $\mu =0$ and $\sigma =1$). Alternatively, a reference item $i_0$ can be chosen for which $a_{i_0}=1$ and $b_{i_0}=0$ are used as fixed values in the estimation. Using a reference item has the advantage that the distribution F of $\theta$ can be flexibly estimated without using constraints on some parameters of F.

The 1PL model or the 2PL model can be estimated using marginal maximum likelihood (MML) or joint maximum likelihood (JML) estimation [2]. It is noteworthy that ${\sum }_{i=1}^I{X}_i$ is a sufficient statistic for $\theta$ in the 1PL model, while ${\sum }_{i=1}^I{a}_i{X}_i$ is the corresponding sufficient statistic in the 2PL model. Hence, the different models imply different interpretations and implications of the trait because the contribution of items to the variable of interest differs considerably [13].

2.3. Dimitrov’s Latent D-Scoring Model

Dimitrov proposes an alternative IRT model that has a bounded metric for the latent trait. His latent D-scoring (LDS) model [3,14] includes a latent trait $\delta$ that takes values in the interval $(0,1)$. The IRF in the LDS model is given as [15]

$$\mathrm{P}(X_i=1|\delta )=\frac{1}{1+{\left[\frac{1-\delta }{\delta }\frac{{\beta }_i}{1-{\beta }_i}\right]}^{{\alpha }_i}},\delta \sim G,$$

(8)

where G is some distribution on $(0,1)$. Item discriminations ${\alpha }_i$ are non-negative and indicate the extent to which item i measures the trait $\delta$. Item difficulties ${\delta }_i$ range between 0 and 1 and primarily determine the proportion of correctly solving item i. The IRF in Equation (8) is also referred to as the rational function model with two item parameters [15]. The IRFs of the LDS model for seven items are shown in Figure 2 (see the figure legend for item parameters ${\alpha }_i$ and ${\beta }_i$).

The LDS model with one item parameter is obtained by setting ${\alpha }_i=1$ [15]:

$$\mathrm{P}(X_i=1|\delta )=\frac{1}{1+\frac{1-\delta }{\delta }\frac{{\beta }_i}{1-{\beta }_i}},\delta \sim G.$$

(9)

The LDS model with three item parameters that accommodates guessing effects is defined as [15]

$$\mathrm{P}(X_i=1|\delta )={\gamma }_i+(1-{\gamma }_i)\frac{1}{1+{\left[\frac{1-\delta }{\delta }\frac{{\beta }_i}{1-{\beta }_i}\right]}^{{\alpha }_i}},\delta \sim G.$$

(10)

In the following, we mainly consider the case of the LDS model with two item parameters.

The LDS model can be estimated with MML [3] or JML [16]. In Section 3, we show that identification constraints are needed for the estimation of the model. The latent D-scoring model is applied in psychometric areas of linking and equating [16], differential item functioning [14], and the development of multistage tests [17].

3. Relation of the Latent D-Scoring Model and the 2PL Model

In this section, we show the close correspondence of the 2PL model and the LDS model. It is demonstrated that the two models are equivalent using analytical (Section 3.1) and numerical (Section 3.2) arguments. However, the two models imply different consequences regarding measurement precision and interpretations (Section 3.3). Finally, we propose an extension of the LDS model to multiple dimensions in Section 3.4.

3.1. Equivalence of the Latent D-Scoring Model and the 2PL Model

In this subsection, we analytically show that the LDS model is statistically equivalent to the 2PL model. Consequently, the model parameters of the 2PL model can be transformed to obtain model parameters of the LDS model.

The IRF of the LDS model (Equation (8)) can be rewritten as

$$\mathrm{P}(X_i=1|\delta )=\frac{1}{1+exp\left(-{\alpha }_i\left[log\frac{\delta }{1-\delta }-log\frac{{\beta }_i}{1-{\beta }_i}\right]\right)},\delta \sim G,$$

(11)

where G is the distribution function of $\delta$. By defining $\theta =log\frac{\delta }{1-\delta }$, $b_i=log\frac{{\beta }_i}{1-{\beta }_i}$, and $a_i={\alpha }_i$, one can rephrase the LDS model in Equation (11) as the 2PL model. Equivalently, we can write $\delta =\Psi \left(\theta \right)={[1+exp(-\theta )]}^{-1}$ as the logistic transform of $\theta$. Note that the logistic transform of $\delta =\Psi \left(\theta \right)$ was also discussed in [6,7]. Hence, the LDS model is a reparametrization of the 2PL model. Hence, estimation routines for the 2PL model can be used for estimating the latent D-scoring model, and item parameters are transformed afterward; that is, ${\alpha }_i=a_i$ and ${\beta }_i=\Psi \left(b_i\right)$.

Our derivation also implies that the LDS model with one item parameter is equivalent to the 1PL model. Moreover, the LDS model with three parameters is equivalent to the three-parameter logistic IRT model.

The distribution of $\delta$ can also be derived from the distribution of $\theta$. The density function g of $\delta$ can be obtained from the density function f of $\theta$ by applying Equation (3)

$$g\left(\delta \right)=\frac{1}{\delta (1-\delta )}f\left(log\frac{\delta }{1-\delta }\right).$$

(12)

Conversely, the density function $\theta$ can also be obtained from the density function of $\delta$ by

$$f\left(\theta \right)=\Psi \left(\theta \right)\left(1-\Psi \left(\theta \right)\right)g\left(\Psi \left(\theta \right)\right).$$

(13)

The estimation of the LDS model using software for the 2PL model requires a correct specification of the distribution for $\theta$. Suppose that a particular distributional assumption is posed on $\delta$ with density g. In that case, the estimation procedure must ensure that the assumed distribution for $\theta$ aligns with the implied density f for $\theta$ (see Equation (13)) to avoid biased item parameter estimates.

The variance–covariance matrix $\mathit{V}$ for item parameters in the 2PL model can be obtained utilizing the observed information matrix. The transformed item parameters for the LDS model emerge from a nonlinear transformation of the 2PL item parameters. Hence, the delta method can be applied for obtaining item parameters for the LDS model. In more detail, the matrix $\mathit{A}$ of derivatives of the transformed item parameters with respect to the 2PL item parameters is a diagonal matrix, and the variance–covariance matrix for transformed item parameters is $\mathit{A}\mathit{V}{\mathit{A}}^T$.

In Section 2.2, we showed that identification constraints are needed for estimating the 2PL model. Because the latent D-scoring model is equivalent to the 2PL model, the former also needs identification constraints. In the 2PL model, the location (i.e., the mean $\mu$) and the scale (i.e., the standard deviation $\sigma$) for the latent trait $\theta$ can be fixed in the estimation. This would translate into identification constraints for the LDS model. Alternatively, a reference item $i_0$ could be chosen for the LDS model with fixed parameters ${\alpha }_{i_0}=1$ and ${\beta }_{i_0}=0.5$.

3.2. Numerical Illustration

This subsection demonstrates that the LDS model can be estimated using software for the 2PL model. We used item parameters of $I=7$ items of the LDS model that were also used in Figure 2 (see also

Table 1. Estimated item parameters for the numerical illustration assuming a logit-normal distribution and a beta distribution.

Item	LDS		2PL		LogitN (0.6, 1.2${}^2$)		Beta (4,2)
	${\mathit{\alpha }}_{\mathit{i}}$	${\mathit{\beta }}_{\mathit{i}}$	${\mathit{a}}_{\mathit{i}}$	${\mathit{b}}_{\mathit{i}}$	${\widehat{\mathit{\alpha }}}_{\mathit{i}}$	${\widehat{\mathit{\beta }}}_{\mathit{i}}$	${\widehat{\mathit{\alpha }}}_{\mathit{i}}$	${\widehat{\mathit{\beta }}}_{\mathit{i}}$
1	0.50	0.10	0.50	−2.20	0.501	0.101	0.505	0.102
2	1.00	0.30	1.00	−0.85	1.002	0.301	0.988	0.296
3	0.50	0.50	0.50	$\phantom{-}$0.00	0.501	0.500	0.513	0.502
$\phantom{{}^{‡}}$4 ${}^{‡}$	1.00	0.50	1.00	$\phantom{-}$0.00	1.000	0.500	1.000	0.500
5	1.50	0.50	1.50	$\phantom{-}$0.00	1.503	0.500	1.472	0.499
6	1.50	0.70	1.50	$\phantom{-}$0.85	1.504	0.700	1.531	0.701
7	1.00	0.90	1.00	$\phantom{-}$2.20	1.002	0.900	1.071	0.892

Note. LDS = latent D-scoring model; 2PL = two-parameter logistic model; LogitN = logit-normal distribution; Beta = beta distribution; ^‡ = Item 4 was used as a reference item in estimation by fixing a₄ = 1 and b₄ = 0 (i.e., α₄ = 1 and β₄ = 0.50).

). The multivariate distribution of these $I=7$ items according to the LDS model can be written as

$$P(\mathit{X}=\mathit{x})={\int }_0^1\prod _{i=1}^I\left[P_i{(\delta ;{\alpha }_i,{\beta }_i)}^{x_i}{\left(1-P_i(\delta ;{\alpha }_i,{\beta }_i)\right)}^{1-x_i}\right]g\left(\delta \right)\mathrm{d}\delta ,$$

(14)

where $P_i(\delta ;{\alpha }_i,{\beta }_i)$ is the IRF for the ith item of the LDS model, and $\mathit{x}=(x_1,\dots ,x_I)$. Note that there are $2^I=128$ different item response patterns. The corresponding marginal probabilities $P(\mathit{X}=\mathit{x})$ are computed using (14) and numerical integration with respect to $\theta$. This numerical illustration aims to compute the multivariate distribution $P(\mathit{X}=\mathit{x})$ of $\mathit{X}$ in Equation (14) for specified item and distribution parameters and to show that the input parameters can be uniquely and correctly identified from $P(\mathit{X}=\mathit{x})$. This probability distribution corresponds to a population, i.e., a sample with an infinite sample size. Maximum likelihood estimation is applied to estimate the item and distribution parameters. Sampling variability does not play a role in estimated (i.e., identified) parameters by relying on the population. Hence, the illustration demonstrates the parameter equivalence of the 2PL and the LDS model at the population level. Note that no standard errors must be reported for item parameters because the data are defined at the population level.

In Section 3.1, we derived the transformation of item parameters from the 2PL model to the LDS model when showing statistical equivalence. Notably, for establishing statistical equivalence, the distribution G for $\delta$ is a transformation of the distribution of F for $\theta$. When item parameters of the LDS model are obtained with transformed item parameters from the 2PL model, it must be ensured that the distribution F of $\theta$ is correctly specified in the 2PL model. This means that F corresponds to the distribution G for $\delta$ that is used for generating data. Hence, we investigate whether distributional misspecifications of $\theta$ have consequences for transformed item parameters of the LDS model. We considered two distributions for $\delta$ in the data-generating model. First, $\delta$ followed a beta distribution Beta(4,2) [18]. Second, $\delta$ followed a logit-normal distribution LogitN ($0.6,1.2^2)$; that is $\theta =log\frac{\delta }{1-\delta }$ is a normal distribution with a mean of 0.6 and a standard deviation of 1.2 [19,20,21].

The 2PL model was estimated in the R [22] package sirt [23] using a sample weights option that inputs the item response pattern probabilities $P(\mathit{X}=\mathit{x})$. To avoid a restrictive distributional assumption on $\theta$, we used a fixed grid of 61 equidistant $\theta$ values ranging between $-6$ and 6 and assumed a normal distribution for $\theta$. The item parameters of the fourth item were fixed (i.e., $a_4=1$ and $b_4=0$ in the 2PL model, which corresponds to ${\alpha }_4=1$ and ${\beta }_4=0.5$ in the LDS model). The 2PL model was estimated using MML estimation and an EM algorithm [24]. Sample R code for the estimation is provided in Appendix A.

Results for this numerical illustration are presented in Table 1. It can be seen that estimated item parameters ${\widehat{\alpha }}_i$ and ${\widehat{\beta }}_i$ for the LDS model almost perfectly recover true values in the case of the logit-normal distribution. This finding can be expected because the log-linear smoothing approach includes the normal distribution as a particular instance (smoothing up to two moments). Slightly larger deviations were observed if the distribution for $\delta$ was a beta distribution. The logit transform of the beta distribution is not correctly represented by a normal distribution for $\theta$, which explains slight biases in item parameter estimates. For example, ${\widehat{\beta }}_7=0.892$ deviated from the true values ${\beta }_7=0.90$ and ${\widehat{\alpha }}_5=1.472$ deviated from ${\alpha }_5=1.50$. However, these numerical differences are probably negligible in practical applications and confirm our analytical reasoning for the equivalence of the 2PL and the LDS model.

3.3. Conditional Standard Errors for the Latent Trait

In this subsection, we study the amount of information for the latent trait that can be extracted with the 2PL model and the LDS model by using the concept of item information. Let ${\mathit{x}}_{pi}=(x_{p1},\dots ,x_{pI})$ denote the vector of item responses of person p. For IRFs $P_i$ (depending on already estimated item parameters), the maximum likelihood estimate ${\widehat{\theta }}_p$ for the latent trait of person p is given as [1]

$${\widehat{\theta }}_p=\underset{\theta }{arg\; max}\sum _{i=1}^I\left\{x_{pi}log{P}_i\left(\theta \right)+(1-x_{pi})log\left(1-P_i\left(\theta \right)\right)\right\}.$$

(15)

Hence, the standard error associated with the estimate ${\widehat{\theta }}_p$ is related to the information function that is obtained as the negative value of the second derivative of the log-likelihood function evaluated at ${\widehat{\theta }}_p$. The information that is provided by item i in (15) is then given as

$$-x_{pi}\frac{{\mathrm{d}}^2}{\mathrm{d}{\theta }^2}log{P}_i\left(\theta \right)-(1-x_{pi})\frac{{\mathrm{d}}^2}{\mathrm{d}{\theta }^2}log\left(1-P_i\left(\theta \right)\right).$$

(16)

This allows defining the (expected) item information $I_i$ for item i [25]

$$I_i\left(\theta \right)=-{\pi }_i\frac{{\mathrm{d}}^2}{\mathrm{d}{\theta }^2}log{P}_i\left(\theta \right)-(1-{\pi }_i)\frac{{\mathrm{d}}^2}{\mathrm{d}{\theta }^2}log\left(1-P_i\left(\theta \right)\right),$$

(17)

where ${\pi }_i=\mathrm{E}\left(X_i\right)$ is the expected value for item i. In the literature, the observed item information

$$O{I}_i\left(\theta \right)=-\frac{{\mathrm{d}}^2}{\mathrm{d}{\theta }^2}log{P}_i\left(\theta \right)$$

(18)

is often defined as the item information function. However, this function can become negative for some IRT models and the LDS model in particular [25], which is why preferring (17) for ensuring positivity of the item information function. For the 2PL model, the expected and observed item information coincide and are given as

$$I_i\left(\theta \right)=a_i^2{P}_i\left(\theta \right)\left(1-P_i\left(\theta \right)\right).$$

(19)

Equation (19) implies that the least information is available for extreme $\theta$ values (i.e., extremely negative or positive).

The test information $I\left(\theta \right)$ is defined as $I\left(\theta \right)={\sum }_{i=1}^I{I}_i\left(\theta \right)$. It quantifies the information that is provided by the test at each latent trait value $\theta$. The conditional standard error for the latent trait $\theta$ is given by $\mathrm{SE}\left(\theta \right)=1/\sqrt{I\left(\theta \right)}$.

One can similarly define the item information function for $\delta$ for the LDS model (see also [15]):

$$I_i\left(\delta \right)=-{\pi }_i\frac{{\mathrm{d}}^2}{\mathrm{d}{\delta }^2}log{P}_i\left(\delta \right)-(1-{\pi }_i)\frac{{\mathrm{d}}^2}{\mathrm{d}{\delta }^2}log\left(1-P_i\left(\delta \right)\right).$$

(20)

Analogously, the test information function $I\left(\delta \right)={\sum }_{i=1}^I{I}_i\left(\delta \right)$ can be defined for the latent trait $\delta$.

Because the latent D-scoring model is equivalent to the 2PL model (see Section 3.1), $\delta =\Psi \left(\theta \right)$ is a monotonous transformation of $\theta$, and the test information function for $\theta$ can be converted into the test transformation for $\delta$. More generally, let $\delta =m\left(\theta \right)$ be a monotone differentiable transformation. The test information function for $\delta$ can be computed from the test information function for $\theta$ (see [2]):

$$I\left(\delta \right)=I\left(m\left(\theta \right)\right)=\frac{1}{m^{\prime }\left(\theta \right)}I\left(\theta \right),$$

(21)

where $m^{\prime }=\frac{\mathrm{d}m}{\mathrm{d}\theta }$. Equation (21) can be rewritten for conditional standard errors as

$$\mathrm{SE}\left(\delta \right)=\mathrm{SE}\left(m\left(\theta \right)\right)=\sqrt{m^{\prime }\left(\theta \right)}\mathrm{SE}\left(\theta \right).$$

(22)

Hence, the conditional standard error $\mathrm{SE}\left(\delta \right)$ for the LDS model is given as

$$\mathrm{SE}\left(\delta \right)=\mathrm{SE}(\Psi \left(\theta \right))=\sqrt{\Psi \left(\theta \right)(1-\Psi (\theta \left)\right)}\mathrm{SE}\left(\theta \right).$$

(23)

In Section 3.2, we demonstrated that the LDS model is equivalent to the 2PL model. For the item parameters of the seven items used in the demonstration (see Table 1), the conditional standard errors for $\theta$ and $\delta$ are shown in Figure 3. It can be seen that the 2PL model measures the latent trait $\theta$ less precisely for extremely large negative and extremely large positive values; that is, for low- and high-achieving persons. In line with the results of [3], the converse holds for the LDS model. Conditional standard errors are smallest for persons with $\delta$ values near 0 or 1. Hence, statements about measurement precision in different ranges of values for the latent trait strongly depend on the chosen metric (see also [26]). Interestingly, the transformed latent trait $\xi =m\left(\theta \right)={\int }_{-\infty }^{\theta }\sqrt{I\left(u\right)}\mathrm{d}u$ (the so-called arc length metric; see [6]) has homogeneous standard errors among the latent trait

$$\mathrm{SE}\left(\xi \right)=\mathrm{SE}\left(m\right(\theta \left)\right)=1.$$

(24)

These observations indicate that it is difficult to state for which subgroups of persons adaptive or multistage testing [27] provides measurement precision gains because such statements depend on the chosen metric.

An anonymous reviewer presented an insightful explanation of the behavior of conditional standard errors. For people with a high D-score, a low standard error will result because one can be very confident that these persons will get a new but parallel item correct. In contrast, for persons with a high $\theta$ score, a high standard error will be observed because it is uncertain what the hardest item they could write is. Overall, the different scoring methods give different interpretations and, therefore, different interpretations to their respective standard errors.

3.4. A Multidimensional Latent D-Scoring Model

To our knowledge, the LDS model has only been investigated for a unidimensional latent variable $\delta$. However, in applications, multidimensional traits are often of interest [28,29]. We now show that an apparent extension of the LDS model to multiple dimensions can be obtained by using the same transformations of the multidimensional variant of the 2PL model. We illustrate the arguments for two dimensions ${\theta }_1$ and ${\theta }_2$.

The multidimensional logistic IRT model can be written as [29]

$$\mathrm{P}(X_i=1|{\theta }_1,{\theta }_2)=\frac{1}{1+exp(-a_{i1}{\theta }_1-a_{i2}{\theta }_2+d_i)},({\theta }_1,{\theta }_2)\sim F,$$

(25)

where F is a bivariate distribution of $({\theta }_1,{\theta }_2)$ and ${\theta }_d$ ($d=1,2$) attain values on the real line. Define transformed latent traits ${\delta }_d=\Psi \left({\theta }_d\right)={[1+exp(-{\theta }_d)]}^{-1}$ ($d=1,2$) as the logistic transformations of ${\theta }_d$. Like in the unidimensional LDS model, the ${\delta }_d$ variables attain values in the interval $(0,1)$. Note that the inverse transformation is given as ${\theta }_d=log\frac{{\delta }_d}{1-{\delta }_d}$. Then, employing the same strategy as in Section 3.1, one can rewrite Equation (25) by using ${\beta }_i=\Psi \left(d_i\right)$ and ${\alpha }_{id}=a_{id}$ as

$$\mathrm{P}(X_i=1|{\delta }_1,{\delta }_2)=\frac{1}{1+{\left[\frac{1-{\delta }_1}{{\delta }_1}\right]}^{{\alpha }_{i1}}{\left[\frac{1-{\delta }_2}{{\delta }_2}\right]}^{{\alpha }_{i2}}\frac{{\beta }_i}{1-{\beta }_i}},({\delta }_1,{\delta }_2)\sim G.$$

(26)

Hence, the multidimensional 2PL model can easily be reparametrized for defining a multidimensional LDS model. The generalization to more than two dimensions is straightforward. Given that multidimensional IRT models are more difficult to estimate than unidimensional IRT models, it is advantageous that existing software implementations of multidimensional logistic IRT models can be used for estimating a multidimensional variant of the LDS model.

4. Empirical Example: PISA 2006 Reading

4.1. Method

In order to illustrate the consequences of the choice of different metrics of the latent trait in multiple-group comparisons, we analyzed the data from the Programme for International Student Assessment (PISA) conducted in 2006 (PISA 2006; [30]). In this situation, groups constitute countries. We included 26 countries (see

Table 2. Country-level results for PISA 2006 reading for different ability metrics.

cnt	Country	N	M				SD				Rank M
			$\mathit{\theta }$	$\mathit{\delta }$	$\mathit{\tau }$	$\mathit{\rho }$	$\mathit{\theta }$	$\mathit{\delta }$	$\mathit{\tau }$	$\mathit{\rho }$	$\mathit{\theta }$	$\mathit{\delta }$	$\mathit{\tau }$	$\mathit{\rho }$	maxrk
KOR	South Korea	2790	$\phantom{-}$0.471	0.603	0.663	0.646	0.831	0.176	0.166	0.246	1	1	1	1	0
FIN	Finland	2536	$\phantom{-}$0.327	0.576	0.646	0.614	0.570	0.130	0.124	0.193	2	2	2	2	0
CAN	Canada	12,142$\phantom{8}$	$\phantom{-}$0.234	0.553	0.616	0.577	0.823	0.179	0.176	0.255	3	3	3	3	0
IRL	Ireland	2468	$\phantom{-}$0.170	0.538	0.599	0.554	0.911	0.193	0.192	0.272	4	4	4	4	0
AUS	Australia	7562	$\phantom{-}$0.144	0.534	0.596	0.550	0.876	0.188	0.189	0.267	5	5	5	5	0
SWE	Sweden	2374	$\phantom{-}$0.098	0.523	0.581	0.535	1.015	0.213	0.214	0.295	6	6	6	6	0
NLD	Netherlands	2666	$\phantom{-}$0.084	0.521	0.577	0.531	1.051	0.219	0.221	0.302	7	7	7	8	1
POL	Poland	2968	$\phantom{-}$0.065	0.515	0.573	0.521	0.981	0.209	0.211	0.293	8	9	8	9	1
BEL	Belgium	4840	$\phantom{-}$0.031	0.517	0.567	0.532	1.278	0.250	0.257	0.333	9	8	11	7	4
JPN	Japan	3203	$\phantom{-}$0.015	0.507	0.562	0.512	1.103	0.225	0.229	0.308	10	10	13	10	3
CHE	Switzerland	6578	$\phantom{-}$0.015	0.506	0.569	0.511	0.852	0.186	0.190	0.265	11	11	10	11	1
DNK	Danmark	2431	$\phantom{-}$0.008	0.502	0.566	0.503	0.828	0.181	0.183	0.260	12	12	12	14	2
EST	Estonia	2630	$\phantom{-}$0.002	0.501	0.571	0.501	0.616	0.142	0.143	0.211	13	13	9	15	6
GBR	Great Britain	7061	−0.028	0.498	0.557	0.500	0.989	0.206	0.211	0.286	14	15	15	16	2
FRA	France	2524	−0.039	0.500	0.559	0.508	1.004	0.206	0.215	0.285	15	14	14	13	2
ISL	Iceland	2010	−0.055	0.489	0.556	0.486	0.741	0.165	0.170	0.239	16	18	16	18	2
AUT	Austria	2646	−0.057	0.493	0.547	0.495	1.125	0.230	0.237	0.314	17	17	17	17	0
DEU	Germany	2701	−0.098	0.497	0.539	0.510	1.485	0.280	0.290	0.364	18	16	19	12	7
HUN	Hungary	2399	−0.110	0.477	0.544	0.468	0.694	0.156	0.163	0.229	19	20	18	20	2
NOR	Norway	2504	−0.135	0.479	0.535	0.478	1.079	0.221	0.231	0.303	20	19	21	19	2
ESP	Spain	10,506$\phantom{8}$	−0.168	0.460	0.535	0.440	0.432	0.102	0.108	0.155	21	23	20	23	3
LUX	Luxembourg	2443	−0.210	0.463	0.519	0.456	1.073	0.219	0.231	0.300	22	21	22	22	1
PRT	Portugal	2773	−0.219	0.455	0.517	0.439	0.863	0.185	0.195	0.262	23	24	23	24	1
CZE	Czech Republic	3246	−0.237	0.462	0.506	0.457	1.398	0.270	0.280	0.355	24	22	24	21	3
ITA	Italy	11,629$\phantom{8}$	−0.288	0.443	0.502	0.426	0.966	0.199	0.212	0.276	25	25	25	25	0
GRC	Greece	2606	−0.385	0.419	0.479	0.388	0.868	0.183	0.196	0.256	26	26	26	26	0

Note. cnt = country label; N = sample size per country; M = mean; SD = standard deviation; Rank M = country rank with respect to mean M; $\theta$ = logit ability metric from two-parameter logistic (2PL) model; $\delta$ = metric of the latent D-scoring (LDS) model; $\tau$ = true score metric; $\rho$ = rank score metric; maxrk = maximum rank difference among ability metrics $\theta$, $\delta$, $\tau$, and $\rho$.

) that participated in 2006 and focused on the reading test (see [31,32] for other studies using this dataset).

Items for the reading domain were only administered to a subset of the participating students. We included only those students who received a test booklet with at least one reading item. This resulted in a total sample size of 110,236 students (ranging from 2010 to 12,142 students between countries). In total, 28 reading items nested within eight reading texts were used in PISA 2006. Six of the 28 items were polytomous and were dichotomously recorded, with only the highest category being recorded as correct.

In all analyses, student weights were taken into account. Within a country, student weights were normalized to a sum of 5000, so that all countries contributed equally to the analyses.

In a first step, the 2PL model was estimated based on the data comprising students of all 26 countries. Student weights were taken into account, and a normal distribution was posed for $\theta$ in the estimation. The obtained item parameters ${\widehat{a}}_i$ and ${\widehat{b}}_i$ were fixed in the second step when estimating the trait distribution in each country. More concretely, the 2PL model was fitted using the R [22] package sirt [23] using MML estimation. The 2PL model was estimated by using a discrete grid of $T=121$ equidistant $\theta$ points ranging between $-6$ and 6 for numerical integration of the involved integrals in the log-likelihood function of the 2PL model. As in Section 3.2, log-linear smoothing up to four moments of the trait distribution [12] within a country was employed to allow non-normal distributions. Assume that the estimated parametric distribution for $\theta$ in country g is ${\pi }_{gt}=P({\theta }_t;{\mathbf{\delta }}_g)$ for grid values ${\theta }_t$ ($t=1,\dots ,T$) and country-specific distribution parameters ${\mathbf{\delta }}_g$. Afterward, individual posterior distributions $h_p\left({\theta }_t\right|{\mathit{x}}_p)$$(t=1,\dots ,T$) were computed as

$$h_p({\theta }_t|{\mathit{x}}_p)=\frac{{\displaystyle \prod _{i=1}^I\left[P_i{({\theta }_t;{\widehat{a}}_i,{\widehat{b}}_i)}^{x_{pi}}{\left(1-P_i({\theta }_t;{\widehat{a}}_i,{\widehat{b}}_i)\right)}^{1-x_{pi}}\right]{\pi }_{gt}}}{{\displaystyle \sum _{u=1}^T\prod _{i=1}^I\left[P_i{({\theta }_u;{\widehat{a}}_i,{\widehat{b}}_i)}^{x_{pi}}{\left(1-P_i({\theta }_u;{\widehat{a}}_i,{\widehat{b}}_i)\right)}^{1-x_{pi}}\right]{\pi }_{gu}}},$$

(27)

where $P_i({\theta }_t;{\widehat{a}}_i,{\widehat{b}}_i)$ is the IRF of item i from the 2PL model using estimated item parameters ${\widehat{a}}_i$ and ${\widehat{b}}_i$ from the total sample. By construction, it holds that ${\sum }_{t=1}^T{h}_p\left({\theta }_t\right|{\mathit{x}}_p)=1$. For $N_g$ persons per country g, the country means ${\widehat{\mu }}_{\theta ,g}$ on the logit metric $\theta$ was estimated by

$${\widehat{\mu }}_{\theta ,g}=\frac{1}{W}\sum _{p=1}^{N_g}{w}_p\sum _{t=1}^T{\theta }_t{h}_p\left({\theta }_t\right|{\mathit{x}}_p),$$

(28)

where the person weights $w_p$ sum to $W=5000$ within a country (i.e., ${\sum }_{p=1}^{N_g}{w}_p=W$). Country-specific standard deviations ${\widehat{\sigma }}_{\theta ,g}$ can be computed similarly:

$${\widehat{\sigma }}_{\theta ,g}=\sqrt{\frac{1}{W}\sum _{p=1}^{N_g}{w}_p\sum _{t=1}^T{\theta }_t^2{h}_p\left({\theta }_t\right|{\mathit{x}}_p)-{\widehat{\mu }}_{\theta ,g}^2}.$$

(29)

Besides the logit metric $\theta$, we also investigated the metric $\delta$ based on the LDS model, the true score metric $\tau$ (see Equation (4)), and the rank score metric $\rho$ (see Equation (5)). All three alternative metrics are monotone transformations $m\left(\theta \right)$ of $\theta$. The country mean ${\widehat{\mu }}_{m\left(\theta \right),g}$ at the transformed metric was calculated as

$${\widehat{\mu }}_{m\left(\theta \right),g}=\frac{1}{W}\sum _{p=1}^{N_g}{w}_p\sum _{t=1}^{T}m\left({\theta }_t\right)h_p\left({\theta }_t\right|{\mathit{x}}_p).$$

(30)

Using (30), the standard deviation of $m\left(\theta \right)$ can be computed similarly to (29). Furthermore, conditional standard errors for the four latent trait metrics are computed for the whole sample containing all students. The item information is obtained by using the second derivatives of IRFs with respect to the metrics $\theta$, $\delta$, $\tau$, and $\rho$ (see Equation (17)).

4.2. Results

In

Table A1. Estimated item parameters for the PISA 2006 reading dataset.

		2PL		LDS
Item	${\mathit{\pi }}_{\mathit{i}}$	${\mathit{a}}_{\mathit{i}}$	${\mathit{b}}_{\mathit{i}}$	${\mathit{\alpha }}_{\mathit{i}}$	${\mathit{\beta }}_{\mathit{i}}$
R055Q01	0.817	1.395	−1.486	1.395	0.185
R055Q02	0.480	1.379	$\phantom{-}$0.043	1.379	0.511
R055Q03	0.584	1.620	−0.334	1.620	0.417
R055Q05	0.719	2.118	−0.778	2.118	0.315
R067Q01	0.892	1.227	−2.072	1.227	0.112
R067Q04	0.382	0.832	$\phantom{-}$0.723	0.832	0.673
R067Q05	0.582	1.088	−0.307	1.088	0.424
R102Q04A	0.343	1.460	$\phantom{-}$0.669	1.460	0.661
R102Q05	0.457	1.330	$\phantom{-}$0.244	1.330	0.561
R102Q07	0.842	1.417	−1.493	1.417	0.183
R104Q01	0.816	1.627	−1.322	1.627	0.211
R104Q02	0.326	0.584	$\phantom{-}$1.333	0.584	0.791
R104Q05	0.046	1.132	$\phantom{-}$3.131	1.132	0.958
R111Q01	0.643	1.365	−0.604	1.365	0.353
R111Q02B	0.155	1.046	$\phantom{-}$1.912	1.046	0.871
R111Q06B	0.351	1.588	$\phantom{-}$0.542	1.588	0.632
R219Q01E	0.582	1.633	−0.250	1.633	0.438
R219Q01T	0.699	1.860	−0.664	1.860	0.340
R219Q02	0.792	1.534	−1.179	1.534	0.235
R220Q01	0.434	1.762	$\phantom{-}$0.305	1.762	0.576
R220Q02B	0.621	1.520	−0.376	1.520	0.407
R220Q04	0.596	1.302	−0.312	1.302	0.423
R220Q05	0.823	1.977	−1.145	1.977	0.241
R220Q06	0.669	1.167	−0.675	1.167	0.337
R227Q01	0.521	0.778	−0.151	0.778	0.462
R227Q02T	0.337	0.993	$\phantom{-}$0.793	0.993	0.688
R227Q03	0.546	1.664	−0.183	1.664	0.454
R227Q06	0.706	1.766	−0.777	1.766	0.315

Note. 2PL = two-parameter logistic model; LDS = latent D-scoring model; π_i = proportion correct.

of Appendix B, estimated item parameters ${\widehat{a}}_i$ and ${\widehat{b}}_i$ from the 2PL model are shown. These item parameters were transformed into parameters of the equivalent LDS model (see columns ${\widehat{\alpha }}_i$ and ${\widehat{\beta }}_i$ in Table A1). The IRFs of seven selected items are displayed in Figure A1 in Appendix B for the four latent trait metrics $\theta$, $\delta$, $\tau$ and $\rho$. IRFs for the bounded metrics $\delta$, $\tau$ and $\rho$ look very similar.

In Figure 4, the transformation functions $\delta =\delta \left(\theta \right)$, $\tau =\tau \left(\theta \right)$ and $\rho =\rho \left(\theta \right)$ are depicted. The latent D-score $\delta$ and the true score $\tau$ follow a very close transformation function. The rank score $\rho$ differs from the former two in the tails of the $\theta$ distribution. Hence, it can be expected that $\delta$ and $\tau$ provide similar country rankings, while using $\rho$ might lead to slightly different country rankings.

In Figure 5, conditional standard errors are displayed. It can be seen that $\theta$ has a U-shaped form, while the three other metrics are inverted U-shaped. Interestingly, the standard errors $\mathrm{SE}\left(\delta \right)$ and $\mathrm{SE}\left(\tau \right)$ approach 0 for $\delta$ or $\tau$ near to 0 or 1. This is not the case for the rank score metric $\rho$, for which standard errors for $\rho =0$ and $\rho =1$ are larger than 0. Assume that Country C1 is low-performing (negative $\theta$ value) and Country C2 has average performance ($\theta$ average of about 0). Then, it can be the case that the latent trait is less precisely assessed for Country C1 than for Country C2 in the $\theta$ metric but more precisely assessed for Country C1 than C2 in one of the three alternative metrics $\delta$, $\tau$, or $\rho$. These statements rely on the somewhat arbitrary choice of the latent trait metric used to quantify differences between countries.

Table 2 contains detailed results of means, standard deviations, and country ranks based on means for the 26 countries. For the first six high-performing countries, country ranks are the same for all four trait metrics. However, there are countries for which ranks differ considerably. Relatively large deviations are observed for Belgium (BEL; maximum rank difference (maxrk) of 4), Estonia (EST; makrk = 6), and Germany (DEU; maxrk = 7). The most crucial difference occurs for the $\tau$ and the $\rho$ metric. For the three mentioned countries, the standard deviation of $\theta$ was relatively low or high compared to all other countries in the sample. This observation explains the differences among ranks because the tails of the $\theta$ distributions are differently weighted (i.e., differently transformed) for $\tau$ and $\rho$.

Overall, the Spearman rank correlations of country means ranged between 0.949 (between $\tau$ and $\rho$) and 0.992 (between $\theta$ and $\delta$). The average rank difference of country means across different metrics was 2.000 (see column “maxrk” in Table 2; $\mathrm{SD}=1.853$, $\mathrm{Min}=0$, $\mathrm{Max}=7$). The Spearman rank correlations of country standard deviations ranged between 0.973 (between $\tau$ and $\rho$) and 0.999 (between $\delta$ and $\rho$). The average rank difference of country standard deviations across different metrics was 1.000 ($\mathrm{SD}=1.301$, $\mathrm{Min}=0$, $\mathrm{Max}=5$). To sum up, the choice of the ability metric can have relevance for some countries for the reporting of country means.

5. Discussion

This article shows that the newly proposed LDS model of Dimitrov can be interpreted as a reparametrization of the well-studied 2PL model. Hence, all established statistical techniques for the 2PL model can be used for practical applications of the LDS model. It has been shown that the latent trait score $\delta$ from the LDS model is a monotonous (logistic) transformation of the $\theta$ score from the 2PL model. All other psychometric areas such as differential item functioning, equating and linking, or test assembly must not be reinvented for the LDS model because known techniques for the 2PL model can be used.

Although these findings might be interpreted as somehow destructive for the research surrounding the LDS model, we do not think that the LDS model is not of interest at all. We wanted to argue that the choice latent trait metric is arbitrary in IRT models, and the $\theta$ or the $\delta$ metric can be both useful in applications. The authors of this paper tend to prefer bounded trait metrics in applications because it seems more challenging to interpret the possibility of unbounded negative and positive trait values of $\theta$ [33]. However, we would prefer the true score metric $\tau$ or the rank score $\rho$ over $\delta$. The latent D-score $\delta$ can be interpreted as a particular true score in which only a reference item with $a_i=1$ and $b_i=0$ is used. We believe that using a well-chosen reference test with its item parameters provides a better interpretable latent trait metric in practical applications. The rank score $\rho$ has the advantage that it does not depend on item parameters. For example, in the PISA study, one fixes the $\theta$ metric in the starting study (e.g., in PISA 2000) to a mean of 500 and a standard deviation of 100. Using the rank metric $\rho$ would imply that the metric is identified by assuming a uniform distribution on $[0,1]$ for identification. Both approaches might be legitimate in practical applications. Notably, linking and equating for bounded metrics are more difficult to conduct than for unbounded metrics. However, we would opt for using the unbounded metric from the 2PL model for the operational use for linking but bounded metrics for reporting ability distributions.

In IRT models, items are typically treated as fixed. However, they can alternatively be interpreted as exchangeable. Item sampling models [34,35,36] have fewer assumptions in this respect and could be alternatively employed in assessment studies.

The LDS model has been motivated as an IRT analog of the so-called manifest D-scoring method [37]. The scoring rule ${\sum }_{i=1}^I(1-{\pi }_i)X_i$ is used in this approach, where ${\pi }_i=\mathrm{P}(X_i=1)$ is the probability of getting item i correct. In manifest D-scoring, more difficult items receive larger weights. This property might have appeal in some applications. However, we believe that this scoring rule does not adequately represent all items in a test in typical assessment studies and might lead to country comparisons with reduced validity.