Notations

• $m$: number of multiple imputations
• $Y$: data of the sample
  • Includes both covariates and response
  • Dimension $n\times p$
• $R$: observation indicator matrix, known
  • A $n\times p$ 0-1 matrix
  • $r_{ij}=0$ for missing and 1 for observed
• $Y_{\text{obs}}$: observed data
• $Y_{\text{mis}}$: missing data

• $Y=(Y_{\text{obs}},Y_{\text{mis}})$: complete data

• $\psi$: the parameter for the missing mechanism

• $\theta$: the parameter for the full data $Y$

Concepts of MCAR, MAR, and MNAR, with notations

• Missing completely at random (MCAR) $$P(R=0\mid Y_{\text{obs}},Y_{\text{mis}},\psi )=P(R=0\mid \psi )$$

• Missing at random (MAR) $$P(R=0\mid Y_{\text{obs}},Y_{\text{mis}},\psi )=P(R=0\mid Y_{\text{obs}},\psi )$$

• Missing not at random (MNAR) $$P(R=0\mid Y_{\text{obs}},Y_{\text{mis}},\psi )\text{does not simplify}$$

Ignorable

• The missing data mechanism is ignorable for likelihood inference (on $\theta$), if

  1. MAR, and
  2. Distinctness: the parameters $\theta$ and $\psi$ are independent (from a Bayesian’s view)

• If the nonresponse if ignorable, then $$P(Y_{\text{mis}}\mid Y_{\text{obs}},R)=P(Y_{\text{mis}}\mid Y_{\text{obs}})$$ Thus, if the missing data model is ignorable, we can model $\theta$ just using the observed data

Goal of multiple imputation

• Note: for most multiple imputation practice, this goal is to train a (predictive) model with as small variances of the parameters as possible

• $Q$: estimand (the parameter to be estimated)
• $\hat{Q}$: estimate
  • Unbias $$E(\hat{Q}\mid Y)=Q$$
  • Confidence valid: $$E(U\mid Y)\ge V(\hat{Q}\mid Y)$$ where $U$ is the estimated covariance matrix of $\hat{Q}$, the expectation is over all possible samples, and $V(\hat{Q}\mid Y)$ is the variance caused by the sampling process

Within-variance and between-variance

$$\begin{array}{cc}E(Q\mid Y_{\text{obs}})& =E_{Y_{\text{mis}}\mid Y_{\text{obs}}}\left\{E\right(Q\mid Y_{\text{obs}},Y_{\text{mis}}\left)\right\}\\ V(Q\mid Y_{\text{obs}})& =\underset{\text{within-variance}}{\underset{}{E_{Y_{\text{mis}}\mid Y_{\text{obs}}}\left\{V\right(Q\mid Y_{\text{obs}},Y_{\text{mis}}\left)\right\}}}+\underset{\text{between variance}}{\underset{}{V_{Y_{\text{mis}}\mid Y_{\text{obs}}}\left\{E\right(Q\mid Y_{\text{obs}},Y_{\text{mis}}\left)\right\}}}\end{array}$$

• Within-variance: average of the repeated complete-data posterior variance of $Q$, estimated by $$\overline{U}=\frac{1}{m}\sum _{l=1}^m{\overline{U}}_l,$$ where ${\overline{U}}_l$ is the variance of ${\hat{Q}}_l$ in the $l$th imputation

• Between-variance: variance between the complete-data posterior means of $Q$, estimated by $$B=\frac{1}{m-1}\sum _{l=1}^m({\hat{Q}}_l-\overline{Q}){({\hat{Q}}_l-\overline{Q})}^{\prime },\overline{Q}=\frac{1}{m}\sum _{l=1}^m{\hat{Q}}_l$$

Decomposition of total variation

• Since $\overline{Q}$ is estimated using finite $m$, the contribution to the variance is about $B/m$. Thus, the total posterior variance of $Q$ can be decomposed into three parts: $$T=\overline{U}+B+B/m=\overline{U}+(1+\frac{1}{m})B$$

• $\overline{U}$: the conventional variance, due to sampling rather than getting the entire population.

• $B$: the extra variance due to missing values

• $B/m$: the extra simulation variance because $\overline{Q}$ is estimated for finite $m$

  • Traditionally choices are $m=3,5,10$, but the current advice is to use a larger $m$, e.g., $m=50$

Properness of an imputation procedure

• An imputation procedure is confidence proper for complete-data statistics $\hat{Q},U$, if it satisfies the following three conditions approximately at large $m$ $$\begin{array}{cc}E(\overline{Q}\mid Y)& =\hat{Q}\\ E(\overline{U}\mid Y)& =U\\ (1+\frac{1}{m})E(B\mid Y)& \ge V\left(\overline{Q}\right)\end{array}$$

  • Here $\hat{Q}$ is the complete-sample estimator of $Q$, and $U$ is its covariance
  • If we replace the $\ge$ by $>$ in the third formula, then the procedure is said to be proper
  • It is not always easy to check whether a procedure is proper.

Scope of the imputation model

• Broad: one set of imputations to be used for all projects and analyses

• Intermediate: one set of imputations per project and use this for all analyses

• Narrow: a separate imputed dataset is created for each analysis

• Which one is better: depends on the use case

Variance ratios

• Proportion of variation attributable to the missing data $$\lambda =\frac{B+B/m}{T}$$
  • If $\lambda >0.5$, then the influence of the imputation model on the final result is larger than that of the complete-data model

• Relative increase in variance due to nonresponse $$r=\frac{B+B/m}{\overline{U}}=\frac{\lambda }{1-\lambda }$$

• Fraction of information about $Q$ missing due to nonresponse $$\gamma =\frac{r+2/(\nu +3)}{1+r}=\frac{\nu +1}{\nu +3}\lambda +\frac{2}{\nu +3}$$
  • Here, $\nu$ is the degrees of freedom (see next)
  • When $\nu$ is large, $\gamma$ is very close to $\lambda$

Degrees of freedom (df)

• The degrees of freedom is the number of observations after accounting for the number of parameters in the model.

• The “old” formula (as in Rubin 1987): may produce values larger than the sample size in the complete data $${\nu }_{\text{old}}=(m-1)(1+\frac{1}{r^2})=\frac{m-1}{{\lambda }^2}$$

• Let ${\nu }_{\text{com}}$ be the conventional df in a complete-data inference problem. If the number of parameters in the model is $k$ and the sample size is $n$, then ${\nu }_{\text{com}}=n-k$. The estimated observed data df that accounts for the missing information is $${\nu }_{\text{obs}}=\frac{{\nu }_{\text{com}}+1}{{\nu }_{\text{com}}+3}{\nu }_{\text{com}}(1-\lambda )$$

• Barnard-Rubin correction: the adjusted df to be used for testing in multiple imputation is $$\nu =\frac{{\nu }_{\text{old}}{\nu }_{\text{obs}}}{{\nu }_{\text{old}}+{\nu }_{\text{obs}}}$$

A numerical example

## Load the mice package
library(mice); 
imp <- mice(nhanes, print = FALSE, m = 10, seed = 24415)
fit <- with(imp, lm(bmi ~ age))
est <- pool(fit); print(est, digits = 2)

## Class: mipo    m = 10 
##          term  m estimate ubar    b   t dfcom   df  riv lambda  fmi
## 1 (Intercept) 10     30.8  3.4 2.52 6.2    23  9.2 0.82   0.45 0.54
## 2         age 10     -2.3  0.9 0.39 1.3    23 12.3 0.48   0.32 0.41

• Columns ubar, b, and t are the variances
• Column dfcom is ${\nu }_{\text{com}}$
• Column df is the Barnard-Rubin correction $\nu$

T-test for regression coefficients

• Use the Barnard-Rubin correction of $\nu$ as the shape parameter of t-distribution.

print(summary(est, conf.int = TRUE), digits = 1)

##          term estimate std.error statistic df p.value 2.5 % 97.5 %
## 1 (Intercept)       31         2        12  9   5e-07    25   36.4
## 2         age       -2         1        -2 12   7e-02    -5    0.2

Imputation evaluation criteria

• The following criteria are useful in simulation studies (when you know the true $Q$)

1. Raw bias (RB): upper limit $5\%$ $$\text{RB}=\left|\frac{E\left(\overline{Q}\right)-Q}{Q}\right|$$

2. Coverage rate (CR): A CR below $90\%$ for the nominal $95\%$ interval is bad

3. Average width (AW) of confidence interval

4. Root mean squared error (RMSE): the smaller the better $$\text{RMSE}=\sqrt{{(E\left(\overline{Q}\right)-Q)}^2}$$

Imputation is not prediction

• Shall we evaluate an imputation method by examine how it can closely recover the missing values?
  • For example, using the RMSE to see if the imputed values ${\dot{y}}_i$ are close to the true (removed) missing data $y_i^{\text{mis}}$? $$\text{RMSE}=\sqrt{\frac{1}{n_{\text{mis}}}\sum _{i=1}^{n_{\text{mis}}}{(y_i^{\text{mis}}-{\dot{y}}_i)}^2}$$
• NO! This will favor least squares estimates, and it will find the same values over and over; and thus it is single imputation. This ignores the inherent uncertainty of the missing values.

When not to use multiple imputation

• For predictive modeling, if the missing values are in the target variable $Y$, then complete-case analysis and multiple imputation are equivalent.

• Two special cases where listwise deletion is better than multiple imputation

1. If the probability to be missing does not depend on $Y$

2. If the complete data model is logistic regression, and the missing data are confined to $Y$, not $X$

References

• Van Buuren, S. (2018). Flexible Imputation of Missing Data, 2nd Edition. CRC press.

  • https://stefvanbuuren.name/fimd/

• Rubin, D. (1987). Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons.