Conformal inference for individual treatment effects

conformalIte supports four algorithms: the nested approach with exact and inexact calibration for cases with both potential outcomes missing, the naive approach for cases with both potential outcomes missing and the counterfactual inference for cases with only one potential outcome missing. For each algorithm, it supports both split conformal inference and CV+, including weighted Jackknife+ as a special case. For each type, it supports both conformalized quantile regression (CQR) and standard conformal inference based on conditional mean regression.

conformalIte(
  X,
  Y,
  T,
  alpha = 0.1,
  algo = c("nest", "naive", "counterfactual"),
  exact = FALSE,
  type = c("CQR", "mean"),
  side = c("two", "above", "below"),
  quantiles = NULL,
  outfun = NULL,
  outparams = list(),
  psfun = NULL,
  psparams = list(),
  cfprop = 0.5,
  citype = c("CQR", "mean"),
  lofun = NULL,
  loquantile = 0.4,
  loparams = list(),
  upfun = NULL,
  upquantile = 0.6,
  upparams = list(),
  useCV = FALSE,
  trainprop = 0.75,
  nfolds = 10,
  wthigh = 20,
  wtlow = 0.05
)

Arguments

X

covariates.

Y

observed outcome vector.

T

treatment indicator, a binary vector.

alpha

confidence level.

algo

a string that takes values in {"nest", "naive", "counterfactual"}. See Details.

exact

a logical indicating whether the exact calibration is used for nested approach. Used only when algo = "nest". See Details.

type

a string that takes values in {"CQR", "mean"}.

side

a string that takes values in {"two", "above", "below"}. See Details.

quantiles

for covariates in the training data. Used only when type = "CQR". See Details.

outfun

a function that models the conditional mean or quantiles, or a valid string. The default is random forest when type = "mean" and quantile random forest when type = "CQR". See Details.

outparams

a list of other parameters to be passed into outfun.

psfun

a function that models the missing mechanism (probability of missing given X), or a valid string. The default is "Boosting". See Details.

psparams

a list of other parameters to be passed into psfun.

cfprop

the proportion of units to be used to compute ITE intervals in nested approach. Used only when algo = "nest".

citype

the type of interval conformal inference used in the nested approach with exact calibration. Used only when algo = "nest" and exact = TRUE.

lofun

the function to fit the lower bound, or a valid string. Used only when algo = "nest". See Details.

loquantile

the quantile to fit for lofun; see Details. Used only when algo = "nest" and citype = "CQR". See Details.

loparams

a list of other parameters to be passed into lofun.

upfun

the function to fit the upper bound, or a valid string. Used only when algo = "nest". See Details.

upquantile

the quantile to fit for upfun. Used only when algo = "nest" and citype = "CQR". See Details.

upparams

a list of other parameters to be passed into upfun.

useCV

FALSE for split conformal inference and TRUE for CV+.

trainprop

proportion of units for training outfun. The default if 75%. Used only when useCV = FALSE.

nfolds

number of folds. The default is 10. Used only when useCV = TRUE.

wthigh

upper truncation level of weights. See predict.conformalSplit or predict.conformalCV.

wtlow

lower truncation level of weights. See predict.conformalSplit or predict.conformalCV.

Value

a function that outputs the interval estimates on a given dataset. When algo = "nest" or "naive", it takes a single input X; when algo = "counterfactual", it takes three inputs X, Y and T.

#' @seealso conformal, conformalInt, conformalCf

Details

The algorithm to be used is controlled by algo and exact:

  • (Default) when algo = "nest" and exact = FALSE, the inexact nested approach is used. It first splits the data into two folds, with the second fold including cfprop fraction of units. Then it applies conformalCf on the first fold to compute counterfactual intervals on the second fold, which further yields interval estimates of ITE \(\hat{C}(X_i)\). Finally it fits \(\hat{C}^{L}(X_i)\) and \(\hat{C}^{R}(X_i)\) on \(X_i\)'s.

  • When algo = "nest" and exact = TRUE, the exact nested approach is used. It has the same steps as the inexact nested approach to produce ITE intervals \(\hat{C}(X_i)\)'s on the second fold but then applies conformalInt to calibrate them.

  • When algo = "naive", the naive approach is used. It applies conformalCf on the data and produce counterfactual intervals for both Y(1) and Y(0). The ITE intervals are computed by contrasting two counterfactual intervals.

  • When algo = "counterfactual", it handles the case where the treatment assignments and the observed outcome are both available for each testing point. As with the naive approach, it applies conformalCf on the data and produce counterfactual intervals for both Y(1) and Y(0). The ITE intervals are then computed by contrasting the observed outcome and the interval for the missing potential outcome.

When side = "above", intervals are of form [-Inf, a(x)] and when side = "below" the intervals are of form [a(x), Inf].

When type = "CQR", quantiles must be a vector of 2, regardless of side. When side = "two", quantiles will be used in outfun for both Y(1) and Y(0); when side = "above" or "below", quantiles[1] will be used for Y(0) and quantiles[2] will be used for Y(1).

outfun is applied to both Y(1) and Y(0). outfun can be a valid string, including

  • "RF" for random forest that predicts the conditional mean, a wrapper built on randomForest package. Used when type = "mean".

  • "quantRF" for quantile random forest that predicts the conditional quantiles, a wrapper built on grf package. Used when type = "CQR".

  • "Boosting" for gradient boosting that predicts the conditional mean, a wrapper built on gbm package. Used when type = "mean".

  • "quantBoosting" for quantile gradient boosting that predicts the conditional quantiles, a wrapper built on gbm package. Used when type = "CQR".

  • "BART" for gradient boosting that predicts the conditional mean, a wrapper built on bartMachine package. Used when type = "mean".

  • "quantBART" for quantile gradient boosting that predicts the conditional quantiles, a wrapper built on bartMachine package. Used when type = "CQR".

or a function object whose input must include, but not limited to

  • Y for outcome in the training data.

  • X for covariates in the training data.

  • Xtest for covariates in the testing data.

When type = "CQR", outfun should also include an argument quantiles that is either a vector of length 2 or a scalar, depending on the argument side. The output of outfun must be a matrix with two columns giving the conditional quantile estimates when quantiles is a vector of length 2; otherwise, it must be a vector giving the conditional quantile estimate or conditional mean estimate. Other optional arguments can be passed into outfun through outparams.

lofun and upfun have the same forms as outfun except that the input quantiles must be scalar when citype = "CQR", instead of a vector of 2, because only one conditional quantile is fitted. The argument loquantile is used for lofun and the argument hiquantile is used for upfun. Moreover, the output must be a vector giving the conditional quantile estimate or conditional mean estimate. Other optional arguments can be passed into lofun through loparams and upfun through upparams.

psfun can be a valid string, including

  • "RF" for random forest that predicts the propensity score, a wrapper built on randomForest package. Used when type = "mean".

  • "Boosting" for gradient boosting that predicts the propensity score, a wrapper built on gbm package. Used when type = "mean".

or a function object whose input must include, but not limited to

  • Y for treatment assignment, a binary vector, in the training data.

  • X for covariates in the training data.

  • Xtest for covariates in the testing data.

The output of psfun must be a vector of predicted probabilities. Other optional arguments can be passed into psfun through psparams.

Examples

# Generate potential outcomes from two linear models
set.seed(1)
n <- 1000
d <- 5
X <- matrix(rnorm(n * d), nrow = n)
beta <- rep(1, 5)
Y1 <- X %*% beta + rnorm(n)
Y0 <- rnorm(n)

# Generate treatment indicators
ps <- pnorm(X[, 1])
T <- as.numeric(ps < runif(n))
Y <- ifelse(T == 1, Y1, Y0)

# Generate testing data
ntest <- 5
Xtest <- matrix(rnorm(ntest * d), nrow = ntest)

# Inexact nested method
CIfun <- conformalIte(X, Y, T, alpha = 0.1, algo = "nest", exact = FALSE, type = "CQR",
                      quantiles = c(0.05, 0.95), outfun = "quantRF", useCV = FALSE)
CIfun(Xtest)
#>       lower    upper
#> 1 -5.535404 1.652236
#> 2 -2.734280 3.569504
#> 3 -4.409276 2.428634
#> 4 -2.761593 3.246044
#> 5 -4.601320 3.115250

# Exact nested method
CIfun <- conformalIte(X, Y, T, alpha = 0.1, algo = "nest", exact = TRUE, type = "CQR",
                      quantiles = c(0.05, 0.95), outfun = "quantRF",  useCV = FALSE)
CIfun(Xtest)
#>       lower    upper
#> 1 -8.006887 3.518362
#> 2 -5.756215 7.016254
#> 3 -6.451481 4.941770
#> 4 -6.225063 6.755913
#> 5 -6.476378 6.764593

# naive method
CIfun <- conformalIte(X, Y, T, alpha = 0.1, algo = "naive", type = "CQR",
                      quantiles = c(0.05, 0.95), outfun = "quantRF",  useCV = FALSE)
CIfun(Xtest)
#>       lower    upper
#> 1 -7.608528 2.942607
#> 2 -4.025057 5.658221
#> 3 -7.506229 5.170186
#> 4 -5.784016 5.770620
#> 5 -7.074514 5.060487

# counterfactual method, Y and T needs to be observed
pstest <- pnorm(Xtest[, 1])
Ttest <- as.numeric(pstest < runif(ntest))
Y1test <- Xtest %*% beta + rnorm(ntest)
Y0test <- rnorm(ntest)
Ytest <- ifelse(Ttest == 1, Y1test, Y0test)
CIfun <- conformalIte(X, Y, T, alpha = 0.1, algo = "counterfactual", type = "CQR",
                      quantiles = c(0.05, 0.95), outfun = "quantRF",  useCV = FALSE)
CIfun(Xtest, Ytest, Ttest)
#>       lower     upper
#> 1 -4.571894  1.851149
#> 2 -1.376061  3.937992
#> 3 -5.953691 -1.692984
#> 4 -3.751595  1.802698
#> 5 -4.457433  2.677075