Lab 07 — Invariance (MG-CFA)

Author

Tommaso Feraco

Goals

By the end of this lab, you can:

  • Fit a multi-group CFA in lavaan
  • Run the invariance ladder (configural → metric → scalar → strict)
  • Decide when partial invariance is defensible (and how to implement it)
  • Map common claims (paths / latent means) to the minimum invariance level needed

Setup

Show code 
library(lavaan)
# library(semTools)  # optional helpers (e.g., measurementInvariance)

set.seed(1234)

# --- Simulate a simple 2-factor battery measured in two groups ---
# We simulate data separately per group so we can "bake in" a couple of
# non-invariances that your diagnostics should detect.
#
# Group 0 (reference): baseline measurement
# Group 1: (a) one loading differs, (b) two intercepts differ

model_pop_g0 <- '
  # measurement
  f1 =~ 0.80*x1 + 0.70*x2 + 0.90*x3 + 0.60*x4
  f2 =~ 0.70*x5 + 0.80*x6 + 0.60*x7

  # factor (co)variances
  f1 ~~ 1*f1
  f2 ~~ 1*f2
  f1 ~~ 0.40*f2

  # item intercepts (baseline)
  x1 ~ 0*1
  x2 ~ 0*1
  x3 ~ 0*1
  x4 ~ 0*1
  x5 ~ 0*1
  x6 ~ 0*1
  x7 ~ 0*1
'

model_pop_g1 <- '
  # measurement (ONE loading differs: x3)
  f1 =~ 0.80*x1 + 0.70*x2 + 0.70*x3 + 0.60*x4
  f2 =~ 0.70*x5 + 0.80*x6 + 0.60*x7

  # factor (co)variances
  f1 ~~ 1*f1
  f2 ~~ 1*f2
  f1 ~~ 0.40*f2

  # item intercepts (TWO intercepts differ: x2, x6)
  x1 ~ 0*1
  x2 ~ 0.50*1
  x3 ~ 0*1
  x4 ~ 0*1
  x5 ~ 0*1
  x6 ~ -0.40*1
  x7 ~ 0*1
'

N0 <- 260
N1 <- 330

d0 <- simulateData(model_pop_g0, sample.nobs = N0, meanstructure = TRUE)
d1 <- simulateData(model_pop_g1, sample.nobs = N1, meanstructure = TRUE)

dat <- rbind(
  transform(d0, group = 0),
  transform(d1, group = 1)
)

# Add a little MCAR missingness (~5%) to resemble real data
set.seed(4321)
make_mcar <- function(x, p = 0.05) {
  miss <- rbinom(length(x), 1, p) == 1
  x[miss] <- NA
  x
}
for (v in paste0("x", 1:7)) dat[[v]] <- make_mcar(dat[[v]], p = 0.05)

# Analysis model syntax (the one YOU will fit)
model_cfa <- '
  f1 =~ x1 + x2 + x3 + x4
  f2 =~ x5 + x6 + x7
'

fi <- c("chisq","df","cfi","tli","rmsea","rmsea.ci.lower","rmsea.ci.upper","srmr")

# Quick look at group sizes
table(dat$group)

  0   1 
260 330

Exercise 1 — Configural invariance (same form)

Task

  1. Fit the configural model (same factor structure in both groups; all parameters free across groups).
  2. Report: χ²(df), CFI, TLI, RMSEA (+ CI), SRMR.
  3. Interpret: does configural invariance look plausible?

Tip: for invariance you almost always want a mean structure (meanstructure = TRUE) and (usually) std.lv = TRUE.

Show code 
fit_config <- cfa(
  model_cfa,
  data = dat,
  group = "group",
  meanstructure = TRUE,
  std.lv = TRUE,
  missing = "fiml"
)

fitMeasures(fit_config, fi)
         chisq             df            cfi            tli          rmsea 
        15.839         26.000          1.000          1.029          0.000 
rmsea.ci.lower rmsea.ci.upper           srmr 
         0.000          0.010          0.024

Exercise 2 — Metric (weak) invariance (equal loadings)

Task

  1. Fit the metric model (loadings constrained equal across groups).
  2. Compare to configural using:
    • χ² difference test (anova()), and
    • ΔCFI / ΔRMSEA (compute these).
  3. Decide: does metric invariance look acceptable?
Show code 
fit_metric <- cfa(
  model_cfa,
  data = dat,
  group = "group",
  meanstructure = TRUE,
  std.lv = TRUE,
  missing = "fiml",
  group.equal = c("loadings")
)

anova(fit_config, fit_metric)

Chi-Squared Difference Test

           Df   AIC   BIC  Chisq Chisq diff    RMSEA Df diff Pr(>Chisq)  
fit_config 26 12292 12484 15.839                                         
fit_metric 31 12293 12464 27.155     11.316 0.065437       5    0.04546 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Show code 
c(
  d_cfi = fitMeasures(fit_metric, "cfi") - fitMeasures(fit_config, "cfi"),
  d_rmsea = fitMeasures(fit_metric, "rmsea") - fitMeasures(fit_config, "rmsea"),
  d_srmr = fitMeasures(fit_metric, "srmr") - fitMeasures(fit_config, "srmr")
)
    d_cfi.cfi d_rmsea.rmsea   d_srmr.srmr 
   0.00000000    0.00000000    0.01334833

Exercise 3 — Scalar (strong) invariance (equal intercepts)

Task

  1. Fit the scalar model (loadings + intercepts equal across groups).
  2. Compare to metric (Δ fit).
  3. If scalar fails, use modification indices to find candidate intercepts to free.
  4. Fit a partial scalar model freeing only what you can justify.
Show code 
fit_scalar <- cfa(
  model_cfa,
  data = dat,
  group = "group",
  meanstructure = TRUE,
  std.lv = TRUE,
  missing = "fiml",
  group.equal = c("loadings", "intercepts")
)

anova(fit_metric, fit_scalar)

Chi-Squared Difference Test

           Df   AIC   BIC  Chisq Chisq diff   RMSEA Df diff Pr(>Chisq)    
fit_metric 31 12293 12464 27.155                                          
fit_scalar 36 12321 12470 65.024     37.869 0.14928       5  4.009e-07 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Show code 
c(
  d_cfi = fitMeasures(fit_scalar, "cfi") - fitMeasures(fit_metric, "cfi"),
  d_rmsea = fitMeasures(fit_scalar, "rmsea") - fitMeasures(fit_metric, "rmsea"),
  d_srmr = fitMeasures(fit_scalar, "srmr") - fitMeasures(fit_metric, "srmr")
)
    d_cfi.cfi d_rmsea.rmsea   d_srmr.srmr 
  -0.05071181    0.05227792    0.01457928
Show code 
# Look at the top MI specifically for intercept constraints
mi_int <- lavTestScore(fit_scalar)
mi_int$uni[order(mi_int$uni$X2, decreasing = TRUE),][1:10,]

univariate score tests:

     lhs op   rhs     X2 df p.value
9  .p19. == .p45. 22.647  1   0.000
13 .p23. == .p49. 11.338  1   0.001
14 .p24. == .p50.  9.096  1   0.003
3   .p3. == .p29.  8.918  1   0.003
8  .p18. == .p44.  8.157  1   0.004
4   .p4. == .p30.  4.402  1   0.036
2   .p2. == .p28.  1.050  1   0.306
11 .p21. == .p47.  0.969  1   0.325
12 .p22. == .p48.  0.693  1   0.405
10 .p20. == .p46.  0.469  1   0.493

Now pick one or two intercepts that are plausible (compare with parameterTable()) to free (content/method reasons), and refit as partial scalar using group.partial.

Show code 
fit_scalar_partial <- cfa(
  model_cfa,
  data = dat,
  group = "group",
  meanstructure = TRUE,
  std.lv = TRUE,
  missing = "fiml",
  group.equal = c("loadings", "intercepts"),
  # TODO: replace the two lines below with your choice(s)
  group.partial = c("x2~1", "x6~1")
)

anova(fit_metric, fit_scalar_partial)

Chi-Squared Difference Test

                   Df   AIC   BIC  Chisq Chisq diff    RMSEA Df diff Pr(>Chisq)
fit_metric         31 12293 12464 27.155                                       
fit_scalar_partial 34 12290 12448 30.486     3.3312 0.019344       3     0.3433
Show code 
fitMeasures(fit_scalar_partial, fi)
         chisq             df            cfi            tli          rmsea 
        30.486         34.000          1.000          1.008          0.000 
rmsea.ci.lower rmsea.ci.upper           srmr 
         0.000          0.036          0.038

Exercise 4 — Strict invariance (equal residual variances)

Task

  1. Fit the strict model (loadings + intercepts + residual variances equal).
  2. Compare to your best scalar solution (full or partial scalar).
  3. Decide: do you need strict invariance for your intended claim?
Show code 
fit_strict <- cfa(
  model_cfa,
  data = dat,
  group = "group",
  meanstructure = TRUE,
  std.lv = TRUE,
  missing = "fiml",
  group.equal = c("loadings", "intercepts", "residuals"),
  group.partial = c("x2~1", "x6~1")  # keep your partial constraints, if used
)

anova(fit_scalar_partial, fit_strict)

Chi-Squared Difference Test

                   Df   AIC   BIC  Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
fit_scalar_partial 34 12290 12448 30.486                                    
fit_strict         41 12282 12409 35.973     5.4865     0       7     0.6008
Show code 
c(
  d_cfi = fitMeasures(fit_strict, "cfi") - fitMeasures(fit_scalar_partial, "cfi"),
  d_rmsea = fitMeasures(fit_strict, "rmsea") - fitMeasures(fit_scalar_partial, "rmsea"),
  d_srmr = fitMeasures(fit_strict, "srmr") - fitMeasures(fit_scalar_partial, "srmr")
)
    d_cfi.cfi d_rmsea.rmsea   d_srmr.srmr 
   0.00000000    0.00000000    0.01035549

Exercise 5 — What level do you need for which claim?

Create a small table mapping claim → needed invariance level.

Task

  1. Create a data frame with two columns: claim and needed_invariance.
  2. Include at least these rows:
  • Compare factor structure → configural
  • Compare relationships (regressions/correlations) → metric
  • Compare latent means → scalar (often partial scalar)
  • Compare observed means directly → not recommended without strong evidence
Show code 
claims_tbl <- data.frame(
  claim = c(
    "Compare factor structure (same pattern of loadings)",
    "Compare relationships across groups (paths/correlations among factors)",
    "Compare latent means across groups",
    "Compare observed means across groups"
  ),
  needed_invariance = c(
    "Configural",
    "Metric (often sufficient)",
    "Scalar (often partial scalar)",
    "Not recommended without strong invariance evidence"
  )
)

Common mistakes checklist

  • Treating the invariance ladder as purely mechanical (it’s a theory + measurement argument)
  • Freeing parameters “because MI is large” (modification ≠ justification)
  • Forgetting meanstructure / identification details when comparing latent means
  • Concluding “no invariance” instead of trying minimal partial invariance

Reproducibility

Show code 
sessionInfo()
R version 4.5.1 (2025-06-13 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26100)

Matrix products: default
  LAPACK version 3.12.1

locale:
[1] LC_COLLATE=Italian_Italy.utf8  LC_CTYPE=Italian_Italy.utf8   
[3] LC_MONETARY=Italian_Italy.utf8 LC_NUMERIC=C                  
[5] LC_TIME=Italian_Italy.utf8    

time zone: Europe/Rome
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lavaan_0.6-19

loaded via a namespace (and not attached):
 [1] digest_0.6.37     fastmap_1.2.0     xfun_0.52         parallel_4.5.1   
 [5] knitr_1.50        htmltools_0.5.8.1 pbivnorm_0.6.0    rmarkdown_2.29   
 [9] stats4_4.5.1      cli_3.6.5         mnormt_2.1.1      compiler_4.5.1   
[13] rstudioapi_0.17.1 tools_4.5.1       evaluate_1.0.4    yaml_2.3.10      
[17] quadprog_1.5-8    rlang_1.1.6       jsonlite_2.0.0    htmlwidgets_1.6.4
[21] MASS_7.3-65

References