Today in the workflow

Specify → Identify → Estimate → Evaluate → Revise/Report

In particular:

• Why measurement invariance is a prerequisite for group comparisons
• The MG-CFA ladder (configural → metric → scalar → strict)
• How to evaluate invariance (Δ fit + local diagnostics)
• Partial invariance: what to free, how to justify it
• (Briefly) structural invariance: variances/covariances/means

Learning objectives

By the end of this lesson you can:

• Explain configural / metric / scalar / strict invariance in plain language
• Fit MG-CFA models in lavaan using group.equal
• Compare nested models using ΔCFI / ΔRMSEA / Δχ² (and know the limits)
• Use score tests / modification indices to diagnose non-invariance
• Implement partial invariance via group.partial
• State which inferences are valid under each invariance level

A hot topic

Introduction

The importance of Measurement Invariance

• Researchers often compare groups on psychological constructs, assuming instruments measure the same latent variables across groups.
• This assumption is often untested.
• Measurement invariance is a prerequisite for meaningful comparisons across groups (and across time).

Invariance of a Structural Equation Model

More generally, testing invariance evaluates to what extent a hypothesized SEM can be considered invariant (same parameters) across groups.

Invariance is commonly used for comparisons across:

• gender
• age groups
• clinical vs non-clinical
• culture / language / nationality
• time (longitudinal invariance)

Assessing invariance: the multi-group analysis

• Multi-group analysis is the most widely used method to assess invariance.
• In this lesson we focus on CFA models (MG-CFA).
• Logic: estimate the same model in multiple groups, then add equality constraints and check what breaks.

The starting point (MG-CFA)

The idea (constraints ladder)

1. Start with the same model in all groups (baseline: all structural parameters are free to vary across groups)
   
2. Add constraints (more restrictive models: loadings, intercepts, residual variances, … fixed to be equal)
   
3. Compare fit: does the constraint produce a meaningful worsening?

Invariance steps (measurement)

1. Configural: same factor structure (same “form”)
2. Metric / weak: equal loadings (\(Λ\))
3. Scalar / strong: equal intercepts (\(τ\))
4. Strict: equal residual variances (\(Θ\))

Scalar invariance (full or partial) is the gateway to latent mean comparisons.

What level do you need for which claim?

Step-by-step guide (summary)

MG-CFA scheme (steps 1–4)

Slide stolen from Psicostat’s meeting by Enrico Perinelli.

TO READ

All the next ‘non-invariance’ slides are inspired in large part from the following blog post

Step 0: Separate models for each group

Step 1: Configural invariance

Step 1: Configural (non-)invariance

Configural invariance means that the “form” of the models is the same in the groups of interest. Form entails both the number of latent variables and whether the loadings are non-zero to begin with.

Step 2: Metric invariance

Step 2: Metric (non-)invariance

Metric invariance means that for each item, the loading of the factor on the item is the same in the two groups (or, more precisely, that we cannot reject the hypothesis that the loadings are the same).

Step 2: Metric (non-)invariance

The source of group differences does not come from the latent variable!

Step 3: Scalar invariance

Step 3: Scalar invariance

Scalar invariance means that for each item, the intercept is the same. This means that group differences in the item responses are fully accounted for by group differences in the latent construct.

Step 4: Invariance of observed residual variances

Residual invariance means that for each item, the residual variance—the variance of the ominous E pointing into the items—is the same. We can again phrase this statistically: if we regressed the item scores on the factor, then the variance of the remaining residual would be the same in the groups (i.e., there would be homoscedasticity).

Step 4: Invariance of observed residual variances

The thing about the residual is that it captures everything that’s not explained in the model, and explaining changes in the amount of unexplained things seems a bit futile. Residual invariance is often not tested because it’s not necessary for latent mean comparisons. It’s a bit of an anticlimactic level to end on.

Step 5: Invariance of latent variances

Step 6: Invariance of latent covariances

Step 7: Invariance of latent means

Evaluation: global comparison

Common ingredients:

• Model fit in the more constrained model (χ², RMSEA, CFI/TLI, SRMR)
• Change in fit vs less constrained model:
  • Δχ² (highly N-sensitive)
  • ΔCFI (often used)
  • ΔRMSEA / ΔSRMR (sometimes used)
  • ΔBIC (information criteria)

A marked worsening of fit indices indicates that the considered invariance model is too restrictive and thus must be rejected.

Practical comparison rules

Let MOD-A be a reference model and MOD-B a more restrictive model.

• Δχ² = χ²(B) − χ²(A) (N-sensitive; use with caution)
• ΔCFI = CFI(B) − CFI(A)
  • heuristic: accept if ΔCFI > −.01
• ΔBIC = BIC(B) − BIC(A)
  • negative values favor B

Partial invariance (what it is)

When invariance is untenable at some level (metric/scalar/strict), you can allow some parameters to differ:

• Free non-invariant loadings/intercepts/residual variances
• Keep most constraints to preserve comparability

Options (in practice):

1. Free a small number of parameters (theory-driven)
2. Argue differences are negligible (rarely convincing)
3. Remove problematic indicators (last resort)
4. Conclude constructs are not comparable with the current instrument or theoretically (sometimes correct!)

How do we choose what to free?

• Inspect parameters by group (largest differences)
• Inspect score tests / modification indices for constrained parameters
• Free one at a time and re-evaluate
• Always justify based on theory and measurement logic