Causal Inference in Adaptive Learning Systems: Understanding Learning Path Effectiveness

Adaptive learning systems are widely adopted to personalize instruction, yet their effectiveness is predominantly evaluated using predictive or correlational metrics rather than causal evidence. This study proposes a causal inference framework to evaluate whether adaptive learning paths genuinely improve learning outcomes. Using empirical data from an operational Learning Management System (LMS), adaptive learning paths are modeled as treatments and learner outcomes as potential outcomes under the counterfactual framework. The analysis incorporates directed acyclic graphs to formalize causal assumptions, followed by propensity score–based adjustment to address non-random learning path assignment. Results show substantial baseline imbalance prior to adjustment, with standardized mean differences exceeding 0.40 for key covariates such as prior knowledge and engagement. After inverse probability weighting, covariate imbalance is reduced to below 0.06 across all major variables, indicating effective reconstruction of a pseudo-randomized population. Estimated average treatment effects consistently indicate a positive causal impact of adaptive learning paths, ranging from 0.18 to 0.21 across outcome regression, inverse probability weighting, and doubly robust estimators. Confidence intervals remain strictly positive, confirming the robustness of the findings. Heterogeneous treatment effect analysis further reveals that learners with low prior knowledge experience substantially larger gains (ATE ≈ 0.32) compared to high-performing learners (ATE ≈ 0.08), demonstrating that the benefits of adaptivity are not uniformly distributed. These findings suggest that adaptive learning systems should move beyond predictive personalization toward impact-aware design, where adaptive decisions are guided by estimated causal benefit. By integrating causal evaluation into adaptive learning analytics, this study provides a principled foundation for developing more effective, equitable, and evidence-driven adaptive learning systems.