DescriptionValue-added models (VAMs) are becoming increasingly popular within accountability-based educational policies as they purport to separate out the effects of teacher and schools from student background variables. Given the fact that evaluations based on the inappropriate use of VAMs would significantly impact students, teachers and schools in a high-stake environment, the literature has advocated empirical evaluations of VAM measures before they become formal components of accountability systems. The VAM label is attached to a number of models, which range from simple to highly sophisticated models. However, in practice, educators and policymakers are often being misled into believing that these approaches give nearly identical results, and making decisions without understanding the strengths and limitations of these models. In addition, the empirical evaluations to date have shown that the VAM measures of teacher effects are sensitive to the form of the statistical model and to whether and how student background variables are controlled.
This study proposes a multivariate joint general VAM to investigate the issues raised by the applications of all the currently prominent VAMs, which can be seen as restricted cases of this general model. The general model provides a framework for comparing the restricted models and for evaluating the sensitivity of VAM measures (e.g., teacher and school effects) to the model choice. Markov chain Monte Carlo algorithm is used in a Bayesian context to implement both the general and the restricted models.
A simulation study was conducted to investigate the feasibility and robustness of the general model when the data were generated under varying assumptions. For each condition, three consecutive years of testing scores were generated for 400 students grouped into 16 classes. Real data consisting of three years of longitudinally linked student-level data from a large statewide achievement testing program were also analyzed. The results show that the proposed general model is more robust than other models to different assumptions and the inclusion of the background variable has significant impact on some models when the school/class has an unbalanced mix of advantaged and disadvantaged students.