After reviewing the paper “The relative responsiveness of test instruments can be estimated using a meta-analytic approach: an illustration with treatments for depression” by Kounali et al. 2016, the editor invited me to write a response because I raised the point that a large amount of researchers seem to misunderstand the concept of scale responsiveness.
In my paper entitled “Are more responsive depression scales really superior depression scales?” published in the Journal of Clinical Epidemiology (PDF), I clarify problems with contemporary work on responsiveness pertaining to content validity, longitudinal measurement invariance, and unidimensionality.
The responsiveness of a scale is its ability to detect this clinical change. Imagine we enroll 200 patients with an average of 19 points on the BDI at baseline and an average of 13 points after 8 weeks of treatment. Consistent with the literature, we interpret the reduction of 6 points as the BDIs responsiveness, which we can standardize to compare it to the responsiveness of other scales. But do these 6 points adequately reflect clinical change? As we will see below, this conceptualization rests on three implicit assumptions of unidimensionality, temporal invariance, and content validity that are unlikely to hold in depression scales.
We set out to explore whether responsiveness of depression scales adequately captures patients’ depression improvementdwhich is necessary for the routine interpretation of more responsive scales to be psychometrically superior scales. This is unlikely to be the case for a majority of scales as three key requirements are violated: unidimensionality, temporal MI, and adequate content validity. Responsiveness is moderated by the particular symptoms instruments encompass, and short scales in particular are prone to ignore relevant
aspects of the clinical syndrome. This may explain why patients’ recovery of impaired functioning often lags behind about half a year after symptomatic remission . Multivariate analyses of scales that capture a broad range of depression symptoms may provide better insights
into how patients are doing in clinical trials [26,29].
» Fried, E. I. (2016). Are more responsive depression scales really superior depression scales? Journal of Clinical Epidemiology, 77, 4-6. (PDF)