A few months back, a study was published in JAMA Psychiatry claiming that whole-body hyperthermia is an effective treatment for depression (UPDATE: the paper was published in full now on August 6th 2016, some time after the online first print). For those who don’t know psychiatric journals very well, JAMA Psychiatry is currently ranked highest psychiatric journal in terms of impact-factor.
I never got around to submit the commentary I wrote, but was interviewed a few days ago about the study by Eric Boodman, so I will briefly explain the main issues with the trial.
Due to the comparably low efficacy of classic antidepressant [1–3], psychiatry has recently looked for different avenues to depression treatment. One example is Ketamin [4], although the findings so far are very weak. Another example is a novel drug aimed to selectively influence the hippocampal volume of patients [5] (I previously wrote about this study here) – although hippocampal abnormalities are negligible in patients with Major Depression [6].
These studies are interesting, but have a few common drawbacks like small samples, being highly exploratory, and not being able to show robust results across different outcome measures used. They also have in common that they make extraordinary claims that may impact the life of tens of thousands of patients despite very limited evidence. For instance, the Ketamin study above examined 14 patients only, only 2 of whom showed longer-term improvement [4]. Nonetheless, the authors concluded that “repeated doses of open-label ketamine rapidly and robustly decreased suicidal ideation in pharmacologically treated outpatients with treatment-resistant depression”.
Similarly, the new hyperthermia paper [7] makes rather extraordinary claims: that we may be able to cure the common and highly impairing disorder Major Depression by increasing the patients’ body temperature. As the authors put it:
Whole-body hyperthermia holds promise as a safe, rapid-acting, antidepressant modality with a prolonged therapeutic benefit.
While I applaud the authors for examining new ways to try to help patients suffering from a terribly debilitating disorder, their conclusion is unwarranted for a number of reasons.
- The study only encompasses 29 patients that were randomized into hyperthermia and placebo groups. Even in psychiatry this is considered a miniscule sample size, and drawing general conclusions from such a small sample seems highly problematic. The same holds for the two previous papers the study cites: Rolls et al. 2008 with 12 participants [8], and Hanusch et al. with an n of 16 [9].
- The blinding failed: (71.4%) of the participants randomized to the placebo treatment believed they had received hyperthermia treatment, compared to 93.8% of the experimental group who believed so. In other words, nearly 30% of the placebo group thought they didn’t get the actual treatment. It is really difficult to warrant any form of conclusion in a randomized control trial when the blinding fails.
- The authors examined whether patients improved in their depressive symptomatology over the course of 6 weeks on 4 different outcome measures. On only 1 of these 4 measures did patients in the experimental group improve more than in the placebo group – and this is the only scale the authors write about in their manuscript. The other 3 measures that do not show any differences between the two groups are only to be found in the supplementary materials, and are not mentioned in the paper. I don’t understand how any reviewer or editor can let authors get away with this … if your groups do not differ on 75% of your outcome measures, you cannot just hide 3 of 4 in some extra file that people don’t look at – neither can you conclude that there is an effect.
- The study finds an absurd effect size of 2.23. As Neuroskeptic summarizes correctly: “This is a spectacular Cohen’s d score – given that 0.5 is considered ‘medium’ and 0.8 is considered a ‘large’ effect! For comparison, the average antidepressant medication causes [effects] with a Cohen’s d of around 0.35.”
Major Depression is a disorder with heterogeneous symptomatology and etiology, often preceded by adverse life event or stressor. Explaining this severe and complex mental illness by a suboptimal activity in the pathway from the skin to the brain [9] seems like an extraordinary claim that would require extraordinary evidence – which none of the studies so far have provided.
Additionally, when a study finds that increasing body heat resolves issues of people with Major Depression 5-10 times more efficiently than drugs and psychotherapy that have been developed for 5 or 10 decades, respectively, we may want to be a bit more skeptical and ask for a bit more evidence – and our conclusions should be appropriate to the degree of evidence – than:
Whole-body hyperthermia holds promise as a safe, rapid-acting, antidepressant modality with a prolonged therapeutic benefit.
—
References
[1] Khan, A., & Brown, W. A. (2015). Antidepressants versus placebo in major depression : an overview. World Psychiatry, 14, 294–300. doi:10.1002/wps.20241
[2] Pigott, H. E., Leventhal, A. M., Alter, G. S., & Boren, J. J. (2010). Efficacy and effectiveness of antidepressants: current status of research. Psychotherapy and Psychosomatics, 79(5), 267–79. doi:10.1159/000318293
[3] Kirsch, I., Deacon, B. J., Huedo-Medina, T. B., Scoboria, A., Moore, T. J., & Johnson, B. T. (2008). Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Medicine, 5(2), e45. doi:10.1371/journal.pmed.0050045
[4] Ionescu, D., Swee, M., Pavone, K., Taylor, N., Akeju, O., Bear, L., … Cusin, C. (2016). Rapid and Sustained Reductions in Current Suicidal Ideation Following Repeated Doses of Intravenous Ketamine: Secondary Analysis of an Open-Label Study. Journal of Clinical Psychiatry. doi:10.4088/JCP.15m10056
[5] Fava, M., Johe, K., Ereshefsky, L., Gertsik, L. G., English, B. A., Bilello, J. A., … Freeman, M. P. (2015). A Phase 1B, randomized, double blind, placebo controlled, multiple-dose escalation study of NSI-189 phosphate, a neurogenic compound, in depressed patients. Molecular Psychiatry, 1–9. doi:10.1038/mp.2015.178
[6] Fried, E. I., & Kievit, R. A. (2015). The volumes of subcortical regions in depressed and healthy individuals are strikingly similar: a reinterpretation of the results by Schmaal et al. Molecular Psychiatry, 1, 1–2. doi:10.1038/mp.2015.199
[7] Janssen, C. W., Lowry, C. A., Mehl, M. R., Allen, J. J. B., Kelly, K. L., Gartner, D. E., … Raison, C. L. (2016). Whole-Body Hyperthermia for the Treatment of Major Depressive Disorder. JAMA Psychiatry, 53706, 1–7. doi:10.1001/jamapsychiatry.2016.1031
[8] Rolls, E., Grabenhorst, F., & Parris, B. (2008). Warm pleasant feelings in the brain. Neuroimage, 41(4), 1504–13. doi:10.1016/j.neuroimage.2008.03.005
[9] Hanusch, K.-U., Janssen, C. H., Billheimer, D., Jenkins, I., Spurgeon, E., Lowry, C. A., & Raison, C. L. (2013). Whole-body hyperthermia for the treatment of major depression: associations with thermoregulatory cooling. The American Journal of Psychiatry, 170(7), 802–4. doi:10.1176/appi.ajp.2013.12111395
—
(Thanks for @_Lucibee for pointing out an error in the text)
Pingback: Small samples are inherently problematic (in certain cases) – Eiko Fried
Pingback: Ketamin as promising treatment for suicidal thoughts? | Eiko Fried
Pingback: Adjunctive Nutraceuticals for depression treatment | Eiko Fried