Treating depression with psychedelics: red flags and FAQ

Update, 02/2024: We have now published the paper “History repeating: guidelines to address common problems in psychedelic science” on this topic, in part based on this blog post. We introduces 10 pressing challenges that limit conclusions regarding safety and efficacy. We share a checklist that researchers, journalists, funders, policymakers, and other stakeholders can use to assess the quality of psychedelic science.


There was a lot of hype in the last days around a new study on treating major depressive disorder via psilocybin, the active ingredient of magic mushrooms. In the paper and corresponding social media posts, the researchers make odd claims, e.g. that psychedelics liberate the entrenched depressed brain—claims one would expect from a youtube ad on homeopathy, but not from prominent researchers working on serious academic topics.

This was followed by plenty of discussion on social media among researchers and folks interested in depression and such treatments. In response to my tweets, I saw many insightful comments, also by folks with lived experiences. But there were also some baffling statements by psychiatrists who really should know better (such as that there was this one study published in a good journal so the case really is closed).

I therefore created a list of potential challenges you should be aware of when navigating this topic. I hope this list of issues is useful for journalists, folks with lived experiences, and fellow academics and clinicians. For what it’s worth, I believe I am qualified to provide my views on this: I wrote my PhD on depression and have been working on depression since 2010, with around 70 published peer-reviewed papers on the topic. I also teach courses at Leiden University on clinical trials.

So lets get on with it.

Are psychedelics safe and effective treatments for major depressive disorder?

The short answer is that we really don’t know, because there is not enough research. And the research that exists has a number of big issues. I list some of the most important ones below.

Lack of control groups

Scientific studies often are “open label” studies that lack a control group, which is an issue because it leaves open the question whether psychedelics work better than placebo. “But Eiko, placebos are much better than nothing, so what’s the problem?” The problem is that we already have placebos. They are free and without side-effects. There is no need to expose people to psychedelic substances with potentially harmful effects for the placebo effect. We also already have treatments that work quite well. So to establish evidence that psychedelics are safe and effective antidepressants, we need to compare them to placebo groups and eventually also to groups who receive established treatments.

Lack of blinding

If studies on psychedelics as antidepressants have a control group, participants often know in which group they are in. There are studies using microdosing, but usually, given that the psychedelic experience is thought to be part of the active therapeutic ingredient, participants are not what we call “blind” to the condition they are in. This is a problem for two reasons.

First, because people who believe they are in the control group rate treatments as less effective. That is, the intervention group does not work better than the control group because it works in any way above placebo, but the control group is worse than the intervention group because people actively rate it worse (than if it were just placebo). People in control groups are also more likely to drop out if they know about this. Dropout rates are therefore a nice way to check whether people were blind to the group they were in. In the SMILES trial (dieting as a treatment for depression) that we re-analyzed in 2018, we found that the dropout rate in the control group was 4 times as high as in the dieting group, due to obvious issues with blinding.

Unfortunately, researchers sometimes do not ask for blinding, which is a big red flag for me. After all, considering all the efforts that go into clinical trials, it is very little work to ask participants in which group they thought they were, and can rule out a considerable threat to valid inferences. There is also evidence that if authors ask about group membership, the results are not always written up honestly or transparently. In a recent study on Ayahuasca as an antidepressant, the authors write in the paper “We believe blindness was adequately preserved in our study”. Looking at the supplementary materials, we find that 12 out of 12 (i.e. 100%) participants in Ayahuasca group thought they received Ayahuasca, but only 5 out of 14 (i.e. 42%) participants in placebo group thought they received Ayahuasca.


Get notifications for new blog posts on eiko-fried.com:

Join 1,146 other subscribers

There is a second reason lack of blinding is a problem: because clinicians know what group participants are in. Keep in mind that the study outcomes are often based on the perspective of these clinicians, because they get to determine the treatment progress, based on their assessment of what patients tell them. When they are not blind to the group people are in, this may bias their ratings and intervention and control group are not properly comparable anymore.

Conflicts of interest

Studies on antidepressant drugs (including psychedelic substances) are often funded by the pharma industry. That is ok in principle, but you need to be aware that there is overwhelming evidence that industry-sponsored studies are much more likely than independent investigations to find outcomes that are in favor of the tested drug. In medicine, it is more common to have such conflicts of interest than in psychology, broadly speaking, and of course researchers can maintain highest ethical standards after having been payed e.g. for advising a pharmaceutical industry. But conflicts have become so common and pronounced that I sometimes wonder how trustworthy a paper is. Take, for instance, a recent “international expert opinion” article “synthesizing the evidence for ketamine and esketamine in treatment-resistant depression”, published in one of the highest ranking psychiatry journals. These are the declared conflicts of interest, which include holding patents to treat depression via Ketamine. If massive financial interests come into play, I am skeptical how balanced such an opinion piece can be.

You find such conflicts in many studies in the psychedelics literature. One prominent paper that comes to mind was published in 2016; I wrote about the conflicts of interests in this Ketamine study in another blop post.

External validity

Studies so far often have a very small number of participants. This is problematic due to something we call external validity, i.e. how well do results of the studied sample generalize to the population we want to study. The smaller the sample, the more difficult this is. I wrote a dedicated blog post on this topic with several examples from elections. You wouldn’t trust a poll of 30 people to be a reliable indicator for who wins the next presidency. Why would you then put any more trust in an antidepressant study with 30 participants? It’s the same thing.

A related problem is that people who participate in psychedelic trials for depression are often volunteers who already have experiences with psychedelics. But suggesting that psychedelics are safe and effective requires them to work for most people in most situations (external validity again), and there simply isn’t sufficient evidence for that. “But Eiko, prior studies claim that e.g. only half of the participants said they had used psychedelic drugs before.” Yes, but keep in mind that studies often take place in the US, where taking illicit hard drugs can lead to serious legal repercussions, and there is strong reason to believe that people will not always report prior drug use truthfully. It is also not uncommon to screen people out if they have prior drug experiences, which gives people another reason to lie because they would like to gain access to potentially life-saving treatment. So if people here aren’t fully honest, I fully understand: but we need to be aware of this when it comes to external validity.

Follow-up times

Studies claim that they are testing new antidepressant drugs in people who often have severe, treatment resistant depression—a chronic and recurrent illness. And then they follow them only for a few days or weeks. Even if people get a little better in this time, that does not mean that this works as a treatment, rather than a brief reduction of symptoms.

Remember the Ayahuasca trial from above, in which participants had on average suffered from depression for 11 years? The study lasted for 7 days and then stopped. And the authors claimed their study supports “the safety and therapeutic value of psychedelics”. If you had another severe, recurrent, chronic illness like cancer, would you be ok if treatment studies followed people for 7 days only?

“But Eiko, many other studies have very similar issues, including antidepressant treatments and psychotherapy”. Absolutely. But that does not make studies on psychedelics more credible, unfortunately—it just raises doubts about the particular studies that have such issues.

Other red flags

There are many other red flags. Studies often use multiple measures to see how much people improve, and then selectively report the measures for which treatment worked best. Or they have a primary outcome registered, in the study protocol, but switch to another that “works” better. For the recent psilocybin paper I mentioned at the beginning of the blog post—the one claiming that there is a liberating effect of psilocybin on cortical activity—the authors have actually admitted to switching outcomes now, away from the registered QIDS depression scale and to the BDI depression scale. The justification was to “select a scale that we felt would be most sensitive to a relationship” with the outcome of interest (i.e., to obtain a significant result). There is literature showing that studies with industry sponsorship are more likely to show such patterns. Or studies focus on the reduction of symptoms as a primary outcome, when it has been widely known for decades that even when symptoms go away, people still often face severe problems with quality of life, impairment of functioning, and so on. Measurement of recovery should focus on how people are doing in their lives, in addition to the particular symptoms they show.

It is time consuming and not very rewarding for scientists to look into problems like these. Authors are often sponsored by the industry and their independence is questionable. Journals quickly get defensive about major flaws in papers because it points to structural problems with conflicts of interests and peer review. And even if critical commentaries get published, they often don’t get the attention they deserve. It’s much sexier to write a newspaper article about revolutionary new psychedelic treatments that make the lame walk and the blind see, compared to writing a newspaper article about the fact that a study switched outcomes and had issues of multiple testing and external validity and blinding, leading to several validity threats and raising questions whether these drugs actually work.

Luckily, although this is challenging work, some folks like Dr Ioana Cristea do this work, and you can find examples of identified problems in the growing literature on using recreational drugs to treat depression here and here. We’ve also written about this on a number of occasions, with a focus on Ketamine, here and here.

Moving forward

Broadly speaking, people with mental health problems usually have reasons for these problems. For depression, folks are often stuck in life situations that are difficult to escape, such as being unemployed and having tried hard to find a job but not finding a job. There are many other pathways into depression of course, and many ways to be depressed. But it’s rarely disconnected from the particular lives people lead. If life situations are severe enough, many of us would become depressed, no matter how resilient we are. For anxiety disorders, being not well is often preceded by severe stressors of traumatic event—again, life situations.

The idea that a one-time dose (or even repeated doses) of any drug can solve such problems for good seems bizarre to me, because that is not how the world is structured. Drugs cannot resolve unemployment, disability, sexism, stigma, or an abusive partner. Proponents often say that drugs can help better relate to these experiences, and maybe that’s possible for some people and some problems, and maybe that’s also how some psychotherapies work. But there isn’t any strong scientific evidence that psychedelics do this, and therefore, we ought to remain skeptical.

To conclude, we absolutely owe it to folks with mental health problems to look into all sorts of treatment options, even if they “only” work for some. That’s fine: one-size-fits-all solutions in clinical psychology and psychiatry haven’t worked well. And we should listen to people with lived experiences if they tell us something is helpful (the way this has worked with e.g. medical marijuana that has provided relief for so many), and researchers should also absolutely listen to clinicians who report positive experiences with treatments.

But we also owe it to folks to be skeptical of hypes and fads. There are existing treatments that work. They don’t always work as well as we want them to work, but they work. And new miracle cure treatments are dangerous because they may be given to a person who then does not get an effective, potentially life-saving treatment. We’ve had hundreds of promised miracle cures in the last century. A very small number of them worked reasonably well for some people. None worked amazingly well for everyone. Psychedelics will not be a miracle cure either. And going back to that new Nature Medicine paper: no, they will not “liberate your entrenched depressed brain1. That’s not how depression works. That’s not how drugs work. And that’s not how brains work.


EDIT May 16th 2022, 10pm: Shayla Love over at Vice Motherboard wrote a piece on the psilocybin study that led me to write this blog post in the first place. She talked to a lot of researchers, who identified major issues in the study.

  1. Note that this is an actual tweet of the actual first author of the paper—holy hype, Batman. Similar phrases appear in Nature Medicine paper itself, twice.

29 thoughts on “Treating depression with psychedelics: red flags and FAQ

  1. Frieder

    Dear Eiko,
    You write that “For anxiety disorders, being not well is often preceded by severe stressors of traumatic event—again, life situations.” I don’t know what the word for a “positive traumatic event” is, or if such a word even exists, but it seems to me that that is exactly what psychedelics cause one to experience (if lucky). That is assuming that our brains don’t differentiate between imagined and real events, of course.

    Reply
  2. Pingback: How Psychedelic Hype Is Hurting More People Than We Realize - News 247

  3. Pingback: How Psychedelic Hype Is Hurting More People Than We Realize - usawebstories

  4. Pingback: Psychedelic Hype Is Hurting More People Than We Realizethedigitalchaps – The Digital Chaps

  5. Pingback: Psychedelic Hype Is Hurting More People Than We Realize - Newsofmax

  6. Pingback: Psychedelic Hype Is Hurting More People Than We Realize - News7h

  7. Pingback: How Psychedelic Hype Is Hurting More People Than We Realize - iNFO Vi

  8. Pingback: How Psychedelic Hype Is Hurting More People Than We Realize – The Daily Beast – HeresWhatIthink

  9. Pingback: Psychedelic Hype Is Hurting More People Than We Realize – Right Lore

  10. Pingback: How Psychedelic Hype Is Hurting More People Than We Realize - The New York Folk

  11. Pingback: Psychedelic Hype Is Hurting More People Than We Realize - Goznews

  12. Jason

    The quasi-religious fervor surrounding this treatment is a red flag.

    It’s also very probable that the sample of individuals who would agree to this treatment would be unlike the general population on a number of dimensions.

    The hypothesis would nearly impossible to keep hidden, and the sample is likely to be motivated to help the researchers obtain results they want.

    Perhaps a partial correction would be do head-to-head trial with a team of researchers as excited about their treatment as the psychedelic team. Hypnosis?

    Reply
  13. Jay

    Hi Eiko,

    As a researcher in this field, you indeed raise valid points on the state of psychedelic research, of which we are not unaware. However, I believe this does not emphasize the ineffectiveness of psychedelics as a treatment, but rather, a failure of the research to emphasize their effectiveness. To your point of placebo conditions for psychedelics, it is incredibly apparent when one is not receiving a powerful psychoactive agent, and as of now, there is no established solution to this. Perhaps another issue that comes into play is that the enthusiasm for psychedelics is outpacing the research and the narrative that psychedelics are a “magic bullet” of sorts is perpetuated in the mainstream. This is not the case, psychedelics are a tool for treating mental disorders and without proper guidance or integration will be no more useful than your typical street drug. When used effectively, it can be likened to treating a symptom of high centrality within a psychopathological network. I hope this clarifies things and would like to thank you for your contributions to the treatment of mental disorders through network theory.

    Reply
    1. Eiko Post author

      Thank you for the kind words, Jay. I’m glad we agree that there are pretty big issues, including too much hype combined with validity threats such as lack of blinding.

      /re “I believe this does not emphasize the ineffectiveness of xxx as a treatment, but rather, a failure of the research to emphasize their effectiveness”: this is of course an opinion one can hold, but it has been used as an argument to promote a lot of nonsense in the past, including homeopathy, so I don’t find it convincing by itself.

      /re “When used effectively, it can be likened to treating a symptom of high centrality within a psychopathological network”: evidence for both is missing (in my view), and both are worth exploring further.

      Reply
  14. GordoTEK

    Don’t get me wrong, I think everyone should be skeptical about almost all published studies and you raise valid points, but the scientific evidence in favor psychedelics helping depression seems pretty robust at this point. We will know more with the results of the phase 3 clinical trials Compass is doing. Australia’s regulators just reclassified psilocybin allowing doctors to prescribe it for depression (this was an evidence based decision). Also if you survey people who have taken psilocybin you will likely find that they feel it has an antidepressant effect. Many describe it like a fog being lifted. We now even have longer term follow up data (Imperial College of London) so we have a pretty good idea of how long the effects last and what percentage of people it works in.

    Reply
    1. Eiko Post author

      Australian regulators have indeed reflassified psilocybin, against the express concern of scientists conducting the largest study in Australia:

      “These treatments are not well established at all for a sufficient level of broad-scale implementation,” she said. “We’ve got no data on long-term outcomes at all, so that worries me a lot”
      (https://www.smh.com.au/politics/federal/australia-becomes-first-country-to-recognise-psychedelics-as-medicines-20230203-p5chs6.html)

      Regarding the work at Imperial College, I haven’t seen the papers grapple sufficiently with obvious validity threats. We have a preprint online soon that extends this blog post drastically, and then I’d be happy to continue the conversation.

      Reply
      1. GordoTEK

        I would love to continue the conversation. It sounds like you’ve done quite a bit of research, I assume you’ve searched pubmed for “psilocybin depression”? https://pubmed.ncbi.nlm.nih.gov/?term=psilocybin+depression
        I’m seeing 439 results including meta analysis. Just “at a glance” these results seem to favor psilocybin for treating depression. It’s hard to believe that this is simply the result of biased research and/or sloppy/poorly controlled work. But I will reserve judgement until I see phase 3 clinical trials getting FDA approval.

        As for “We’ve got no data on long-term outcomes at all” what is considered “long term”?
        It’s certainly an “awkward” treatment for depression, what other drug requires a person under supervision to sit in a comfy chair or couch listening to music for 5 hours periodically? ;) Cost might be so high few will be able to afford it (insurance probably won’t cover it either).

        This study seemed pretty fair to me (and not a glowing win for psilocybin either, but comparable to another treatment drug):
        https://www.nejm.org/doi/full/10.1056/NEJMoa2032994

        Reply
        1. Eiko Post author

          We discuss this all in detail in our upcoming (short) paper on main challenges; no point reiterating here, happy to discuss the state of the evidence once we present our list of challenges that we see. The vast majority of these 439 papers are open label studies, which are not really useful. Open label studies on all sorts of nonsense treatments have shown very large effects sizes over the last decades, over and over again, because open label studies cannot properly control for massive threats to valid inferences, especially when it comes to internal and external validity. Add to this COI, lack of long-term followups, no proper reporting standards for adverse events, and so on, and you’re left with little to make FDA approval likely from a scientific point of view. Even recent RCTs have major problems; e.g. the very first “double blind” RCT published a few weeks ago was not double blind: https://twitter.com/EikoFried/status/1588218791825137664?s=20

          Long-term outcomes = checking if these drugs help treat depression, rather than providing short-term symptom relief. This is crucial, otherwise they should be approved (if they work) for short-term symptom relief (which is not what the studies claim the’re doing, especially in populations of treatment resistant depression). https://twitter.com/EikoFried/status/1463965001497169922

          The NEJM paper you mention has considerable validit threats: https://osf.io/ne26f

          Reply
          1. GordoTEK

            “The NEJM paper you mention has considerable validit threats: https://osf.io/ne26f” I don’t see anything earth shattering there, placebo comparison is pretty much impossible with a high dose of psilocybin since the effects are so profound, so it sounds like they ditched that idea in favor of comparison to an existing anti-depressant drug, I think the study was improved by that decision. The other criticisms seem relatively minor and I’m sure the authors can explain…

            But anyway, regarding blinding/control/placebo – what is your proposal for how it could even be done? Psilocybin essentially renders you “glued to a couch” for at least 3 hours while you get to watch a beautiful “movie” generated before your eyes by your own mind, without knocking you out/sleep. Is there some control you can think of that would be sufficient to “trick” both researchers and patients?

  15. Jason

    I don’t know how you could keep a study on psychedelics blind.
    It seems like there would be pressure to resolve cognitive dissonance by the patient. “Well that was a huge investment of time, anticipation, and other people’s careers. So….well, yes!! I AM cured, Dr.!”
    Anyone who would agree to such a study would be biased toward a positive outcome of the study.
    Also i can’t seem to get access to the data. Studies of most bona fide treatments report 10-15% of participants experienced iatrogenic effects. . What % of participants in psychedelic research feel worse after the treatment?

    Reply
    1. Eiko Post author

      There are placebos for Ketamine now that appear to have a similar effect, but that’s not possible for MDMA. Which requires another good control group, such a treatment as usual (e.g. with psychotherapy) while holding threats to e.g. internal validity constant (e.g. time spent talking to a therapist).

      “Studies of most bona fide treatments report 10-15% of participants experienced iatrogenic effects”
      I really don’t believe this to be the case for e.g. CBT, which is the gold standard for mood and anxiety disorders. But the adverse effects are often swept under the rug in this literature, because for some reason researchers get to decide whether they think the effects are related to the drug, and only then need to write about it in detail.

      Reply
      1. Jack

        Surely the iatrogenic adverse effect of CBT is when your life becomes a daily grinding misery of examining every normal thought and feeling for evidence that it’s “illogical” and therefore will get you in trouble

        Reply
  16. CGesange

    Thank you for bringing some sanity to this issue. The usual drug activists (including Leafly and other pro-pot outlets) have been pushing psychedelic mushrooms as “medicine” because that trick was used by NORML (by its own explicit admission) as a “red herring” (NORML’s own term) to dupe people into legalizing the drug for recreational usage, and it worked. The usual suspects are now doing it again. And as someone who has dealt with chronic depression for most of my life, I would point out that the only way to deal with depression successfully is to solve the problems in one’s life that are causing depression in the first place. The idea of using “magic mushrooms” to solve this problem is ridiculous: if someone is depressed because they lost a close loved one (for example) or because of a disability or social problems etc, using a mind-altering substance is not going to make the core problem go away.

    Reply
    1. Bruno Miranda Gaidargi

      I believe that psychedelics do not serve as a cure, but as ways of facing and solving problems. At least that’s how people who study this field are supposed to believe.

      Reply
  17. Pingback: Nº 73 Abril 2022 – Psicobotikas

  18. Paul

    Out of curiosity, I’m wondering if you Eiko have any personal experiences with psychedelics?

    It seems that the biggest predictor of people’s perception of this literature, and the potential of psychedelic medicine more generally, is whether they have personal experiences with psychedelics.

    Reply
    1. Eiko Post author

      Hi Paul, interesting hypothesis! I have no personal experience with psychotherapy, antidepressants, dieting, shamanic rituals, hypothermia, or psychedelics (to list just a few depression treatments I have seen publications on in the last few years).

      So from your point of view, at least, I’d be equally biased against all of these ;).

      A bit more seriously, I do believe that some people make experiences that help them in their lives. Some of these may be based on taking drugs. But that does not make such drugs safe & effective treatments, unfortunately. It requires more than that.

      Reply
  19. Antonis

    Thank you for this Eiko. I have to admit that I have lurked around the thread discussing this and other papers related to psychedelics x depression. Related to overhyping, I have attended a couple of talks by the last author, and I have to say it sounds very much his style. I have to admit, I kinda want this hype to be true, that’s why I did not really care about these ill-defined, overhyped claims and blocked them. I suspect many other scientists maybe on a similar boat, so any positive-ish data are welcome, Of course this not an excuse, we should be searching for real answers here, not TEDtalk invites.

    Overall, the rule is that extraordinary claims require extraordinary evidence, and so far that has not been the case.

    I have “one” general, almost philosophical question here. You say that we already have placebos, so no reason to use psychedelics if that’s the way they work. But do all placebos work in the same way? Do all work equally well? And can we learn anything about the pathophysiology of a condition depending on what type of placebo works well or not?

    Reply
    1. Eiko Post author

      Sorry for the late reply Antonis.

      “Can we learn anything about the pathophysiology of a condition depending on what type of placebo works well or not?”
      Placebos work better for some than other conditions (placebos have lower effect sizes for leukemia treatments than for treating mood and anxiety disorders). To what degree this is related to pathophysiology, I couldn’t answer.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.