Perspectives on Type III Statistical Errors: Exaggerating the Effects of Placebo in Neurofeedback

  • Mark Trullinger 1) The Chicago School of Professional Psychology, Washington, District of Columbia, USA; 2) NeuroThrive, LLC, Lutherville, Maryland, USA
  • Allen Novian 1) St. Mary’s University, San Antonio, Texas, USA; 2) integrative Counseling & Neurofeedback Solutions (iCNS), San Antonio, Texas, USA
  • Lori Russell-Chapin 1) Bradley University, Peoria, Illinois, USA; 2) 6Neurotherapy Institute of Central Illinois, Peoria, Illinois, USA
  • Deepti Pradhan 1) The Chicago School of Professional Psychology, Washington, District of Columbia, USA; 2) NeuroThrive, LLC, Lutherville, Maryland, USA
Keywords: EEG-nf, sham, false no-effect, Placebo, ADHD, Type III errors, non-inert shams


Evaluating the efficacy of electroencephalography neurofeedback (EEG-nf) for the treatment of attention-deficit/hyperactivity disorder (ADHD) has been a topic of vigorous debate over the past few years.  However, many of the articles state a lack of efficacy and insist on placebo as the explanation for any positive effects found in the EEG-nf treatment group.  Several critical flaws in this analysis are discussed including the existence of non-inert shams, the false no-effect, and placebo as an ad hoc explanation.  These flaws lead to Type III statistical errors, which are often repeated in other articles.  It is recommended that journals, books, and media articles publishing new research and reviews on the efficacy of EEG-nf be vigilant for these errors in order to improve the quality of the EEG-nf body of research.  Requiring researchers and authors reviewing the literature to verify assumptions of non-inert shams, ensure the use of best practices in the EEG-nf treatment groups, and clearly identify ad hoc conclusions can avoid these Type III errors.


Bussalb, A., Congedo, M., Barthélemy, Q., Ojeda, D., Acquaviva, E., Delorme, R., & Mayaud, L. (2019). Clinical and experimental factors influencing the efficacy of neurofeedback in ADHD: A meta-analysis. Frontiers in Psychiatry, 10, 35.

Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18.

Cortese, S., Ferrin, M., Brandeis, D., Holtmann, M., Aggensteiner, P., Daley, D., ... Sonuga-Barke, E. J. S. (2016). Neurofeedback for attention-deficit/hyperactivity disorder: Meta-Analysis of clinical and neuropsychological outcomes from randomized controlled trials. Journal of the American Academy of Child & Adolescent Psychiatry, 55(6), 444–455.

Ghaziri, J., & Thibault, R. T. (2019). Neurofeedback: An inside perspective. In A. Raz & R. T. Thibault (Eds.), Casting Light on the Dark Side of Brain Imaging (pp. 113–116). Cambridge, MA: Academic Press.

Horn, B., Balk, J., & Gold, J. I. (2011). Revisiting the sham: Is it all smoke and mirrors? Evidence-Based Complementary and Alternative Medicine, 2011, 842767.

Logemann, H. N. A., Lansbergen, M. M., Van Os, T. W. D. P., Böcker, K. B. E., & Kenemans, J. L. (2010). The effectiveness of EEG-feedback on attention, impulsivity and EEG: A sham feedback controlled study. Neuroscience Letters, 479(1), 49–53.

Loo, S. K., & Makeig, S. (2012). Clinical utility of EEG in attention-deficit/hyperactivity disorder: A research update. Neurotherapeutics, 9(3), 569–587.

Pigott, H. E., Cannon, R., & Trullinger, M. (2018). The fallacy of sham-controlled neurofeedback trials: A reply to Thibault and colleagues (2018). Journal of Attention Disorders, 1087054718790802.

Schönenberg, M., Wiedemann, E., Schneidt, A., Scheeff, J., Logemann, A., Keune, P. M., & Hautzinger, M. (2017). Neurofeedback, sham neurofeedback, and cognitive-behavioural group therapy in adults with attention-deficit hyperactivity disorder: A triple-blind, randomised, controlled trial. The Lancet Psychiatry, 4(9), 673–684.

Tate, C. U. (2015). Type III and Type IV errors: Statistical decision-making considerations in addition to rejecting and retaining the null hypothesis. Retrieved from:

Thatcher, R. W., & Lubar, J. F. (Eds.). (2014). Z score neurofeedback: Clinical applications. San Diego, CA: Academic Press.

Thibault, R. T., Lifshitz, M., Birbaumer, N., & Raz, A. (2015). Neurofeedback, self-regulation, and brain imaging: Clinical science and fad in the service of mental disorders. Psychotherapy and Psychosomatics, 84(4), 193–207.

Thibault, R. T., Lifshitz, M., & Raz, A. (2016). The self-regulating brain and neurofeedback: Experimental science and clinical promise. Cortex, 74, 247–261.

Thibault, R. T., Lifshitz, M., & Raz, A. (2017a). Neurofeedback or neuroplacebo? Brain, 140(4), 862–864.

Thibault, R. T., Lifshitz, M., & Raz, A. (2017b). The climate of neurofeedback: Scientific rigour and the perils of ideology. Brain, 141(2), e11.

Thibault, R. T., & Raz, A. (2017). The psychology of neurofeedback: Clinical intervention even if applied placebo. American Psychologist, 72(7), 679–688.

Thibault, R. T., Veissière, S., Olson, J. A., & Raz, A. (2018). Treating ADHD with suggestion: Neurofeedback and placebo therapeutics. Journal of Attention Disorders, 22(8), 707–711.

Thornton, K. E. (2018). Perspectives on placebo: The psychology of neurofeedback. NeuroRegulation, 5(4), 137–149.

van Dongen-Boomsma, M., Vollebregt, M. A., Slaats-Willemse, D., & Buitelaar, J. K. (2013). A randomized placebo-controlled trial of electroencephalographic (EEG) neurofeedback in children with attention-deficit/hyperactivity disorder. The Journal of Clinical Psychiatry, 74(8), 821–827.

Van Doren, J., Arns, M., Heinrich, H., Vollebregt, M. A., Strehl, U., & Loo, S. K. (2018). Sustained effects of neurofeedback in ADHD: A systematic review and meta-analysis. European Child & Adolescent Psychiatry, 28(3), 293–305.

Vollebregt, M. A., van Dongen-Boomsma, M., Buitelaar, J. K., & Slaats-Willemse, D. (2014). Does EEG-neurofeedback improve neurocognitive functioning in children with attention- deficit/hyperactivity disorder? A systematic review and a double-blind placebo-controlled study. The Journal of Child Psychology and Psychiatry, and Allied Disciplines, 55(5), 460–472.