Perspectives on Type III Statistical Errors: Exaggerating the Effects of Placebo in Neurofeedback
Evaluating the efficacy of electroencephalography neurofeedback (EEG-nf) for the treatment of attention-deficit/hyperactivity disorder (ADHD) has been a topic of vigorous debate over the past few years. However, many of the articles state a lack of efficacy and insist on placebo as the explanation for any positive effects found in the EEG-nf treatment group. Several critical flaws in this analysis are discussed including the existence of non-inert shams, the false no-effect, and placebo as an ad hoc explanation. These flaws lead to Type III statistical errors, which are often repeated in other articles. It is recommended that journals, books, and media articles publishing new research and reviews on the efficacy of EEG-nf be vigilant for these errors in order to improve the quality of the EEG-nf body of research. Requiring researchers and authors reviewing the literature to verify assumptions of non-inert shams, ensure the use of best practices in the EEG-nf treatment groups, and clearly identify ad hoc conclusions can avoid these Type III errors.
Bussalb, A., Congedo, M., Barthélemy, Q., Ojeda, D., Acquaviva, E., Delorme, R., & Mayaud, L. (2019). Clinical and experimental factors influencing the efficacy of neurofeedback in ADHD: A meta-analysis. Frontiers in Psychiatry, 10, 35. http://dx.doi.org/10.3389/fpsyt.2019.00035
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18. http://dx.doi.org/10.1037/0022-006X.66.1.7
Cortese, S., Ferrin, M., Brandeis, D., Holtmann, M., Aggensteiner, P., Daley, D., ... Sonuga-Barke, E. J. S. (2016). Neurofeedback for attention-deficit/hyperactivity disorder: Meta-Analysis of clinical and neuropsychological outcomes from randomized controlled trials. Journal of the American Academy of Child & Adolescent Psychiatry, 55(6), 444–455. http://dx.doi.org/10.1016/j.jaac.2016.03.007
Ghaziri, J., & Thibault, R. T. (2019). Neurofeedback: An inside perspective. In A. Raz & R. T. Thibault (Eds.), Casting Light on the Dark Side of Brain Imaging (pp. 113–116). Cambridge, MA: Academic Press. http://dx.doi.org/10.1016/B978-0-12-816179-1.00019-0
Horn, B., Balk, J., & Gold, J. I. (2011). Revisiting the sham: Is it all smoke and mirrors? Evidence-Based Complementary and Alternative Medicine, 2011, 842767. http://dx.doi.org/10.1093/ecam/neq074
Logemann, H. N. A., Lansbergen, M. M., Van Os, T. W. D. P., Böcker, K. B. E., & Kenemans, J. L. (2010). The effectiveness of EEG-feedback on attention, impulsivity and EEG: A sham feedback controlled study. Neuroscience Letters, 479(1), 49–53. http://dx.doi.org/10.1016/j.neulet.2010.05.026
Loo, S. K., & Makeig, S. (2012). Clinical utility of EEG in attention-deficit/hyperactivity disorder: A research update. Neurotherapeutics, 9(3), 569–587. http://dx.doi.org/10.1007/s13311-012-0131-z
Pigott, H. E., Cannon, R., & Trullinger, M. (2018). The fallacy of sham-controlled neurofeedback trials: A reply to Thibault and colleagues (2018). Journal of Attention Disorders, 1087054718790802. http://dx.doi.org/10.1177/1087054718790802
Schönenberg, M., Wiedemann, E., Schneidt, A., Scheeff, J., Logemann, A., Keune, P. M., & Hautzinger, M. (2017). Neurofeedback, sham neurofeedback, and cognitive-behavioural group therapy in adults with attention-deficit hyperactivity disorder: A triple-blind, randomised, controlled trial. The Lancet Psychiatry, 4(9), 673–684. http://dx.doi.org/10.1016/S2215-0366(17)30291-2
Tate, C. U. (2015). Type III and Type IV errors: Statistical decision-making considerations in addition to rejecting and retaining the null hypothesis. Retrieved from: http://web.stanford.edu/group/bps/cgi-bin/wordpress/wpcontent/uploads/2015/06/Tate.pdf
Thatcher, R. W., & Lubar, J. F. (Eds.). (2014). Z score neurofeedback: Clinical applications. San Diego, CA: Academic Press.
Thibault, R. T., Lifshitz, M., Birbaumer, N., & Raz, A. (2015). Neurofeedback, self-regulation, and brain imaging: Clinical science and fad in the service of mental disorders. Psychotherapy and Psychosomatics, 84(4), 193–207. http://dx.doi.org/10.1159/000371714
Thibault, R. T., Lifshitz, M., & Raz, A. (2016). The self-regulating brain and neurofeedback: Experimental science and clinical promise. Cortex, 74, 247–261. http://dx.doi.org/10.1016/j.cortex.2015.10.024
Thibault, R. T., Lifshitz, M., & Raz, A. (2017a). Neurofeedback or neuroplacebo? Brain, 140(4), 862–864. http://dx.doi.org/10.1093/brain/awx033
Thibault, R. T., Lifshitz, M., & Raz, A. (2017b). The climate of neurofeedback: Scientific rigour and the perils of ideology. Brain, 141(2), e11. http://dx.doi.org/10.1093/brain/awx330
Thibault, R. T., & Raz, A. (2017). The psychology of neurofeedback: Clinical intervention even if applied placebo. American Psychologist, 72(7), 679–688. http://dx.doi.org/10.1037/amp0000118
Thibault, R. T., Veissière, S., Olson, J. A., & Raz, A. (2018). Treating ADHD with suggestion: Neurofeedback and placebo therapeutics. Journal of Attention Disorders, 22(8), 707–711. http://dx.doi.org/10.1177/1087054718770012
Thornton, K. E. (2018). Perspectives on placebo: The psychology of neurofeedback. NeuroRegulation, 5(4), 137–149. http://dx.doi.org/10.15540/nr.5.4.137
van Dongen-Boomsma, M., Vollebregt, M. A., Slaats-Willemse, D., & Buitelaar, J. K. (2013). A randomized placebo-controlled trial of electroencephalographic (EEG) neurofeedback in children with attention-deficit/hyperactivity disorder. The Journal of Clinical Psychiatry, 74(8), 821–827. http://dx.doi.org/10.4088/JCP.12m08321
Van Doren, J., Arns, M., Heinrich, H., Vollebregt, M. A., Strehl, U., & Loo, S. K. (2018). Sustained effects of neurofeedback in ADHD: A systematic review and meta-analysis. European Child & Adolescent Psychiatry, 28(3), 293–305. http://dx.doi.org/10.1007/s00787-018-1121-4
Vollebregt, M. A., van Dongen-Boomsma, M., Buitelaar, J. K., & Slaats-Willemse, D. (2014). Does EEG-neurofeedback improve neurocognitive functioning in children with attention- deficit/hyperactivity disorder? A systematic review and a double-blind placebo-controlled study. The Journal of Child Psychology and Psychiatry, and Allied Disciplines, 55(5), 460–472. http://dx.doi.org/10.1111/jcpp.12143
Copyright (c) 2019 Trullinger, M., Novian, A., Russel-Chapin, L., & Pradhan, D.
This work is licensed under a Creative Commons Attribution 4.0 International License.Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC-BY) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).