Perspectives on Type III Statistical Errors: Exaggerating the Effects of Placebo in Neurofeedback

Authors

  • Mark Trullinger 1) The Chicago School of Professional Psychology, Washington, District of Columbia, USA; 2) NeuroThrive, LLC, Lutherville, Maryland, USA
  • Allen Novian 1) St. Mary’s University, San Antonio, Texas, USA; 2) integrative Counseling & Neurofeedback Solutions (iCNS), San Antonio, Texas, USA
  • Lori Russell-Chapin 1) Bradley University, Peoria, Illinois, USA; 2) 6Neurotherapy Institute of Central Illinois, Peoria, Illinois, USA
  • Deepti Pradhan 1) The Chicago School of Professional Psychology, Washington, District of Columbia, USA; 2) NeuroThrive, LLC, Lutherville, Maryland, USA

DOI:

https://doi.org/10.15540/nr.6.1.38

Keywords:

EEG-nf, sham, false no-effect, Placebo, ADHD, Type III errors, non-inert shams

Abstract

Evaluating the efficacy of electroencephalography neurofeedback (EEG-nf) for the treatment of attention-deficit/hyperactivity disorder (ADHD) has been a topic of vigorous debate over the past few years.  However, many of the articles state a lack of efficacy and insist on placebo as the explanation for any positive effects found in the EEG-nf treatment group.  Several critical flaws in this analysis are discussed including the existence of non-inert shams, the false no-effect, and placebo as an ad hoc explanation.  These flaws lead to Type III statistical errors, which are often repeated in other articles.  It is recommended that journals, books, and media articles publishing new research and reviews on the efficacy of EEG-nf be vigilant for these errors in order to improve the quality of the EEG-nf body of research.  Requiring researchers and authors reviewing the literature to verify assumptions of non-inert shams, ensure the use of best practices in the EEG-nf treatment groups, and clearly identify ad hoc conclusions can avoid these Type III errors.

References

Bussalb, A., Congedo, M., Barthélemy, Q., Ojeda, D., Acquaviva, E., Delorme, R., & Mayaud, L. (2019). Clinical and experimental factors influencing the efficacy of neurofeedback in ADHD: A meta-analysis. Frontiers in Psychiatry, 10, 35. http://dx.doi.org/10.3389/fpsyt.2019.00035

Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18. http://dx.doi.org/10.1037/0022-006X.66.1.7

Cortese, S., Ferrin, M., Brandeis, D., Holtmann, M., Aggensteiner, P., Daley, D., ... Sonuga-Barke, E. J. S. (2016). Neurofeedback for attention-deficit/hyperactivity disorder: Meta-Analysis of clinical and neuropsychological outcomes from randomized controlled trials. Journal of the American Academy of Child & Adolescent Psychiatry, 55(6), 444–455. http://dx.doi.org/10.1016/j.jaac.2016.03.007

Ghaziri, J., & Thibault, R. T. (2019). Neurofeedback: An inside perspective. In A. Raz & R. T. Thibault (Eds.), Casting Light on the Dark Side of Brain Imaging (pp. 113–116). Cambridge, MA: Academic Press. http://dx.doi.org/10.1016/B978-0-12-816179-1.00019-0

Horn, B., Balk, J., & Gold, J. I. (2011). Revisiting the sham: Is it all smoke and mirrors? Evidence-Based Complementary and Alternative Medicine, 2011, 842767. http://dx.doi.org/10.1093/ecam/neq074

Logemann, H. N. A., Lansbergen, M. M., Van Os, T. W. D. P., Böcker, K. B. E., & Kenemans, J. L. (2010). The effectiveness of EEG-feedback on attention, impulsivity and EEG: A sham feedback controlled study. Neuroscience Letters, 479(1), 49–53. http://dx.doi.org/10.1016/j.neulet.2010.05.026

Loo, S. K., & Makeig, S. (2012). Clinical utility of EEG in attention-deficit/hyperactivity disorder: A research update. Neurotherapeutics, 9(3), 569–587. http://dx.doi.org/10.1007/s13311-012-0131-z

Pigott, H. E., Cannon, R., & Trullinger, M. (2018). The fallacy of sham-controlled neurofeedback trials: A reply to Thibault and colleagues (2018). Journal of Attention Disorders, 1087054718790802. http://dx.doi.org/10.1177/1087054718790802

Schönenberg, M., Wiedemann, E., Schneidt, A., Scheeff, J., Logemann, A., Keune, P. M., & Hautzinger, M. (2017). Neurofeedback, sham neurofeedback, and cognitive-behavioural group therapy in adults with attention-deficit hyperactivity disorder: A triple-blind, randomised, controlled trial. The Lancet Psychiatry, 4(9), 673–684. http://dx.doi.org/10.1016/S2215-0366(17)30291-2

Tate, C. U. (2015). Type III and Type IV errors: Statistical decision-making considerations in addition to rejecting and retaining the null hypothesis. Retrieved from: http://web.stanford.edu/group/bps/cgi-bin/wordpress/wpcontent/uploads/2015/06/Tate.pdf

Thatcher, R. W., & Lubar, J. F. (Eds.). (2014). Z score neurofeedback: Clinical applications. San Diego, CA: Academic Press.

Thibault, R. T., Lifshitz, M., Birbaumer, N., & Raz, A. (2015). Neurofeedback, self-regulation, and brain imaging: Clinical science and fad in the service of mental disorders. Psychotherapy and Psychosomatics, 84(4), 193–207. http://dx.doi.org/10.1159/000371714

Thibault, R. T., Lifshitz, M., & Raz, A. (2016). The self-regulating brain and neurofeedback: Experimental science and clinical promise. Cortex, 74, 247–261. http://dx.doi.org/10.1016/j.cortex.2015.10.024

Thibault, R. T., Lifshitz, M., & Raz, A. (2017a). Neurofeedback or neuroplacebo? Brain, 140(4), 862–864. http://dx.doi.org/10.1093/brain/awx033

Thibault, R. T., Lifshitz, M., & Raz, A. (2017b). The climate of neurofeedback: Scientific rigour and the perils of ideology. Brain, 141(2), e11. http://dx.doi.org/10.1093/brain/awx330

Thibault, R. T., & Raz, A. (2017). The psychology of neurofeedback: Clinical intervention even if applied placebo. American Psychologist, 72(7), 679–688. http://dx.doi.org/10.1037/amp0000118

Thibault, R. T., Veissière, S., Olson, J. A., & Raz, A. (2018). Treating ADHD with suggestion: Neurofeedback and placebo therapeutics. Journal of Attention Disorders, 22(8), 707–711. http://dx.doi.org/10.1177/1087054718770012

Thornton, K. E. (2018). Perspectives on placebo: The psychology of neurofeedback. NeuroRegulation, 5(4), 137–149. http://dx.doi.org/10.15540/nr.5.4.137

van Dongen-Boomsma, M., Vollebregt, M. A., Slaats-Willemse, D., & Buitelaar, J. K. (2013). A randomized placebo-controlled trial of electroencephalographic (EEG) neurofeedback in children with attention-deficit/hyperactivity disorder. The Journal of Clinical Psychiatry, 74(8), 821–827. http://dx.doi.org/10.4088/JCP.12m08321

Van Doren, J., Arns, M., Heinrich, H., Vollebregt, M. A., Strehl, U., & Loo, S. K. (2018). Sustained effects of neurofeedback in ADHD: A systematic review and meta-analysis. European Child & Adolescent Psychiatry, 28(3), 293–305. http://dx.doi.org/10.1007/s00787-018-1121-4

Vollebregt, M. A., van Dongen-Boomsma, M., Buitelaar, J. K., & Slaats-Willemse, D. (2014). Does EEG-neurofeedback improve neurocognitive functioning in children with attention- deficit/hyperactivity disorder? A systematic review and a double-blind placebo-controlled study. The Journal of Child Psychology and Psychiatry, and Allied Disciplines, 55(5), 460–472. http://dx.doi.org/10.1111/jcpp.12143

Downloads

Published

2019-03-25

Issue

Vol. 6 No. 1 (2019)

Section

Perspectives

License

Authors who publish with this journal agree to the following terms:
  • Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC-BY) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  • Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  • Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).