Effect of Threshold Setting on Neurofeedback Training

This study aimed to confirm the effect of threshold setting on the performance of neurofeedback training. The experimental conditions used to confirm the effect of the different threshold settings on the degree of electroencephalographic (EEG) changes in the initial training conditions were unfamiliar to neurofeedback. Rewards were presented in low , medium , and high frequency groups according to the different threshold settings. The sensory-motor rhythm (SMR; 12–15 Hz) neurofeedback protocol was performed for all groups. We looked at whether the posttraining brain wave increases were significant in each group compared to the brain waves during training. The SMR protocol was performed in a single session and consisted of four blocks totaling 10 minutes. EEG data was collected before training as a baseline, during training, and posttraining. The results of the group analysis showed that the mean SMR value of the posterior EEG in the high frequency group was significantly higher than the SMR value in the first EEG block. The threshold settings affected learning in neurofeedback training. It was found that initially setting the threshold value for easy compensation was more effective than the setting for hard compensation.


Introduction
Neurofeedback is used in a wide range of areassuch as muscle activity, skin temperature, respiration, heart rate, and blood pressure-and is also known as electroencephalographic (EEG)-biofeedback Schwartz & Andrasik, 2017). Weber, 2011;Pacheco, 2011;Ros et al., 2009). Besides, it has been found to be beneficial in stimulating the body's natural healing potential and helping clients to become more active in their care (Hill & Castro, 2009;Niv, 2013;Redwood, 2000).
Neurofeedback is based on neurobiological findings, but the implementation and reward methods follow the principles of psychotherapy (Morales-Quezada et al., 2019;Strehl, 2014). Discussion regarding the types of rewards and the amount and frequency of feedback is still ongoing (Sherlin et al., 2011;Sulzer et al., 2013). According to learning psychology, reward-given behavior is more likely to reappear. Neurofeedback provides visual and auditory rewards when the targeted power of the EEG is increased, decreased, or maintained (Strehl, 2014). The trainee identifies real-time reflections of his or her mental state and provides feedback accordingly. This feedback helps the trainees adjust their body and mind accordingly (Gilbert & Moss, 2003;Yuan & Bieber, 2003;Yucha & Montgomery, 2008).
Feedback in neurofeedback training (NFT) is comparable to rewards in learning psychology and depends on the target of the EEG (Collura, 1999;Collura, 2007;Hammond, 2007). Just as learning psychology provides rewards (e.g., praise, food, or token) to maintain target behavior, NFT provides feedback signals (e.g., visual or auditory stimulation) to increase or decrease the targeted brain waves (Dayan & Balleine, 2002;Sherlin et al., 2011;Watanabe, Sasaki, Shibata, & Kawato;. The type of feedback (reinforcements) in NFT is related to the purpose and characteristics of the training. In protocols such as theta and alpha increase and high beta reduction for relaxation and the reduction of anxiety and arousal, subtle musical sounds, nonpatterned sounds, and natural sounds were used as rewards (Batty, Bonnington, Tang, Hawken, & Gruzelier, 2006). Protocols such as raising SMR are related to cognitive awakening in physical relaxation situations. Some examples of techniques that help the trainee concentrate are a car moving in a racing game, a bell sound when a goal is reached, or a change in the color of the graph provided (Cortoos, De Valck, Arns, Breteler & Cluydts, 2010;Doppelmayr & Weber, 2011).
The neurofeedback training follows the principle of learning psychology, emphasizing the importance of setting an appropriate frequency of reinforcement (Skinner, 1953). If a reward is not received due to the difficulty of the action, learning does not progress, and motivation is lowered. As a result, there will not be much improvement seen even after completing the sessions. Contrastingly, if the tasks are too easy, subjects can lose interest and stop trying, making the training less effective. In summary, this means that if proper compensation is not provided during learning behavior, learning does not occur (Terborg & Miller, 1978). Although neurofeedback studies have been closely related to learning theory and compensation plans, it has been challenging to find studies that deal with reinforcement schedules or the ease of obtaining rewards, which affects the effectiveness of training (Grice, 1948;Hardt & Kamiya, 1976;Ossadtchi, Shamaeva, Okorokova, Moiseeva, & Lebedev, 2017).
The standard for determining the difficulty of training when performing neurofeedback is called the threshold, and the frequency of compensation to be assigned to the trainee can vary depending on the threshold (Bauer, Fels, Royter, Raco, & Gharabaghi, 2016;Collura, 2007). Existing neurofeedback studies provide feedback by setting thresholds in various ways (Vernon et al., 2009). First, the absolute value is added to or subtracted from the mean value of the EEG, and the threshold value is set 1-2 points higher or 0.2-0.6 points lower than the mean value when aiming for a decrease or increase, respectively (Lubar, Swartwood, Swartwood, & O'Donnell, 1995;Thompson & Thompson, 1998).
Second, the threshold can be calculated by multiplying the average power by a specific value. The mean value of the frequency band to be increased from baseline is multiplied by 0.8, and the mean value of the frequency band to be suppressed is multiplied by 1.2 to 1.6 (Egner, Zech & Gruzelier, 2004;Gruzelier, 2014a;Ros et al., 2009). Both of these methods can help to set the difficulty level and compensation by setting the threshold value. Third, the threshold value can be set to maintain a range of success rates (%; compensated time / total time x 100) during the session (Arns, Feddema, & Kenemans, 2014;Sime, 2004). For example, when training the band to be increased, either a 25% enhancement rate or a 65% success rate should be maintained. The higher the enhancement rate, the easier it is to get compensation and vice versa. Third, set a certain reinforcement rate so that it can be compensated for a percentage (%) regardless of the performance level. However, this has limitations as the subjects are rewarded the same even when they are not performing better (Arns et al., 2014;Ros et al., 2009). However, the second method can compensate for the limitations of the third in that the subject is compensated only when the trainee is performing well (Egner et al., 2004;Ros et al., 2009). Recently, the second and third methods have been used in neurofeedback studies. Sherlin et al. (2011) pointed out the importance of threshold setting in neurofeedback training, but in many of the neurofeedback studies, the methods, and values for setting the thresholds were not presented (Arns et al., 2014;Egner et al., 2004;Gruzelier, Inoue, Smart, Steed, & Steffert, 2010;Sime, 2004). The rationale and basis for setting the threshold elucidate psychophysiological changes due to a threshold value. However, there is a lack of research in this area. Therefore, this study can be used as a reference for setting thresholds for neurofeedback trainees (Gruzelier, 2014b;Sherlin et al., 2011). Considering the effect of rewards on learning, it is necessary to study the setting of thresholds, which is required for determining the possibility and the frequency of compensation in neurofeedback, and its application. Therefore, in this study, we tried to clarify the relationship between threshold setting and EEG changes (learning).
The outcome of the initial session, which is the starting point of neurofeedback training, is important because it can motivate future training and is the foundation for subsequent treatment plans (Gruzelier et al., 2010;Yoo et al., 2006). The trainee is presented with the results of their initial performance, which helps to increase their motivation to participate (Gruzelier et al., 2010;Yoo et al., 2006). In neurofeedback, it is necessary to plan and implement a protocol so that the subject can receive appropriate compensation in order to control the EEG effectively during the initial session. According to learning psychology, the frequency of compensation affects learning. This is based on a previous study that found that difficult training affects learning (Gottlieb, 2004;Reynolds, 1958;Wagner, 1961). The results of studies related to initial learning support the static correlation between the frequency of rewards and rapid behavior acquisition. Continuous reinforcement is effective in learning, and neurofeedback mainly provides rewards with successive reinforcement schemes (Sherlin et al., 2011;Sulzer et al., 2013).
Shaping, which corresponds to continuous reinforcement in learning psychology, is useful for learning new and difficult behaviors (Konidaris & Barto, 2006). In shaping, actions are progressively performed until participants reach the target behavior, and the criteria for compensation is modified each step. If we apply shaping to neurofeedback, we can use successive approximations to learn the difficult behavior of the EEG control. To learn the difficult behavior of EEG control, we first need to provide compensation even at levels below the target EEG and then raise the standard for providing compensation (Sherlin et al., 2011;Sterman & Egner, 2006).
There are two views on the threshold setting according to shaping. From the trainers' perspective, they have to decide whether to provide the first reward for an easy task or a more difficult task (Miltenberger, 2011). For example, if you want to teach a child how to open a door, you need to define this act by looking at the door or taking a step towards the door. In shaping, an absolute standard act that results in a reward does not exist because the standard will gradually change.
For the trainee, being able to receive a lot of compensation with the easiest behavior is important. In the example above, if you set the first reward level to the easiest level, subjects can get rewarded by just looking at the door. However, if you set the first reward level at taking a step towards the door, the frequency of compensation will be less because it is more difficult than the former example (Miltenberger, 2011).
The control of EEG, which is the target of neurofeedback, is exceedingly difficult for the training subjects because they try to maintain the state of the EEG at higher or lower than average. Therefore, it is possible to effectively apply shaping to neurofeedback in order to perform a new action with a high degree of difficulty.
In this study, the sensory-motor rhythm (SMR; 12-15 Hz) increase protocol was used to determine the effect of threshold setting on neurofeedback training performance. According to a previous study on SMR, Vernon et al. (2003) reported that neurofeedback training was applied to the Cz region, which is a sensory-motor cortex. Perceptual sensitivity and attentional performance were improved. Based on this study, Ros et al. (2009) conducted an SMR protocol for ophthalmology and found positive changes, such as the improvement of overall sealing techniques, shortening of execution time, and a reduction of anxiety. In addition, SMR training was conducted with athletes to improve their performance, skills, and concentration (Xiang, Hou, Liao, Liao, & Hu, 2018). As mentioned above, the SMR protocols have been widely applied.
In this study, different thresholds were set for each group to examine the degree of EEG changes according to the threshold settings. The SMR neurofeedback training was divided into three groups: low, medium, and high, according to the threshold. The low group was less likely to receive compensation, and the high group was more likely to receive compensation. It was expected that the degree of change of SMR EEG in the high probability group would be significantly higher than in the other groups.

Participants
The subjects of this study were recruited through advertisements on wall posters, the university homepage noticeboard, and social media focused on undergraduate or graduate adults from a college in Seoul city. The participants were screened by a telephone interview which lasted for about 10 minutes. The screening criteria included caffeine intake, smoking, alcohol consumption, medical history, educational background, and handgrip. Excessive caffeine intake can lead to arousal and affect EEGs. People who consumed more caffeine than the recommended daily intake, which is 400 mg of caffeine, were excluded (Hammond, 2003;Okello, Abadi & Abadi, 2016). To collect information related to nicotine and alcohol addiction, questions were asked on their weekly intake frequency and intake amount.
People were excluded if they had experienced trauma or had a personal history that could cause neurobiological abnormalities (Good et al., 2001). In the present study, the Edinburgh Handedness Inventory was used to measure righthandedness (Oldfield, 1971).
None of the participants had previously experienced neurofeedback training because the results of the initial learning experience were wanted (Rasey, Lubar, McIntyre, Zoffuto, & Abbott, 1995). Among the 90 participants who indicated their willingness to participate by telephone, 64 were selected in the first screening process.
The participants that were excluded were due to being left-handed (n = 6), taking drugs (n = 3), not being able to be reached (n = 7), and not being able to speak (n = 10).
Clinical interviews were conducted by a clinical psychologist using the structured clinical interview for DSM-IV (SCID-I) to determine whether the first 64 participants were normal without any comorbidities. To control the level of intelligence, the K-WAIS-IV short forms (Choe et al., 2014) were used to exclude participants with an IQ of less than 80 or more than 120. All subjects received a sufficient explanation about the study from the researcher, read and signed the research agreement, participated in the study, and received a participation fee of $21. In the second screening process, a total of eight participants were identified as having a depressive disorder (n = 2), sleep disorder (n = 1), alcohol abuse (n = 1), social phobia (n = 1), specific phobia (n = 2), or posttraumatic stress disorder (n = 1). Finally, a total of 56 participants (12 males and 44 females) qualified for this study.

Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee (IRB) and the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Assessment Scale Edinburgh Handedness Inventory.
To control the influence of handedness, we used the Edinburgh Handhold Test proposed by Oldfield (1971). The score for each item is left-handed (−10), mainly lefthanded (−5), using both hands (0), mainly righthanded (+5), and right-handed (+10). The total score has a distribution of −100 points to +100 points. In this study, only right-handed individuals with more than 50 points were selected.

Structured Clinical Interview for DSM-IV (SCID-I).
A structured interview tool based on the DSM-IV diagnostic criteria was conducted to assess mood, anxiety, somatoform, eating, adaptive, selective disorders, and alcohol and other substance use. Any participants with a mental illness were excluded.

Korean version of the Wechsler Adult Intelligence
Scale, 4th edition (K-WAIS-IV). The Korean version of the Wechsler Adult Intelligence Scale (4th edition) measures various cognitive functions. People with an IQ of 80 or lower are considered to have a borderline intellectual disability or an intellectual disability (American Psychiatric Association, 2000). Those with an IQ of less than 80 were excluded due to concerns that the neurofeedback training would not be effective in the time allowed.

Beck Depression Inventory (BDI). The Beck
Depression Inventory (BDI) was used to measure the degree of depression by self-report. The degree of subjective depression can affect training even though it is not enough to clinically satisfy the diagnostic criteria of a depressive disorder. The BDI is a questionnaire that consists of 21 items and measures the severity of depression. The score ranges from 0 to 63. If the score is 16 or more, intervention for depressive symptoms is required.  Barea, Boquete, Mazo, and López (2002), the signal changed by about 20 V every time the eye moved. Removing the artifacts based on a criterion of  25 V can eliminate signals such as body movement and blinking. Since the rejection threshold standard that trainers often use is  25 V, physical channel rejection of autorejection was set to 25, thereby removing any artifacts in this study (Frank, Thought Technology Ltd, personal communication, March 17, 2016). The EEG data was collected in the Cz region according to the 10-20 international electrode arrangement, and the reference and ground electrodes were attached to both ear lobes, A1, and A2 ( Figure 1).

Neurofeedback training.
The training took place between December 2015 and October 2016, from 12 p.m. to 6 p.m. Participants were instructed to sleep sufficiently before visiting the laboratory and not to consume caffeinated beverages 24 hours prior to training.
The training session included equipment attachment, training, and equipment removal, which took approximately 60 minutes. Training was conducted in a shielded room in the laboratory where the noise was blocked, and metal products, including earrings, necklaces, and watches, were not to be in or on the body in the shielded room.
We provided time for participants to adapt to the unfamiliar laboratory environment before the training and explained the procedure. Training consisted of four 10-min sessions. The EEG data from the baseline was measured for 5 min, and data from the training session (first, second, third, and fourth blocks) was measured for 10 min. In addition, the posttraining EEG was measured for 5 min. The baseline block measured the baseline EEG without any visual or auditory stimuli. The post block, like the baseline block, did not any have visual or auditory stimuli but attempted to control the EEG. Posttraining EEG was used to determine whether the subject learned how to control brain wave activity during four blocks of neurofeedback training (10 min per block). Based on the mean EEG values collected at baseline, we set different thresholds according to the group. During the training, participants were asked not to move and not to deliberately have positive thoughts or imagery such as imagining a mountain or peaceful scene (Bashashati, Ward, Birch, Hashemi, & Khalilzadeh, 2003;Jindal, 2013). During all measurements and training, participants were instructed to minimize head and body movements. Between the blocks, participants took a 1-to 2-minute break to rest their eyes and relax their muscles. The SMR (12-15 Hz) increase protocol was used as the neurofeedback protocol, and the training aimed to increase the SMR and suppress theta (4-8 Hz) and high beta (25-30 Hz;Cortoos et al., 2010). Each frequency band was presented as a bar graph, and the color of the graph showed the performance of each participant. When the participant performed well, the color of the graph turned to green. Contrastingly, when the participant did not focus on the training, the graph turned red (SMR: keep above the threshold, theta, high beta: keep below threshold). In addition, when the training was going well, the trainee could see that a piece of the puzzle slotted into place on the screen with a ringing sound, functioning as visual and auditory feedback, respectively.
Threshold setting. To see the difference in training according to the experimental purpose, the frequency of rewards was set differently for the three groups, as were the thresholds. For the threshold setting, a multiplicative method was used to maintain a constant reinforcement rate. This solves the problem of being compensated even when the level of performance falls, which is a limitation of the method of obtaining percentage compensation regardless of performance (Arns et al., 2014;Egner et al., 2004;Ros et al., 2009).

Triangular wave.
The values of the multiple that multiply the average EEG in the low, medium, and high groups were deduced based on the trigonometric function and Fourier theorem, which can be applied to periodic waveforms, such as seismic, sound, and brain waves (Shaker, 2007). Compensation differences according to the threshold setting are divided into three levels: low, medium, and high. The low group set the threshold based on 30%, which was set in a previous study (Egner et al., 2004;Ros et al., 2009). Theta, SMR, and high beta graphs corresponding to the SMR protocol must be trained at the same time to satisfy the conditions, and it is expected that the enhancement will be substantially lower than 30%. The medium group had a threshold set for compensation similar at 50%, and the high group at 80% or higher when satisfying the three graphs.

Statistical analysis.
To evaluate the effectiveness of training, baseline, 1 block, and post block EEGs were analyzed after eliminating artifacts, and the average amplitude of each frequency band was calculated.
Data were analyzed using SPSS (Statistical Package for Social Science) version 21.0, and nonparametric tests were used because the data was not normally distributed. The Chi-square and Kruskal-Wallis tests were performed to verify the homogeneity of the demographic variables in each group. The demographic data included sex, age, education duration, grip, and IQ. The Kruskal-Wallis test was used to compare BDI and BAI scores between the groups, and the homogeneity of the baseline waves was verified. The Wilcoxon signedrank test was performed to compare each group's EEG during neurofeedback and postneurofeedback training.

Comparison of EEG Changes
The Kruskal-Wallis test showed no differences between the groups in baseline EEG, which means that the neurofeedback training was performed under the same conditions (Table 2).
A Wilcoxon signed-rank test was performed between the mean SMR in 1 block and the post block SMR with all participants to verify the increased SMR level due to neurofeedback training. As a result, it was found that the SMR value had increased significantly (z(56) = −2.317, p = .021). To validate the effect of neurofeedback training on the EEG, a Wilcoxon signed-rank test was performed between 1 block and post block for each group. The mean EEG values of the SMR were analyzed as follows (Table 3).
There was a statistically significant increase in SMR between 1 block and the post block (z = −2.39, p < .05) in the high-frequency group, while there was no difference in the SMR between the low and medium groups. Theta and high beta values are suppressed so that they do not rise. As a result of the analysis of 1 block and post block EEG, there were no statistically significant decreases or increases (Table  4).

Discussion
This study aimed to investigate whether the reward difficulty, according to the threshold setting, affects the changes in EEG (learning) in neurofeedback. The frequency of rewards varies based on the threshold setting, and the settings were divided into three groups: low, medium, and high. The aim was to determine if there was a significant difference in the EEG changes between the three groups. The baseline measurement used for setting the threshold was done without any intentional effort to change the EEG. However, the postmeasurement assessment measures the intuitively learned methods that could be compensated in the past training. Therefore, the baseline condition and the postcondition are not identical, and thus cannot be compared. In other words, post block and 1 block are under the same condition, and it is a criterion to analyze the degree of fluctuation of EEG in training sessions.
As a main result of the research, the SMR protocol showed that the mean value of SMR increased in the post block compared to the 1 block and that EEG significantly increased through the neurofeedback protocol. In addition, only the high group, which is more likely to receive compensation, showed a more significant increase in EEG in the post block compared to the 1 block in the training session. In particular, the SMR increase in the post block compared to the SMR in the training block suggests that frequent compensation helps to increase SMR during training, which is consistent with the principle of compensation (Sherlin et al., 2011). In the medium and low groups, it was difficult to receive compensation, and the frequency of compensation was lower. This is consistent with the results of previous studies. According to Wagner's (1961) study, mice that received more frequent rewards from the first session to the fourth session reached the destination faster than the other groups. In addition, mice that received continuous reinforcement (100%) were faster than those with intermittent reinforcement (50%). Gottlieb (2004) and Reynolds (1958) also found that the successive reinforcement group was faster in the early sessions than the intermittent reinforcement group.
The mean EEG did not increase gradually as the 1 to 4 blocks progressed. The results are similar to those of previous studies in which EEG changes were not statistically significant during training (Ros et al., 2013). Fatigue, concentration, and stress are some of the reasons why EEG does not rise gradually (Young et al., 2014). Also, it seems that neurofeedback training should be intuitively learned, and it is challenging to see a significant increase over such a short period. In this study, the increase in SMR in the post block was higher in the stable state compared to 1 block, which are situations where visual and auditory stimuli were given, and training was needed. This means that SMR can increase in a stable state after going through a method learned during training.
The implications of this study are as follows: first, although neurofeedback claims to be based on the principle of learning psychology, there are few studies on the frequency of threshold setting and compensation. This study was meaningful because initial experiments were conducted to clarify the relationship between thresholds set and EEG changes. The above results suggest that the initial training for neurofeedback should set a threshold value for easy compensation. That is, in setting the threshold, the difficulty level of the training should be set low so that the subject can receive frequent compensation during the initial session.
Second, discussions regarding sessions are continuing. The trainer should provide the client with a therapeutic effect in the minimum amount of time possible as the longer times, increase the chance of withdrawal. Additionally, trainers should reduce the number of sessions required due to costs (Arnold et al., 2012;Simkin, Thatcher, & Lubar, 2014). However, this needs to be carefully done. To lower the number of sessions, a threshold value can be set low so that compensation can be easily received, thus changing EEG from the initial session. This could also be a way to prevent motivational decline due to the absence of effective EEG changes in the early sessions.

Limitations
The ultimate goal of neurofeedback is to help subjects control their brain waves (Hammond, 2011). This study shows that a lower threshold value can be effective as compensation can be easily received in the initial phase. However, it is not known whether it is effective for mid-term training or generalization. Further studies are needed to establish thresholds for long-term sessions. In the psychology of learning, it is found that the frequency of intermittent compensation is effective in the latter part of learning, and it can be expected that intermittent compensation is more effective in midterm training. Based on the learning theory, it can be suggested that it would be effective to set a threshold value to facilitate the changes of target EEG with low difficulty (frequent frequency) at the beginning of training, and then to set a threshold value to the lower frequency as training progresses.
Early studies on the frequency of threshold setting and compensation were conducted in this study, and the amount of compensation among the factors involved in learning was also known to be a factor. In the psychology of learning, the amount of food is usually used to increase the amount of compensation (Wagner, 1961). On the other hand, it is difficult to control the amount of visual and auditory compensation as a reward for neurofeedback. Changing the amount of compensation may provide multiple visual stimuli, rather than providing a single visual stimulus. The number of rewards can vary the type of visual reward presented, and in the case of auditory rewards, the amount is even more challenging to control. Further research is needed to investigate whether the provision of multiple visual stimuli affects the subject.
Visual and auditory compensation was provided with feedback during training. In learning psychology, unconditional reinforcers (primary reinforcement), candy, and sweets have been used for children, and food has been mainly used for animals (Miltenberger, 2011). In neurofeedback, a secondary reinforcer such as sound, graph, visual feedback (space flight, ball rolling) was used (Sterman & Egner, 2006). The reward that was used in this study during the training session may have worked as a reinforcer. The feedback used in neurofeedback is usually the same as a conditional reinforcer (praise reinforcement). In this study, the subjects were told that "sound and visual changes occur with feedback when they are doing well." Further research is needed on the types of reinforcers that are effective in neurofeedback. The visual and auditory compensation used in this study means "successful" and "good," so they were used in terms of positive reinforcement and compensation for the desired action.
Finally, participants were generally in their early 20s, with a period of education over 14 years. Therefore, the findings of the current study may not be able to be generalized for children, the elderly, and specific clinical groups.

Conclusion
This study investigated different threshold setting methods. Initially, it was shown that the threshold value set for easy compensation was effective for learning (change in EEG). Based on these results, it is expected that neurofeedback trainers will be able to set threshold values, and neurofeedback training will be able to be performed more efficiently.