Research Methodology in Neurofeedback

A couple of recent scientific papers on the placebo effect reached the popular media, and prompt us to revisit the whole issue of how research might be done to everyone's satisfaction in neurofeedback. The first of these articles was a review of some 47 studies of anti-depressants that had been submitted to the FDA to gain approval for anti-depressant medications. In more than half of those studies, the drugs did no better than placebo. And overall, the improvement with drugs was only 18% greater than it was with placebos. The second of these studies involved functional imagery of drug responders, placebo responders, and non-responders. It found that placebo responders also showed changes in their functional imagery, and that these changes differed from those induced by the effective medications.

These studies elevate the importance of the placebo effect once again in everyone's considerations, which can only serve to increase the burden of proof for neurofeedback. Our traditional answer to this challenge has been to point out that the placebo effect is not really on the other team as far as neurofeedback is concerned. Let me explain. From the perspective of drug interventions, the placebo is the catch-all phrase that subsumes all mechanisms other than the drug itself by which recovery may have been achieved. This includes spontaneous recovery, self-regulation-induced recovery, "the natural course of the disease" (e.g., natural waxing and waning of symptoms), and the results of changes in the psychodynamic milieu.

In another recent study reviewing the placebo effect, it was shown that the majority of the benefit being ascribed to the placebo could be understood in terms of the "natural course of the disease." This is particularly true of depression, where episodes of depression are usually episodic, and recovery is achieved in time regardless of any intervention. However, this still leaves room for a "real" placebo effect of some significance. And this real placebo effect is presumably mediated by "spontaneous" or psychodynamically mediated changes in the quality of brain self-regulation. The possibility of such a self-regulation response helps to make the case for neurofeedback. It does not undermine it. Presumably the natural waxing and waning of depression and other conditions is also reflective of changes in the quality of brain self-regulation. From our perspective, then, the placebo effect, in all its manifestations, is just a stalking horse for self-regulation-mediated recovery. As such, it is our ally, and we are inclined to take umbrage at the dismissal of a real change as being "just a placebo effect."

We have to take skeptics where they are, however, and the question will still be asked, what part of neurofeedback-induced recovery is ascribable to "specific" and to "non-specific" effects of the training. The best study to answer this question was Barry Sterman's serendipitously fully controlled, fully blinded cat study with the monomethylhydrazine. However, if we have to answer this same question with regard to each of the conditions we work with, what are we to do?

The traditional answer has been to compare the active treatment with sham treatment, or to do reversal designs. Reversal designs are no longer ethically permitted (we are not allowed to make people worse), and even placebo designs have their ethical problems. But let's pursue the idea of sham-training a little further. It is extraordinarily difficult to do well. When Sterman did it, he acknowledged that trainees in fact discovered the subterfuge (no surprise). So the blind was broken, and the whole objective of doing a blinded study was not met. To do sham-training well, we have to simulate everything about the person's EEG, including in particular the movement artifacts and the eye-blink artifacts. The only way to do this successfully is to work somehow with the person's actual EEG.

The more difficult standard to meet is that the clinician must be just as invested in success with the placebo treatment as with the experimental treatment, so that any placebo effect may be invoked. How do we fool the clinician who is at the controls into being motivated? The obvious answer is that we must employ an active treatment that also promises some benefit, rather than a total sham treatment. Even having done this, an asymmetry prevails. The clinician knows which is the experimental design, and which is the "B" design. There will be a natural tendency to "lean" in favor of the experimental treatment. The only way for a true "equipoise" to exist with respect to two treatments is for the clinician not to know which is better, and therefore to be motivated to succeed with both.

We have here devised what I think is the perfect experimental design for neurofeedback, an A/B design between two active treatments in which a fundamental doubt exists as to which is better, and outcomes are compared. In this approach, we may lose 'contrast' between treatment and sham, but we heighten our sensitivity to differential effects. We already know that it is not a big deal to demonstrate recovery from depression. What the world will take notice of is differential effects of specific treatment protocols.

Now I will point out that in our clinical work, with our generic protocols, we have in fact been working with this research design all along. Essentially all of our training protocols come in pairs, and are used as necessary to balance each other. The original training on the midline at Cz bifurcated for us into left- and right-hemisphere specificity of "C3beta" and "C4SMR" training. The more recent "C3-C4" training is bifurcating for us into a combination of F3-F4 and P3-P4. Higher-frequency training is complemented by lower-frequency training, etc. Each new client faces the clinician with fundamental ambiguities about protocols that are only resolved on the basis of the differential effects of each protocol.

An analogy that comes to mind is the optometrist who refines his choice of optical correction by means of A/B comparisons. This approach allows much greater refinement than the single-ended approach of "can you read this?" Eventually the optometrist reaches the point where the person can no longer tell which is better. Clinicians using our approach likewise do not settle down to a single protocol until an equivalent resolution is obtained by an elaborate A/B comparison process. Because of the high responsiveness of the individual to specific protocols in-session, feedback is usually quick. This means that the therapist is also in a relatively fast feedback loop of typically no more than a few minutes, but lasting at most a session or two.

This gives neurofeedback practitioners an enormous advantage over pharmacologists. After all, the 250 studies on Ritalin don't help to decide whether a particular child will be helped, and at what dosage. They are not much good in deciding what to do in the moment. By contrast, the neurofeedback practitioner is using best research practice with each and every client. So our answer to those who ask: "where are your controlled studies?" is that we are doing them, one subject at a time. There is almost never a time when we are not doing what would be called for in a "best-practices" controlled research design. Things can't get much better than this.

Siegfried Othmer
July 15, 2002