Over the decades, finding a truly useful objective diagnostic test in psychiatry has proven both elusive and frustrating.

The latest candidate is a device called the NEBA system, which was approved by the US Food and Drug Administration (FDA) in July of 2013. NEBA stands for Neuropsychiatric EEG-Based Assessment Aid for ADHD, and the FDA has allowed it to be marketed as a “confirmatory” test for the diagnosis of ADHD in children ages 6 to 17. It is a 20-minute procedure that can be done in the office and the out-of-pocket cost to patients is a maximum of $425, according to NEBA Health’s website.

In this article, as in so many articles about the usefulness of diagnostic tests, you will encounter terms such as sensitivity and specificity—statistics that are meant to show how accurate a test is. But before we discuss the numbers, let’s be clear about what such numbers can and cannot show you—by using the analogy of an apple.

If you pick up an apple, you would label it as an “apple” because you’ve seen so many of them in your life and you have high confidence in your ability to recognize them. Let’s imagine that there’s a new apple-recognizing device on the market called the “Apple Rec,” which uses various technologies to measure the wavelength of light reflected by an object, its mathematical curvature, etc. The manufacturer provides impressive data showing that the Apple Rec has 100% sensitivity and 100% specificity for diagnosing (recognizing) an object as being an apple. Given these dazzling statistics, would you buy the Apple Rec? No, because even though it’s exquisitely accurate, it provides you with no useful diagnostic information beyond what you can obtain by looking at the apple yourself. However, if the Apple Rec provided you with added value, you might consider it a good investment. For example, if, in addition to correctly recognizing it as an apple, it also calculated its sweetness and crispness, the Apple Rec suddenly becomes a useful tool, because these are qualities that you would otherwise struggle to ascertain.

The apple principle applies to diagnostic tests in psychiatry. Before you refer your patients to an expensive test that diagnoses ADHD, you need to make sure that it does something that you can’t easily do yourself. Keep that in mind as we look at the evidence for NEBA.

How EEGs Work

First, let’s review some of the basics of EEGs. First developed in the 1920s, the electroencephalograph involves applying electrodes to the scalp’s surface in order to visually examine brain waves. Brain waves are labeled according to their frequency. The faster frequency bands are associated with wakefulness, the lower frequency bands with relaxation or sleep. A good way to memorize the confusing names is to use the mnemonic BAT-D (progressing from most alert to most asleep):

• Beta (16 hz, or waves per second): Alert, intellectual activity
• Alpha (8–11 hz): Relaxed, daydreaming
• Theta (4–8 hz): Deep relaxation, meditation
• Delta (1–3 hz): Deep sleep

Researchers have been studying EEG for the diagnosis and treatment of ADHD for a surprisingly long time—over 40 years. Using a refinement of EEG called Quantitative EEG, or QEEG, a fairly consistent finding has been that ADHD is associated with more activity of theta waves and less activity of beta waves. This makes intuitive sense, if you think of ADHD as a disorder in which dreaminess (theta) takes precedence over focused mental activity (beta). In most studies, the so-called TBR—theta/beta ratio—is found to be higher in ADHD kids, but that’s not always true. A recent study, in fact, compared 32 children and 22 adults with ADHD to matched healthy controls and found no evidence of increased theta activity (Liechti MD et al, Brain Topogr 2013;26(1):135–151). Nor is higher theta activity necessarily specific to ADHD, since studies have shown an association with bipolar disorder, polysubstance abuse, and epilepsy.

Regardless of these inconsistencies, NEBA claims that they have developed a system that helps clinicians diagnose ADHD by using the TBR. The study has not yet been published in a peer-reviewed journal, but the FDA reviewed it and was impressed enough with the data to allow the test to be marketed. You can find all this data on the FDA website at http://1.usa.gov/1A5g05d.

The Study the FDA Looked At

NEBA Health, the Georgia-based company that won approval for the NEBA system, divided its study into two phases, called Study 1 and Study 2. First, in Study 1, they recruited 275 children and teenagers who presented with “attention and/or behavioral concerns” to various mental health clinics throughout the US. All the patients were evaluated by clinicians via a comprehensive evaluation including an interview, various symptom scales, physical exams, and other testing as deemed necessary by the individual practitioners. In addition to the clinical evaluations, the subjects were given QEEGs via the NEBA system by separate investigators who were blinded to the clinical diagnosis. The original clinicians were blinded to the results of the EEG.

So far so good—we have the makings of a well-conducted comparison of a clinical diagnosis with an objective test. But here’s where the study design got a little too complicated. Instead of using a real world diagnosis as the gold standard by which to judge the accuracy of the EEG, the researchers used a multidisciplinary team diagnosis as the gold standard. The team consisted of a clinical psychologist, a neurodevelopmental pediatrician, and a child psychiatrist, and they reviewed all the patient files created by the clinician. However, unlike the clinician, the team didn’t actually see or talk to the patients.

Nonetheless, this once-removed team diagnosis was considered by researchers to be the gold standard by which the NEBA system would be judged, and was termed the “Best Estimate Diagnosis” or BED.

To summarize, the study’s comparison is between the following two diagnostic methods for ADHD:

1. Best estimate diagnosis: A multidisciplinary team that never saw the patient and is basing the diagnosis on retrospectively reading chart notes and psychological testing results.
2. The NEBA interpretation: The EEG theta/beta ratio, combined in some way with the clinician’s diagnosis.

 

The Study Results

Here are the basic results for the NEBA as compared to their gold standard diagnosis, separated out by age group.

1. Adolescents (12-18):
   1. Sensitivity: 89% (Sensitivity= the proportion of people with the disease who test positive)
   2. Specificity: 79% (Specificity= the proportion of patients without the disease who test negative)
   3. Positive predictive value: 81% (PPV= the proportion of patients with a positive test who actually have the disease)
   4. Negative predictive value: 93% (NPV= the proportion of patients with negative tests who do not have the disease)
2. Children (6-12):
   1. Sensitivity: 79%
   2. Specificity: 97%
   3. PPV: 96%
   4. NPV: 82%

These numbers are pretty good. For example, a 96% PPV for children means that 96% of kids with a positive NEBA had ADHD. The PPV for adolescents is less impressive—81%, meaning that 19% of adolescents with a positive NEBA did not have ADHD. Nonetheless, these are all robust numbers—as long as the information that we get is actually clinically useful.

Is it Clinically Useful?

A positive NEBA provides “confirmatory support” that your patient has a cluster of symptoms that we commonly label “ADHD.” But wait—isn’t this what we already do? We ask a series of questions and we make observations in order to ascertain whether there is a particular cluster of symptoms labeled ADHD. What the NEBA does for us is it says, “Yes, I confirm that you recognized that your patient has an attention issue, labeled ADHD by DSM-5.” How does this help me? I’m not sure. It’s telling me that an apple is, indeed, an apple.

Recognizing that there is an attention problem is the easy part—we often know this before the child even enters the room, and we definitely know after talking to the parents for 30 seconds. We don’t need an EEG to tell us this. And NEBA can’t provide information to help us with any of the following crucial questions:

• What comorbidities does the patient have that will affect our choice of treatment? A positive NEBA result may be specific for differentiating some problem from no problem, but it cannot distinguish ADHD from a host of conditions that often accompany the disorder—or can be mistaken for ADHD. At least two-thirds of kids with ADHD have a comorbid diagnosis, such as anxiety disorders, oppositional defiant disorder, learning disorders, or mood disorders (http://bit.ly/1Bla0EL).
• How severe are the symptoms? Are they mild enough so that the child can stay in the same school or so severe that a change might be needed?
• Should I start a stimulant?
• Should I refer to a behavioral therapist?

NEBA provides no information on any of the issues that affect treatment—which is just another way of saying that NEBA has little, if any, clinical utility. All NEBA can do is tell us that an attentional issue is an attentional issue. Full stop. How much is that worth to you—or, more relevantly, to your patients?

Lest it appear that we are trashing NEBA, we are not—we’re simply pointing out that it’s a promising technology with no proven clinical benefit. The next step for the company would be to demonstrate that benefit. A good way to do this would be to conduct a randomized trial—recruit 200 kids arriving at clinics for evaluations and then randomly assign 100 to standard evaluation and 100 to standard evaluation plus NEBA. Re-evaluate three to six months later, comparing the two groups. Here are some questions I’d be interested in:

• Does NEBA improve diagnostic certainty, as measured by a scale given to clinicians?
• Can it result in more rapid initiation of treatment?
• Does it lead to more rapid symptoms improvement?
• Will it improve parental satisfaction?

There are many other questions. These could be answered by appropriate research, and we hope to see such studies in the future.

CCPR’s Verdict: Not ready for prime time. We recommend avoiding NEBA and instead focusing your energy on the less technologically exciting basics—figuring out the individualized treatment plans needed to help kids succeed despite their ADHD.