Classifying Whiplash Recovery Status Using the
Neck Disability Index: Optimized Cutoff Points
Derived From Receiver Operating CharacteristicThis section was compiled by Frank M. Painter, D.C.
Send all comments or additions to: Frankp@chiro.org
FROM: J Chiropractic Medicine 2016 (Jun); 15 (2): 95101 ~ FULL TEXT
Arthur C. Croft, PhD, DC, MSc, MPH, Julie A. Workman, DC,
Michael P. Szatalowicz, DC, Philip E. Roberts, DC,
Leonard R. Suiter, DC
Spine Research Institute of San Diego,
Diego, CA.
OBJECTIVE: Researchers often use Neck Disability Index (NDI) scores to classify recovery status in whiplash patients. The purpose of this study was to investigate the optimal cutoff point score for the NDI as a mechanism for differentiating recovery from nonrecovery after whiplash.
METHODS: Subjects (N = 123) who had previously sustained whiplash injuries were recruited from 12 clinics. Subjects rated themselves as being recovered (36%) or nonrecovered (64%). This state variable was compared with their NDI score as test variable using the receiver operating characteristic statistic. The area under the receiver operating characteristic curve and optimized cutoff points were computed for the whole group and also dichotomized for sex and age.
RESULTS: The mean NDI score for the recovered group was 7.8. It was 27.1 for the nonrecovered group. The cutoff point that optimized sensitivity and specificity for the whole group was an NDI score of 15. For women, it was 19; for older persons, it was 21.
CONCLUSION: The optimal NDI score cutoff point for differentiating the recovery state after whiplash is 15. Misclassification errors are likely when using lower values.
KEYWORDS: Disability evaluation; Surveys and questionnaire; Whiplash injuries.
From the Full-Text Article:
Introduction
In the 1960s and 1970s, patient self-reports of pain and dysfunction were generally regarded by researchers and clinicians as being too subjective to serve as reliable measures of disability. Accordingly, they sought more objective forms of disability assessment such as measures of the ranges of motion of joints, physical strength as assessed by manual muscle testing, and the results of orthodox orthopedic and neurological tests. However, the boundary between hard (ie, objective or measurable) and soft (ie, subjective self-report) data is often blurred. [1] Range of motion measurement is associated with a degree of uncertainty arising from the limited precision in recording the ranges, as well as the temporal variability in voluntary range of motion associated with the subjects day-to-day experience of pain. Manual muscle testing has been shown to be lacking in sensitivity. Trained examiners have been unable to detect changes in muscular strength until the loss in strength approaches 50%. [2]
Disability rating assessment methodologies of the past have been grounded on assumptions that do not always coincide with real-life conditions. For example, although a loss of range of motion is usually considered an indicator of impairment, in some cases, reduced spinal mobility may be associated with reductions in pain and disability scores. [3] Conversely, joint instability is often associated with pain and impairment but can also coincide with greater than normal joint motion. Accordingly, Feinstein [4] argued that the crucial attribute of hardness is simply the reliability and reproducibility of a finding.
Fairbank et al [5] developed a questionnaire for the assessment of disability related to low back pain. Known today as the Revised Oswestry Disability Index, it is widely used both in the clinical setting and in research. [6] It is a self-assessment questionnaire patients can quickly complete in a few minutes, and it can be easily scored by therapists or physicians. The format is categorical, but each category is ordinal. As such, the scale is not arithmetically isomorphic, and the scores cannot be considered linearly correlated with disablement.
In 1991, Vernon and Mior [7] modified the Revised Oswestry Disability Questionnaire so that it would be responsive to disability related to neck pain and named it the Neck Disability Index (NDI). The NDI was subsequently validated in a whiplash patient population and achieved a high degree of internal consistency, reliability, and responsiveness. [7] In subsequent years, a number of other questionnaires have been developed for evaluating neck pain. These include the Whiplash Disability Questionnaire, [810] the Functional Rating Index,11 the Northwick Park Neck Pain Questionnaire, [9, 10] the Neck Pain and Disability Scale, [9] the Copenhagen Neck Functional Disability Index, [9, 12, 13] and the EQ-5D. [14] To date, the NDI has been the most extensively used questionnaire in clinical trials and prognosis studies. [9, 13, 1543]
The general format follows that of the Oswestry questionnaire, with a 10item design. These items or subscales include Pain Intensity, Personal Care, Lifting, Reading, Headaches, Concentration, Work, Driving, Sleeping, and Recreation. Each has 6 possible choices, with the first representing the normal state and carrying a score of 0, and the last representing the greatest degree of suffering or disability and carrying a score of 5. The 10 subscales are summed and multiplied by 2, thereby providing a range of 0%100% disability.
Although the theoretical perfect NDI score would be 0, it is likely that many adults will endorse 1 or more of the NDI subscales in the lower or moderate ranges. Several authors have used the NDI to classify subjects recovery status in clinical trials or outcome studies. Whereas some authors, using the NDI within a whiplash population to establish recovery, have set the threshold for recovery at 2% or less for headaches and at 1% or less for all other symptoms, [39] in 3 other published studies, recovery from whiplash injury based on the NDI score was set at a score of less than 8%. [4447]
Notwithstanding published cutoff points for NDI scores, limited normative data have been available until recently. The mean background score obtained from a large adult Japanese study was 6.98. [48] Another group has recently published normative data for Indian adolescent boys (3.59) and girls (4.92). [49] The purpose of this study was to investigate the optimal cutoff point for NDI score for a group of American adults who had suffered whiplash injury, using their self-assessment of recovery as the state variable or criterion standard.
Methods
Clinic patients from 12 clinics throughout the United States were recruited to participate in this study. Inclusion criteria included a whiplash injury resulting from a motor vehicle collision at least 24 weeks prior to recruitment, at least 3 treatment sessions with a licensed health care practitioner for the injury, fluency in English, and the requirement that they were at least 18 years of age when the injury occurred. Exclusionary criteria included prior cervical spine surgery or serious spinal disease/disorder, chronic neck pain not related to a whiplash injury, or pending litigation. Persons who suffered spinal cord injury, spinal fracture, or spinal dislocation in their motor vehicle collision were also excluded. In total, 123 subjects were enrolled (55 men, 68 women). Of these, 44 (36%) self-rated themselves as having recovered and 79 (64%) self-rated themselves as not recovered. The ages of volunteers were normally distributed, with a range from 18 to 68 years. Fifty percent of the volunteers were younger than 40 years. Informed consent was obtained, and institutional review board approval was obtained from Walden University.
All subjects completed an NDI and general information questionnaire which included a statement as to their self-assessment concerning their recovery status after their whiplash injury (recovered or not recovered). The state values (recovered or not recovered) were then compared with the test values (NDI scores) using the receiver operating characteristic (ROC) statistic (IBM SPSS Statistics, Version 22).
Results
In the present study, 4 volunteers self-rated themselves as nonrecovered even though their NDI scores were less than 8 (ie, false negatives). Twenty-one claimed to have recovered but had scores of 8 or higher (ie, false positives). Twenty-three claiming to have recovered had NDI scores less than 8 (ie, true positives), and 75 had scores of 8 or higher who were not recovered (ie, true negatives).
Figure 1
Figure 2
Figure 3 The mean NDI score in our recovered group was 7.8. The mean score of nonrecovered persons was 27.1. Men generally had lower scores than women, with a mean recovered group NDI score of 5.5 and a nonrecovered group score of 27.8. For women, the scores were 9.5 and 26.5, respectively. Women have been shown to have consistently higher Revised Oswestry Disability Index scores as well. [6]
The area under the ROC curve (AUC) for the 123 subjects as a group was .910, the standard error was .025, and the asymptotic significance level was P< .0001, with a 95% confidence interval of .856 to .964 (Fig 1). By tradition, the abscissa represents the value of 1 specificity, whereas the ordinate represents sensitivity.
The ROC curve is also plotted for men and women in Fig 2. For men, the AUC was .935, the standard error was .031, and the asymptotic significance was P< .0001, with a 95% confidence interval of .875 to .995. For women, the AUC was .884, the standard error was .046, and the asymptotic significance was P< .0001, with a 95% confidence interval of .795 to .974.
The 123 subjects were then dichotomized on the basis of age into a younger than 40 years group and a 40 years and older group (Fig 3). For the younger than 40 group, the AUC was .898, the standard error was .048, and the asymptotic significance was P< .0001, with a 95% confidence interval of .804 to .993. For the 40 and older group, the AUC was .944, the standard error was .028, and the asymptotic significance was P< .0001, with a 95% confidence interval of .890 to .999.
On the basis of this study, a reasonable optimal cutoff point for the classification of persons on the basis of NDI scores would be 15, which would provide a sensitivity of .818 and a specificity of .810. For women, the best cutoff point was 19, and for persons older than 40, it was 21.
Discussion
The ROC curve was first developed by electrical engineers during World War II as a means of determining the accuracy of radar to differentiate items of concern for more benign ones. Subsequently, it has been adopted for evaluating the sensitivity and specificity of diagnostic tests.
In this data set, the ROC curve rises rapidly toward the upper left-hand corner of the graph, indicating a low false-positive and false-negative rate. In general, the closer the AUC is to 1, the better the test. A value of .5 is the theoretical null value and would be represented by a diagonal line from the lower left to upper right parts of the graph.
In a recent Japanese study of 1,200 persons registered with an Internet research company, Kato et al [48] reported a mean NDI score 6.98, which is quite close to the 7.8 mean score we measured from our whiplash injury recovered group. They also suggested that an optimal cutoff point would be 15 for a general population and then dichotomized this number on the basis of sex, suggesting a score of 13 for men and 17 for women. And for persons in their 60s and 70s, they recommended a score of 21. Although this was a larger study of a general population, their conclusions are quite close to ours, which were based on a smaller data set of whiplash-injured persons.
It is noteworthy that the optimized scores are reported as odd numbers, whereas it is not possible to obtain an odd-numbered NDI score because the total of the subscales is always doubled. The odd numbers are merely a function of IBMs SPSS program, which was also used by Kato et al.48 From our analysis of the full 123 subjects, an NDI score of 13 had a sensitivity of .795 and a specificity of .861, which is reasonably close to the optimized score of 15 (.818 and.810). It is clear, however, that using a cutoff point of under 8 will result in a large misclassification error. In our study, for example, at a cutoff score of 7, the sensitivity and specificity were .523 and .949, and at a cutoff score of 9, the numbers were .591 and .937, which would result in a large type II error.
Limitations
This was a case-control design and the subjects who volunteered were self-selected, although there would have been no obvious ulterior motivation to participate. Subjects were asked in the general information questionnaire to answer (yes or no) the question as to whether they had recovered from their whiplash injury. The term recovery, however, was not formally defined, and it is conceivable that some subjects could have interpreted it as meaning something other than the more orthodox meaning (ie, the resolution of the symptoms or problems caused by the injury). It is also possible that, in some instances, subjects might have discounted minor ongoing symptoms when characterizing themselves as having recovered vs not recovered.
There are, of course, many human conditions, diseases, and disorders other than whiplash injury that can contribute to the score of the NDI. Because we did not have access to or inquire about preinjury NDI scores, it is conceivable that some of the subjects characterizing themselves as having recovered, despite NDI scores of 8 or higher, would have endorsed the same or nearly the same responses prior to their injury. But this missing information also limits the interpretation of the NDI in many if not most of the whiplash studies that have used the NDI.
The results of this study may provide some guidance for researchers and clinicians, but it should be expected that the results derived from other groups of patients will vary, and we encourage others to repeat this study.
The ROC AUC was greater than .900 in all but 1 subgroup analysis, indicating that the NDI had good overall sensitivity and specificity. However, although some authors have used a score of < 8 as a proxy for recovery status after whiplash injury, our study suggests that the optimized cutoff point should be 15 for men and for a general demographic, 19 for women, and as high as 21 for older populations of men and women. These optimized points are based on the combination of sensitivity and specificity. Our recommended cutoff points compare with those of a larger (N = 1200) Japanese study of persons registered with a research company and not selected for having experienced a whiplash injury: 15 for a general population, 13 for men, 17 for women, and 21 for older adults.
One limitation to the interpretation of the NDI in whiplash or other injury studies, or in a clinical setting, is the uncertainty concerning what preinjury NDI scores might have been. One potential solution might be to dichotomize the NDI into a before/after format in which subjects could provide responses to subscales based on their precrash and postcrash symptoms and impairments.
Conclusion
Although the optimal or perfect NDI score is 0, population studies have indicated that scores of generally healthy asymptomatic persons range from 4 to 5 in children to 7 in adults. Our goal was to investigate the optimal cutoff point for NDI score for a group of American adults who had suffered whiplash injury using their self-assessment of recovery as the state variable or criterion standard. The results of our investigation indicate that the optimal NDI score cutoff point for differentiating the recovery state after whiplash is 15. Misclassification errors are likely when using lower values.
Funding Sources and Potential Conflicts of Interest
This study was funded by the Spine Research Institute of San Diego.
No conflicts of interest were reported for this study.
References:
Deyo R.
Measuring the functional status of patients with low back pain.
Arch Phys Med Rehabil. 1988;69:10441053Ambroz A, Zucher R, Ambroz C.
Strength testing in pain assessment.
Pract Pain Manag. 2006:4247.Lankhorst GJ, Van de Stadt RJ, Van der Korst JK.
Natural history of idiopathic low back pain: three-year follow-up study of spinal motion, pain and functional capacity.
Scand J Rehabil Med. 1985;17:14Feinstein AR.
Clinical biostatistics XLI. Hard science, soft data, and challenges of choosing clinical variables in research.
Clin Pharmacol Ther. 1977;22:485498Fairbank JCT, Davies JB, Couper J, O'Brien JP.
The Oswestry low back pain disability questionnaire.
Physiotherapy. 1980;66(8):271273Fairbank JC, Pynsent PB.
The Oswestry Disability Index
Spine (Phila Pa 1976) 2000 (Nov 15); 25 (22): 29402952Vernon H, Mior S.
The Neck Disability Index:
A Study of Reliability and Validity
J Manipulative Physiol Ther 1991 (Sep); 14 (7): 409415Willis C, Niere KR, Hoving JL, Green S, O'Leary EF, Buchbinder R.
Reproducibility and responsiveness of the Whiplash Disability Questionnaire.
Pain. 2004;110(3):681688Pinfold M, Niere KR, O'Leary EF, Hoving JL, Green S, Buchbinder R.
Validity and internal consistency of a whiplash-specific disability measure.
Spine. 2004;29(3):263268Pho C, Godges J.
Management of whiplash-associated disorder addressing thoracic and cervical spine impairments: a case report.
J Orthop Sports Phys Ther. 2004;34(9):511519Cameron ID, Rebbeck T, Sindhusake D, Rubin G, Feyer A-M, Walsh J.
Legislative change is associated with improved health status in people with whiplash.
Spine. 2008;33(4):250254Vernon H.
Assessment of self-rated disability, impairment, and sincerity of effort in whiplash-associated disorder.
J Musculoskelet Pain. 2000;8:155167.Stewart M, Maher CG, Refshauge KM, Bogduk N, Nicholas M.
Responsiveness of pain and disability measures for chronic whiplash.
Spine. 2007;32(5):580585Pink J, Petrou S, Williamson E, Williams M, Lamb SE.
Properties of patient-reported outcome measures in individuals following acute whiplash injury.
Health Qual Life Outcomes. 2014;12:38Busse JW, Dufton JA, Kilian BC, Bhandari M.
The impact of non-injury-related factors on disability secondary to whiplash associated disorder type II: a retrospective file review.
J Manipulative Physiol Ther. 2004;27(2):7983Miettinen T, Leino E, Airaksinen O, Lindgren KA.
The Possibility to Use Simple Validated Questionnaires to Predict Long-term
Health Problems After Whiplash Injury
Spine (Phila Pa 1976) 2004 (Feb 1); 29 (3): E4751Falla D, Bilenkij G, Jull G.
Patients with chronic neck pain demonstrate altered patterns of muscle activation during performance of a functional upper limb task.
Spine. 2004;29(13):14361440Bunketorp L, Stener-Victorin E, Carlsson J.
Neck pain and disability following motor vehicle accidentsa cohort study.
Eur Spine J. 2005;14(1):8489Kaale BR, Krakenes J, Albrektsen G, Wester K.
Whiplash-associated disorders impairment rating: neck disability index score according to severity of MRI findings of ligaments and membranes in the upper cervical spine.
J Neurotrauma. 2005;22(4):466475Muller R, Giles LG.
Long-term follow-up of a randomized clinical trial assessing the efficacy of medication, acupuncture, and spinal manipulation for chronic mechanical spinal pain syndromes.
J Manipulative Physiol Ther. 2005;28(1):311Fernandez-de-las-Penas C, Fernandez-Carnero J, Palomeque del Cerro L, Miangolarra-Page JC.
Manipulative treatment vs. conventional physiotherapy treatment in whiplash injury: a randomized controlled trial.
J Whiplash Relat Disord. 2004;3:7390.Keating JL, Kent P, Davidson M, Duke R, McKinnon L, De Nardis R.
Predicting short term response and non-response to neck strengthening exercise for chronic neck pain.
J Whiplash Relat Disord. 2005;4(1):4355.Dvir Z, Gal-Eshel N, Shamir B, Pevzner E, Peretz C, Knoller N.
Simulated pain and cervical motion in patients with chronic disorders of the cervical spine.
Pain Res Manag. 2004;9(3):131136Hurwitz EL, Morgenstern H, Vassilaki M, Chaing L-M.
Frequency and clinical predictors of adverse reactions to chiropractic care in the UCLA neck pain study.
Spine. 2005;30(13):14771484Kumbhare DA, Balsor B, Parkinson WL.
Measurement of cervical flexor endurance following whiplash.
Disabil Rehabil. 2005;27(14):801807Treleaven J, Jull G, Lowchoy N.
Standing balance in persistent whiplash: a comparison between subjects with and without dizziness.
J Rehabil Med. 2005;37(4):224229Treleaven J, Jull G, LowChoy N.
Smooth pursuit neck torsion test in whiplash-associated disorders: relationship to self-reports of neck pain and disability, dizziness and anxiety.
J Rehabil Med. 2005;37(4):219223Bunketorp L, Carlsson J, Kowalski J, Stener-Victorin E.
Evaluating the reliability of multi-item scales: a non-parametric approach to the ordered categorical structure of data collected with the Swedish version of the Tampa Scale for Kinesiophobia and the Self-Efficacy Scale.
J Rehabil Med. 2005;37(5):330334Sterling M, Jull G, Vicenzino B, Kenardy J, Darnell R.
Physical and psychological factors predict outcome following whiplash injury.
Pain. 2005;114(12):141148Prushansky T, Gepstein R, Gordon C, Dvir Z.
Cervical muscles weakness in chronic whiplash patients.
Clin Biomech (Bristol, Avon) 2005;20(8):794798Armstrong BS, McNair PJ, Williams M.
Head and neck position sense in whiplash patients and healthy individuals and the effect of the cranio-cervical flexion action.
Clin Biomech (Bristol, Avon) 2005;20(7):675684Hooper RA, Frizzell JB, Faris P.
Case series on chronic whiplash related neck pain treated with intraarticular zygapophysial joint regeneration injection therapy.
Pain Physician. 2007;10:313318Lamb SE, Gates S, Underwood MR.
MINT Study Team Managing Injuries of the Neck Trial (MINT): design of a randomised controlled trial of treatments for whiplash associated disorders.
BMC Musculoskelet Disord. 2007;8(7):17Rebbeck TJ, Refshauge KM, Maher CG, Stewart M.
Evaluation of the core outcome measure in whiplash.
Spine. 2007;32(6):696702Prushansky T, Handelzalts S, Pevzner E.
Reproducibility of pressure pain threshold and visual analog scale findings in chronic whiplash patients.
Clin J Pain. 2007;23(4):339345Landers MR, Cheung W, Miller D.
Workers' compensation and litigation status influence the functional outcome of patients with neck pain.
Clin J Pain. 2007;23(8):676682Carroll A, Barnes M, Comiskey C.
A prospective randomized controlled study of the role of botulium toxin in whiplash-associated disorder.
Clin Rehabil. 2008;22:513519Periera MJ, Jull GA, Treleaven JM.
Self-reported driving habits in subjects with persistent whiplash-associated disorder: relationship to sensorimotor and psychologic features.
Arch Phys Med Rehabil. 2008;89:10971102Geldman M, Moore A, Cheek L.
The effect of pre-injury physical fitness on the initial severity and recovery from whiplash injury at six-month follow-up.
Clin Rehabil. 2008;22:364376Nieto R, Miro J, Huguet A.
Disability in subacute whiplash patients.
Spine. 2008;33(18):E630E635Elliott J, Sterling M, Noteboom JT, Darnell R, Galloway G, Jull G.
Fatty infiltrate in the cervical extensor muscles is not a feature of chronic, insidious-onset neck pain.
Clin Radiol. 2008;63(6):681687Chien A, Eliav E, Sterling M.
Whiplash (grade II) and cervical radiculopathy share a similar sensory presentation: an investigation using quantitative sensory testing.
Clin J Pain. 2008;24(7):595603Gabel CP, Burkett B, Neller A, Yelland M.
Can long-term impairment in general practitioner whiplash patients be predicted using screening and patient-reported outcomes?
Int J Rehabil Res. 2008;31(1):7980Sterling M, Jull G, Vicenzino B, Kenardy J.
Sensory Hypersensitivity Occurs Soon After Whiplash Injury and Is Associated With Poor Recovery
Pain. 2003 (Aug); 104 (3): 509517Vetti N, Krakenes J, Eide GE, Rorvik J, Gilhus NE, Espeland A.
Are MRI high-signal changes of alar and transverse ligaments in acute whiplash injury related to outcome?
BMC Musculoskelet Disord. 2010;11(1):260Merrick D, Stalnackle B-M.
Five years post whiplash injury: symptoms and psychological factors in recovered versus non-recovered.
BMC Res Notes. 2010;3:190.Sterling M, Elliott JM, Cabot PJ.
The Course of Serum Inflammatory Biomarkers Following Whiplash Injury and their Relationship
to Sensory and Muscle Measures: A Longitudinal Cohort Study
PLoS One. 2013 (Oct 17); 8 (10): e77903Kato S, Takeshita K, Matsudaira K, Tonosu J, Hara N, Chikuda H.
Normative score and cut-off value of the Neck Disability Index.
J Orthop Sci. 2012;17:687693Koley S, Kaushal M, Sharma R.
Correlations of neck disability index with anthropometric and neck biomechanical variables in Indian obese adolescents.
Ann Biol Res. 2014;5(2):5154
Return to WHIPLASH
Return to OUTCOME ASSESSMENT
Since 12-28-2016
| Home Page | Visit Our Sponsors | Become a Sponsor |
Please read our DISCLAIMER |