Foundation for Chiropractic
Education and Research
Thanks to FCER for permission to reproduce this article!
Response to Evidence Review of Back Pain Therapies
by Cherkin Et Al
Anthony L. Rosner, Ph.D., LL.D. (Hon.)
June 30, 2003
back-to-back in the same issue of the Annals of Internal Medicine
with the flawed meta-analysis by Assendelft et al.  is a
"review of the evidence" by Cherkin et al. for the
management of back pain by a number of physical interventions, including
acupuncture, massage therapy, and spinal manipulation.  From
the point of view of a reviewer basing his perspective from the literature
pertaining to spinal manipulation, this overview of treatment alternatives
is unfortunately highly problematic. In striving to "provide
clinicians, patients and health plans with a clear and balanced
understanding of the current evidence" pertaining to the use of these
CAM (complementary and alternative medical) therapies, this paper
regrettably offers precisely the opposite.
Most perplexing is its arbitrary inclusion and exclusion of studies
to arrive at its conclusions. Although the authors' original intent was to
cite systematic reviews and original articles describing the results of
RCTs published since the reviews were conducted, there are too many
(i) qualifying RCTs that were excluded,
(ii) RCTs of dubious
quality that were included,
(iii) descriptions of pertinent RCTs with no
further indication why or how they were either included or excluded, and
(iv) the omission of earlier systematic reviews or meta-analyses with
inadequate explanations of their shortcomings or flaws — or even how the
current study built upon these earlier investigations. It is almost as if
one were witnessing revisionist history.
More specifically, some of the shortcomings in Cherkin's paper can be
summarized as follows:
1. Failure to resolve conflicting previous reviews:
In an effort to filter out low-quality studies, rating systems of trial
quality have abounded as an attempt to assure that the edifice of evidence
used to warrant a therapeutic approach is more than a house of cards.
These form the cornerstone to both systematic literature reviews and
meta-analyses, the former defined as a comprehensive and rigorous review
of the peer-reviewed scientific literature requiring a predetermined
threshold of graded quality in order to be included. In meta-analyses,
on the other hand, actual effect sizes are calculated from pooled results
of different clinical trials using a variety of statistical procedures and
taking into account the size of each study.
A meta-analysis by one of the authors of the current study (Shekelle)
published in the Annals of Internal Medicine, retrieved 58 articles
representing 25 trials and supported the short-term benefit of spinal
manipulation in some patients, particularly those with uncomplicated,
acute low back pain. Data regarding chronic low-back pain at the
time of this publication were judged insufficient to evaluate the efficacy
of spinal manipulation in managing this particular condition. 
Several years later, this qualification no longer stood. The rise in
the stature of evidence supporting the use of spinal manipulation in
managing chronic low-back pain could be described as meteoric. In a
systematic review of 16 randomized controlled trials involving
manipulation, van Tulder and his colleagues singled out two of
high-quality. [4, 5] Van Tulder's review explicitly stated that
evidence supporting manipulation for chronic low-back pain is found to be
actually stronger than that for acute conditions ("There is
limited evidence that manipulation is more effective than a placebo
treatment for acute LBP [level 3]. There is no evidence that manipulation
is more effective than [other] physiotherapeutic applications . . . or
drug therapy. . . . There is strong evidence that manipulation is more
effective than a placebo treatment for chronic LBP [level 1]. There is
moderate evidence that manipulation is more effective for chronic LBP than
usual care by the general practitioner, bed-rest, analgesics, and massage
[level 2])". 
A somewhat differing interpretation was reached in Bronfort's
systematic review.  Here, the evidence supporting spinal
manipulation for managing either acute or chronic low-back pain was judged
to be "moderate," while that for a mix of chronic and
acute low-back pain was considered "inconclusive." Furthermore,
all but one of the back pain studies considered to be of sufficient
validity were eliminated by the criteria invoked by van Tulder, while the
latter study included one trial  that had been rejected by
Yet another systematic review of randomized clinical trials cites
adequate follow-up periods, avoidance of cointerventions, and avoidance of
dropouts as frequent strengths. Recurrent weaknesses, however, include
randomization procedures, sample sizes, and blinded assessments of
outcomes—the latter being virtually impossible to perform in a trial
involving manual therapy.  Finally, a meta-analysis of 51
literature reviews of spinal manipulative therapy suggests that, although
the overall methodologic quality was low, 9 of the 10 methodologically
best reviews reached positive conclusions regarding spinal adjustments. 
The fact that systematic reviews may conflict in both their conclusions
and as to which studies to accept is admittedly troublesome. None of
these inconsistencies, conflicts or resolutions were ever brought to light
by the current authors as to why their most recent review should be
accepted as more definitive than past similar efforts.
2. Exclusions of previous studies:
In terms of outcomes, there is no apparent reason why the highly
positive findings of van Tulder  or Giles  were not
integrated into the body of the authors' discussion concerning spinal
manipulation. To further illustrate the inconsistency and arbitrariness of
this review, one need only wonder why the Giles study, in particular, was
mentioned only in context of the acupuncture literature—in which neither
back dysfunction nor pain significantly improved. In the context of
spinal manipulation, in which significant improvements in both criteria were
noted, there is no mention of any kind of this study in Cherkin's paper.
In reviewing cost-effectiveness, the authors argue that they have only
chosen "the few effectiveness RCTs that measured cost" in order
to minimize bias that might have been experienced with observational data.
It then perplexes this reviewer why they entirely overlooked a trial
concerned with lumbar disc herniation (a subject which the authors
introduced only in connection with cauda equina syndrome). This particular
orphaned investigation examined 40 patients with unremitting sciatica
diagnosed as due to lumbar disc herniation with no clinical indication for
surgical intervention. Subjects were randomized into two treatments: (i)
chemonucleolysis (chymopapain injection under general anesthesia) and
manipulation (15-minute treatments over 12 weeks, including soft tissue
stretching, low-amplitude passive maneuvers of the lumbar spine and the
judicious use of side-posture manipulations). Back pain and disability
were appreciably lower in the manipulated group at 2 and 6 weeks with no
improvement or deterioration in the chemonucleolytic group. By 12 months
there were improvements in both groups with a tendency toward superiority
in the manipulated cohort. Costs of treatment in the manipulated group
were less than 30% of the total costs encountered by the injected
patients; furthermore, the latter group averaged expenditures of £300 for
treatment failures with no such costs experienced by the
manipulated population. 
3. Inclusion of study of questionable quality:
In discussing cost-effectiveness and spinal manipulation in the context
of RCTs, Cherkin cites only his own study in which any advantages
of spinal manipulation in terms of outcomes or cost-effectiveness in
comparison to physical therapy (McKenzie treatments) or the use of an
educational booklet were not evident.  The shortcomings
of this study, including its misrepresentation of therapies and the
overgeneralization of results, are so extensive that they have been
presented elsewhere in no less than four separate venues. [13-16]
In fact, the deficiencies of its design are so severe to have merited a
special review by the Royal College of General Practitioners—which
concluded that this particular RCT neither adds nor detracts from the
evidence base regarding appropriate interventions for low back pain. 
Since this review can only be as robust as the literature upon which it
rests, one would have to closely scrutinize a review in which the lead
author has chosen his own RCT whose inadequacies may have neutered its
From the perspectives of some of the recent literature addressing
spinal manipulation, this review of outcomes and cost-effectiveness
contains several glaring inconsistencies and omissions which seriously
undermine its credibility. It falls far short of the mark in providing the
"balanced" perspective originally sought by the authors and
demanded of reviews of this nature. To summarize, Cherkin's review falls
short of achieving its objectives due to:
- Its failure to resolve conflicts in previous literature reviews;
- Its inexplicable omissions of previous and highly pertinent studies;
By appearing in an academic journal adjacent to another deeply flawed
study  (the deficiencies of which have been addressed elsewhere )
this paper creates a dangerously misleading impression to the effect that
spinal manipulation is of limited or no value.
- The author's citation of only one of his own, widely discredited
papers as the sole source of data regarding cost-effectiveness and
Assendelft WJJ, Morton
SC, Yu EI, Suttorp MJ, Shekelle PG.
Spinal manipulative therapy for low
back pain: A meta-analysis of effectiveness relative to other therapies.
of Internal Medicine 2003; 138: 871-881.
Cherkin DL, Sherman SJ,
Deyo RA, Shekelle PG.
A review of the evidence for the effectiveness,
safety, and cost of acupuncture, massage therapy, and spinal manipulation
for back pain.
Annals of Internal Medicine 2003; 138: 898-906.
Shekelle PG, Adams AH,
Chassin MR, Hurwitz EL, Brook RH.
Spinal manipulation for low-back pain.
of Internal Medicine 1992; 117: 590-599.
MacDonald RS, Bell CM.
An open controlled assessment of osteopathic manipulation in nonspecific
Spine 1990; 15: 364-370.
Sanders GE, Reinert O,
Tepe R, Maloney P.
Chiropractic adjustive manipulation on subjects with
acute low-back pain: Visual analog scores and plasma beta-endorphin
Journal of Manipulative and Physiological Therapeutics
1990; 13: 391-395.
van Tulder MW, Koes BW,
Conservative treatment of acute and chronic nonspecific low
back pain: A systematic review of randomized controlled trials of the most
Spine 1997; 22 (18): 2128-2156.
of manual therapies of the spine.
Amsterdam, 1997, Thesis Publishers.
Koes BW, Bouter LM, van
der Jeijden GJMG.
Methodological quality of randomized clinical trials on
treatment efficacy in low back pain.
Spine 1995; 20: 228-235.
Assendelft WJJ, Koes BW,
Knipschild PG, Bouter LM.
The relationship between methodological quality
and conclusions in reviews of spinal manipulation.
Journal of the
American Medical Association 1995; 274(24): 1942-1948.
Giles LGF, Muller R.
Chronic Spinal Pain Syndromes: A Clinical Pilot Trial Comparing Acupuncture, A Nonsteroidal Anti-inflammatory Drug, and Spinal Manipulation
J Manipulative Physiol Ther 1999 (Jul); 22 (6): 376–381
Burton AK, Tillotson
KM, Cleary J.
Single-blind Randomised Controlled Trial of Chemonucleolysis and Manipulation in the Treatment of Symptomatic Lumbar Disc Herniation
European Spine Journal 2000 (Jun); 9 (3): 202–207
Cherkin DC, Deyo RA,
Battie M, Street J, Barlow W.
A comparison of physical therapy,
chiropractic manipulation, and provision of an educational booklet for the
treatment of patients with low back pain.
New England Journal of
Medicine 1998; 339(15): 1021-1029.
Evidence-based clinical guidelines for the management of acute low back
pain: Response to the guidelines prepared for the Australian Medical
Health and Research Council.
Journal of Manipulative and Physiological
Therapies 2001; 24(3): 214-220.
Freeman MD, Rossignol
A critical evaluation of the methodology of a low-back pain clinical
Journal of Manipulative and Physiological Therapies 2000;
pain, science, politics and money.
The Chiropractic Report November
Fables or Foibles: Inherent Problems with RCTs
J Manipulative Physiol Ther 2003 (Sep); 26 (7): 460–467
Royal College of
Unpublished update of CSAG guidelines (reference
Response to Meta-Analysis Published By Assendelft Et Al.
www.FCER.org, posted 07/09/03.
Return to the LOW BACK PAIN Section