The Trials of Evidence:
Interpreting Research and the Case for Chiropractic
The Trials of Evidence:
Interpreting Research and the Case for Chiropractic
If you are a clinician at work in a typical chiropractic practice you see many patients with acute and chronic back pain, neck pain and headaches.
If you are making best efforts to keep up with the ongoing flood of research and evidence-informed clinical guidelines you can feel confident that the scientific evidence now supports your clinical experience that spinal manipulation specifically, and chiropractic management incorporating manual care generally, are very helpful for most patients with these complaints. Therefore for example:
For the great majority of patients with both acute and chronic low-back pain, namely those without diagnostic red flags, spinal manipulation is recommended by evidence-informed guidelines from many authoritative sources – whether chiropractic (the UK Evidence Report from Bronfort, Haas et al. ), medical (the 2007 Joint Clinical Practice Guideline from the American College of Physicians and the American Pain Society ) or interdisciplinary (the European Back Pain Guidelines ).
For the great majority of patients with acute and chronic neck pain, and those with cervicogenic headache, spinal manipulation is similarly recommended, most recently and authoritatively by the Bone and Joint Decade Neck Pain Task Force . For headache, including migraine headaches, see evidence reviews and recommendations from the Evidence-Based Practice Center at Duke University  and Bryans Descarreaux et al. in Canada .
What are we to make, then, of a new systematic review for the Cochrane Collaboration, looking at chronic back pain and published last month in Spine? This is from Rubenstein, van Middelkoop et al., an experienced research team at the VU University, Amsterdam which includes noted epidemiologist Dr. Maurits van Tulder, so will attract attention. It concludes that the evidence suggests “there is no clinically relevant difference between spinal manipulative therapy (SMT) and other interventions for reducing pain and improving function in patients with chronic low-back pain”. .
What is going on? What is the explanation for such glaring inconsistency? What is a clinician to do? This issue of The Chiropractic Report addresses these questions, and discusses:
b) What can fairly and appropriately be said about the superior effectiveness of skilled manipulation/mobilization for patients with chronic back pain? Even this new Cochrane Review acknowledges better short term results for SMT than any alternative treatment in terms of reduced pain and improved function.
c) Why chiropractic management is and must be much more than skilled manual treatments.
d) The best case to support chiropractic management of patients with both acute and chronic spinal pain and disability – as also for patients with other conditions or seeking preventive/maintenance care. There is a convincing case for chiropractic care but it is not one of markedly superior effectiveness. The evidence does not support that.
What it does support, however, is that chiropractic care is:
At least as effective as any other alternative treatment approach, better than most with which it has been compared.
Produces much higher patient satisfaction rates.
In a number of countries, is more readily accessible than other options.
B. Systematic Reviews
Case reports and retrospective reviews of past cases offer valuable ideas for research, but to reach conclusions on the effectiveness of a treatment for a given population of patients one must move to these higher levels of evidence:
b) Randomized controlled trial (RCT). This provides superior evidence of effectiveness. A significant advantage of an RCT over a case series is that the treatment being tested for effectiveness can be compared with no treatment and/or a sham/placebo treatment and/ or another treatment. Patients are randomly assigned to different groups. The trial can test the effectiveness of one component of care (e.g. spinal manipulation or an exercise protocol) or overall management more representative of daily practice. An example of the latter is the British Medical Research Council Trial of acute and chronic back pain patients by Meade et al.  in which patients were randomly assigned to either chiropractic care or hospital out-patient care. All chiropractic patients received spinal manipulation and as a group did significantly better in terms of reduced pain and disability than those in the hospital out-patient group.
c) Systematic reviews. Most treatments are not in fact supported by one or more RCT s. For them best evidence must still be found in case series and clinical experience. Where there are RCT s for a treatment the quality varies enormously, and there are often conflicting results from different trials. As the volume of research built in the 1980s and 1990s so did the esearch method of systematic reviews. At first this was known as meta-analysis of research results. Results of different studies were pooled together to present a conclusion from all the relevant research. Efforts were made by individual researchers to measure for quality as well as quantity, but rules had not been developed.
The advent of the Cochrane Collaboration in 1993, a large international network of volunteer researchers presenting the research evidence online for all, led to much development and much greater sophistication of systematic review methods. At first the approaches developed for Cochrane reviews, and appearing in the online Cochrane Handbook for Systematic Reviews, was largely unchallenged. In recent years there has been growing criticism, and there are now many proponents of an alternative approach called best-evidence synthesis.
Cochrane Review Method. The fundamental concept is that the quality of individual studies/RCT s is scored according to a check list of items or criteria. For back pain reviews, for example, there is a check list of 11 items as in Table 1. No one challenges that these criteria are important. The problem is that trials are scored and rated on technical criteria alone. This appeals more to epidemiologists than clinician researchers. It has led to the frequent conclusion in Cochrane reviews that all the evidence is inconclusive – and that more evidence is necessary before clinical recommendations can be made.
Best-evidence Synthesis Method. This is just as rigorous scientifically, but does not use a scoring method or scale. It focuses on whether any design weaknesses in a trial actually led to bias in selection of patients/subjects or the information reported, or in other ways made results unreliable. This was the method of systematic review used by the Bone and Joint Decade Neck Pain Task Force in its widely praised report. The method was chosen as being more likely to lead to clinically relevant conclusions and, in the words of the Task Force, was considered “one of our key methodological strengths”.
The practical importance of this is seen in a 2007 study by van der Velde, van Tulder et al.  which compares these two methods of systematic review in assessing the studies on whether or not exercise is an effective management strategy for workers with neck pain. It is noted:
b) When scored for quality according to the Cochrane Back Review Group Guidelines, with the 12 trials then all considered at their various levels of quality or strength, “no treatment recommendation could be formulated”.
c) When assessed by best-evidence synthesis a clear treatment recommendation was possible. Four of the 12 trials were accepted as scientifically sound and with low risk of bias. Synthesis or combination of the results of these 4 trials, without the confounding influence of the other 8 weaker trials, supported the recommendation that “workers should be activated with exercise given its beneficial effect on patient perceived recovery”.
New Cochrane Review. The new systematic review by Rubenstein et al. relates to the effectiveness of “spinal manipulative therapy (SMT)” for chronic back pain (more than 12 weeks). SMT is defined as “any hands-on treatment, including both manipulation and mobilization of the spine”. As the authors themselves note “there are likely to be objections” to this leveling of the field – putting all SMT by all practitioners regardless of professional training and skill and for all chronic back pain patients into the same pool and then judging effectiveness.
Here are two illustrations of the problems with that. In the Meade et al. trial already mentioned all patients in the chiropractic treatment group received chiropractic manipulation, and did significantly better than patients in the hospital outpatient group – 84% of whom received physiotherapy manipulation and/or mobilization techniques according to Cyriax and Maitland. In this Cochrane review such distinctions are gone.
In a study by Khan, Cook et al.  patients with chronic neck pain from road traffic whiplash injuries who were referred for chiropractic manipulation from a UK hospital orthopedic department were classified in three sub groups – Group 1 with restricted range of neck movement but no neurological deficit, Group 2 with restricted movement but also neurological signs and/or symptoms, and Group 3 with no restricted neck movement or neurological signs or symptoms but disabling neck pain – indicating psychosocial factors.
There was significant improvement in patients in Groups 1 and 2 – patients with objective signs including restricted ROM – but not those in Group 3. Sorting out those with and without physical signs is clearly important. Most trials have made no such distinctions and submitted all chronic pain patients to the treatment being tested. To be fair to Rubenstein et al. they acknowledge this concern. They comment that “identifying subgroups seems important”, and that some studies do suggest that “clinically important effects are observed when treatment is matched to the patient’s signs and symptoms rather than provided to all patients with low-back pain”. However their review does not allow for that.
With respect to other features and conclusions of the review:
b) Over 30 RCTs were excluded, and their results therefore not considered, for what was seen as fatal design flaws, such as patients had either acute or chronic back pain and the number with chronic pain was unclear or under 50%; the contribution of SMT to treatment effect could not be discerned; and a high drop out rate at first assessment of treatment. This excluded many of the trials of chiropractic treatment from experienced and highly regarded researchers such as Meade et al. , Triano, McGregor et al. , Bronfort , Haas, Groupp et al.  and Muller and Giles .
c) This degree of scientific rigor meant that only 26 RCTs survived for inclusion in the review, with only 9 of these scored as having low risk of bias. These 26 were further subdivided into SMT vs. sham SMT (3), SMT vs. other treatment interventions (15), and SMT plus another intervention vs. the other intervention alone (5). This meant, for example, that conclusions on the effectiveness of SMT against sham SMT were made on the basis of (only) 3 studies “all with a high RoB (risk of bias)”. Such conclusions must be regarded as extremely tentative and unreliable.
d) With respect to conclusions on the effectiveness of SMT vs. other interventions (e.g. exercises, education, psychosocial interventions):
For pain, Rubenstein et al. report “there is high-quality evidence that SMT provides statistically significantly better pain relief than other interventions at 1 and 6 months” but also high-quality evidence against that at 12 months.
For functional status, “there is high-quality evidence that SMT provides statistically significantly better functional improvement at 1 month than the other interventions”, but not at 6 and 12 months.
For recovery as perceived by patients, there is moderate-quality evidence “that SMT provides a significantly better chance of recovery than the contrast interventions” at 1 month, but low-quality evidence that that difference is lost at 6 and 12 months.
e) Similarly, when SMT is added to another treatment intervention there is evidence that it has a statistically significant short-term effect on pain relief and functional status. However the size of all beneficial effects mentioned is assessed as small and, as mentioned at the beginning of this article, not clinically relevant.
f) SMT is reported as safe – in all the studies considered “no serious complications were observed”.
g) Rubenstein et al. conclude that the decision to use or refer patients for SMT, which is as good as but not markedly better than other treatments, should be based on costs, preferences of the patient and providers, and relative safety. Implications for research are that subgroups suitable for SMT should be identified, SMT should be evaluated as an additional or adjunct therapy in a multimodal package, and there is a need for cost-effectiveness studies. SMT may be more cost-effective “because the therapy is typically provided in a limited number of treatment sessions”, more limited than for example exercise therapy or behavioral treatments – but more studies are required on this.
C. Chronic Pain – The Case for Chiropractic
Effectiveness. Rubenstein et al. report that SMT has a significantly better effect on pain and function and a patient’s self-assessment of recovery than all other treatments during the first month of follow-up after treatment, but not at 12 months. That should surprise no-one, says Dr. Scott Haldeman, the US neurologist and research leader who chaired the BJD Task Force on Neck Pain and its Associated Disorders.
Chronic or persistent back pain, neck pain and headaches are biopsychosocial problems – they have physical and psychosocial causes. No medical or surgical treatment has long-term effects, observes Haldeman, with the only exception being any intervention that succeeds in causing psychosocial change in the patient leading to altered exercise, life style and coping strategies. 
What can be said on effectiveness about chiropractic practice for acute and chronic back and neck pain, and cervicogenic headaches including migraine, is:
b) With chronic pain it provides patients with a period of relief from pain and disability that represents a window of opportunity for long-term relief.
c) If chiropractic management is multimodal, including patient education and motivation, prescription and monitoring of exercises, and the use of supportive modalities and therapies such as massage and acupuncture as may be needed in individual cases, and other psychosocial interventions, this can bring long-term relief.
Safety. A strong part of the case for chiropractic treatment is safety, avoiding the much more frequent and serious complications of medications and surgery. This publication has reported at length on safety in the past and key points are:
b) Spinal manipulation – back pain. One of the central reasons that national and multi-disciplinary, evidence-based clinical guidelines have recommended spinal manipulation as a first line treatment option is its safety in comparison with use of medications, especially prescription medications. As Rubenstein et al. report, in all the RCT s of SMT they reviewed there was not a single serious adverse event.
c) Cervical spine manipulation. There is a perception by some, fuelled by media campaigns by critics of the chiropractic profession in North America and the UK in recent years, that neck manipulation has significant risk of harm. The evidence is that it does not. Key points and evidence are:
ii. Over years of study many expert panels, including those from the RAND Corporation (1996)  the Evidence-Based Practice Center at Duke University (2001)  and the Bone and Joint Decade Neck Pain Task Force (2008)  have recommended neck manipulation as appropriate for patients with neck pain and/or headaches after assessing the evidence on risks and benefits.
iii. The most thorough study investigating the association between chiropractic manipulation and stroke was by Cassidy, Boyle et al. for the BJD Neck Pain Task Force . Using a large government database in Canada that captured all strokes in Ontario over 8 years to March 2002, representing 109 million person years, they found:
Only 818 VBA strokes from all causes – 7.5 cases per million person years. In other words this is a very rare form of stroke from any cause.
The increased risk of stroke amongst those who had seen a chiropractor in the past 7 days was exactly the same as for those who had seen a family physician in that time.
This suggests that strokes are “associated with” chiropractic or medical visits in time rather than “caused by” such visits. Cassidy, Boyle et al. explain that an estimated 80% of VBA stroke patients have neck pain from artery dissection during the days before their stroke. From that point any neck movement can precipitate the stroke. Where there has already been damage to a vertebral artery and formation of a blood clot “a chiropractic manipulation or even simple range of motion examination by any practitioner could result in a thromboembolic event in a patient with pre-existing vertebral dissection” – in other words release of an embolus and stroke.
iv. Finally, the forces actually reaching the vertebral artery (VA) during neck manipulation have now been measured in sophisticated biomechanical research at the University of Calgary in Canada. Symons, Leonard and Herzog found that the maximum forces on a vertebral artery from chiropractic manipulation are no greater than those recorded during common diagnostic range of motion tests regularly performed by many health professionals, and provide only “approximately one ninth of the strain” required to produce first mechanical failure in the tissues of the VA. 
Cost-effectiveness. In the Mercer Report leading US health economists at Mercer Health and Benefits and Harvard University assessed the cost-effectiveness of chiropractic management of patients with neck and back pain. They noted that “low-back and neck pain are extremely common conditions that consume large amounts of healthcare resources” and conclude that “when considering effectiveness and cost together, chiropractic physician care for low-back and neck pain is highly cost-effective, and represents a good value in comparison to medical physician care and to widely accepted cost-effectiveness thresholds”. 
The question of cost-effectiveness, both from the points of view of the individual patient and third party payors, is complex and no field for amateurs. Many studies by health sciences researchers without expertise in economics are inconsistent because of problems such as poor matching of patients, failure to include all costs, invalid attribution of costs and inadequate sample size. Relevant costs that must be incorporated are direct costs of care, costs arising from harm from treatments, compensation costs for disability and time of work, and other indirect costs incurred by patients, families and employers.
For a full discussion of cost-effectiveness, and the research evidence including the Mercer Report, see the November 2009 issue of The Chiropractic Report available at www.chiropracticreport.com under Past Issues. Some key points include:
b) A major issue for third party payors is substitution. Even if chiropractic care is cost-effective for patients with back and neck pain, headaches and other neuromusculoskeletal disorders, will a chiropractic benefit given to patients be an add-on cost similar for example to a dental benefit and most other benefits, and therefore increasing overall costs even though cost-effective in itself, or will chiropractic services given under the benefit truly “substitute” for more expensive medical care? The first large study, on that issue, based on data from 1.7 million members of a managed-care network in California, confirms that virtually all chiropractic services used by plan members were used in direct substitution for medical services. This was for all conditions seen by chiropractors – a range of 654 ICD-9 Codes covering neuromusculoskeletal disorders such as spinal pain, rib disorders, headache, extremity problems and myalgias or arthralgias. 
You may also enjoy our:
Cost-Effectiveness of Chiropractic Page
Patient satisfaction. Level of patient satisfaction, always important to patients, is becoming an increasingly important research area and outcome of care for third party payors. There may be debate about the comparative effectiveness and cost-effectiveness of chiropractic care for the prevention and management of various conditions, but no one can dispute the extensive research findings showing that chiropractic patients are generally very satisfied with their care and that patient satisfaction rates are consistently higher than for medical and other forms of care with which comparisons have been made.
High satisfaction rates were confirmed in a 2006 literature review and report of a new US national survey by Gary Gaumer, PhD, an independent health services researcher from the Department of Health Care Administration, Simmons College, Boston.  Gaumer concludes that “overall satisfaction levels amongst persons using chiropractic is very high” and that “this is remarkable given the fact that much of the financial burden of the care is borne by patients and that the preponderance of care is for the difficult chronic problems of the back and neck”.
Again, a previous issue of this Report available at www.chiropracticreport.com – the January 2007 issue – reviews the evidence and the reasons for patient satisfaction. A 1986 study treatments – it is based upon at least equal effectiveness, safety, cost-effectiveness and patient choice and satisfaction. In a number of countries it is also based on ready availability of services without delay.
The second message is that skilled manual treatments, central as they are and must remain to chiropractic care, are not enough to secure the future success of the profession. Various other interventions have similar effectiveness. Combination of treatments, such as spinal manipulation and exercise for chronic back and neck pain, sometimes brings superior results. Patient preferences and choice need to be accommodated, particularly since preferred treatments are usually more successful.
If doctors of chiropractic are to remain spine care or musculoskeletal specialists in primary care they, themselves and in collaboration with others, must offer what is now described as multi-modal care. This means a range of effective treatments including manual care, rehabilitative exercises, patient education and motivation concerning musculoskeletal pain, counseling on nutrition, exercise and healthy living, provision of orthopedic and orthotic supports, and availability of physical therapy modalities, acupuncture and massage therapy. There must be understanding, management and referral as necessary for psychosocial factors. As Meeker and Haldeman explain in the quote above the chiropractic clinical encounter has always involved much more than joint adjustment or manipulation. This formula for success must be built upon and expanded.
You may also enjoy our:
Patient Satisfaction With Chiropractic Page
1 Bronfort G, Haas M, Evans R, Leiniger B, Triano J.
Effectiveness of Manual Therapies: The UK Evidence Report
Chiropr Osteopat. 2010 (F eb 25); 18 (1): 3
2 Chou R, Qaseem A et al. (2007)
Diagnosis and Treatment of Low Back Pain: A Joint Clinical Practice Guideline from the American College of Physicians and the American Pain Society
Annals of Internal Medicine 2007 (Oct 2); 147 (7): 478–491 ~ FULL TEXT
3 European Guidelines for the Management of Acute and Chronic Non-Specific Low-Back Pain (2004)
4 Haldeman S, Carroll LJ, Cassidy JD, and the Scientific Secretariat, Bone and Joint Decade 2000-2010 Task Force on Neck Pain and its Associated Disorders (2008)
A Best Evidence Synthesis on Neck Pain: Findings from the BJD Task Force on Neck Pain and its Associated Disorders
5 McCrory DC, Penzien DB et al. (2001)
Evidence Report: Behavioral and Physical Treatment for Tension-Type and Cervicogenic Headache
Des Moines, Iowa, Foundation for Chiropractic Education and Research Product No. 2085.
6 Bryans R, Descarreaux M, Duranleau et al. (2011)
Evidence-Based Guidelines for the Chiropractic Treatment of Adults With Headache
J Manipulative Physiol Ther. 2011 (Jun); 34 (5): 274–289
7 Rubenstein SM, Van Middelkoop M et al. (2011)
Spinal Manipulative Therapy for Chronic Low-Back Pain: An Update of a Cochrane Review
8 Kirkaldy-Willis WH, Cassidy JD (1985)
Spinal Manipulation in the Treatment of Low-back Pain
Canadian Family Physician 1985 (Mar); 31: 535–540 ~ FULL TEXT
9 Meade TW, Dyer S, Browne W et al. (1990)
Low Back Pain of Mechanical Origin: Randomised Comparison of Chiropractic and Hospital Outpatient Treatment
British Medical Journal 1990 (Jun 2); 300 (6737): 1431–1437 ~ FULL TEXT
10 van der Velde G, van Tulder et al. (2007)
The Sensitivity of Review Results to Methods Used to Appraise and Incorporate Trial Quality Into Data Synthesis
11 Khan S, Cook J et al. (1999)
A Symptomatic Classification of Whiplash Injury and the Implications for Treatment
Journal of Orthopaedic Medicine 1999; 21 (1): 22–25
12 Triano JJ, McGregor M, Hondras MA et al. (1995)
Manipulative Therapy Versus Education Programs in Chronic Low Back Pain
Spine (Phila Pa 1976). 1995 (Apr 15); 20 (8): 948–955
13 Bronfort G (1989)
Chiropractic versus General Medical Treatment of Low-Back Pain: A Small-Scale Controlled Clinical Trial
Am J Chirop 2:145-50.
14 Haas M, Groupp E, Kraemer DF (2004)
Dose-response for Chiropractic Care of Chronic Low Back Pain
Spine J 2004 (Sep); 4 (5): 574–583
15 Muller R, Giles LG (2005)
Long-term Follow-up of a Randomized Clinical Trial Assessing the Efficacy of Medication, Acupuncture, and Spinal Manipulation for Chronic Mechanical Spinal Pain Syndromes
J Manipulative Physiol Ther 2005 (Jan); 28 (1): 3–11
16 Personal Communication, July 12, 2011.
17 Terrett AGJ (2001)
Current Concepts in Vertebrobasilar Complications Following Spinal Manipulation ~ FULL TEXT
NCMIC Group Ind. West Des Moines, Iowa.
18 Haldeman S, Kohlbeck FJ, McGregor M (1999)
Risk Factors and Precipitating Neck Movements Causing Vertebrobasilar Artery Dissection After Cervical Trauma and Spinal Manipulation
SPINE (Phila Pa 1976) 1999 (Apr 15); 24 (8): 785–794
19 Coulter ID, Hurwitz EL et al (1996)
The Appropriateness of Manipulation and Mobilization of the Cervical Spine ~ FULL TEXT
Santa Monica, California: RAN D, Document No. MR-781-CR.
20 Cassidy JD, Boyle E et al. (2008)
Risk of Vertebrobasilar Stroke and Chiropractic Care: Results of a Population-based Case-control and Case-crossover Study
SPINE (Phila Pa 1976) 2008 (Feb 15); 33 (4 Suppl): S176–183
21 Symons BP, Leonard T, Herzog W (2002)
Internal Forces Sustained by the Vertebral Artery During Spinal Manipulative Therapy
J Manipulative Physiol Ther 2002 (Oct); 25 (8): 504–510
22 Choudhry N, Milstein A (2009)
Do Chiropractic Physician Services for Treatment of Low-Back and Neck Pain Improve the Value of Health Benefit Plans? An Evidence-Based Assessment of Incremental Impact on Population Health and Total Health Care Spending Harvard Medical School, Boston, Mercer Health and Benefits, San Francisco.
23 Manga P. Angus D (1998)
Enhanced Chiropractic Coverage Under OHIP as a Means of Reducing Health Outcomes and Achieving Equitable Access to Select Health Services
Ontario Chiropractic Association, Toronto.
24 Legorreta AP, Metz RD, Nelson CF et al. (2004)
Comparative Analysis of Individuals With and Without Chiropractic Coverage: Patient Characteristics, Utilization, and Costs
Arch Intern Med 2004 (Oct 11); 164 (18): 1985–1892 ~ FULL TEXT
25 Gaumer G (2006)
Factors Associated With Patient Satisfaction With Chiropractic Care: Survey and Review of the Literature
J Manipulative Physiol Ther 2006 (Jul); 29 (6): 455–462
26 Paskowski I, Schneider M, Stevans J et al. (2011)
A Hospital-Based Standardized Spine Care Pathway: Report of a Multidisciplinary, Evidence-Based Process
J Manipulative Physiol Ther 2011 (Feb); 34 (2): 98–106
27 Meeker WC, Haldeman S (2002)
Chiropractic: A Profession at the Crossroads of Mainstream and Alternative Medicine
Ann Intern Med 2002 (Feb 5); 136 (3): 216–227 ~ FULL TEXT