Select Committee on Science and Technology Sixth Report


Difficulties of Randomised Controlled Trials (see para 7.25)

Concerns over RCTs distorting a therapy or disguising its efficacy are not the unique concerns of CAM practitioners. Vincent & Furnham suggest that as attempts to apply the RCT to a wider and wider range of treatments have occurred, more and more problems have been uncovered. They list 10 such problems:

(i)"Problems may arise because subjects randomised to different treatment groups may meet and discuss their treatment. Assignment to natural groups (e.g. comparison of two school districts) may be preferable to randomisation.
(ii)Blinding may not be feasible for some treatments...there is no clear equivalent to placebo drugs for some treatments.
(iii)Participation in a study may affect the behaviour of people taking part. Simply being monitored and assessed regularly may have a beneficial effect…
(iv)Subjects agreeing to take part in a trial may not be typical of the general population of patients with that particular problem. Entry criteria are strict to ensure comparability between groups…Patients with atypical symptoms, multiple problems or a poor prognosis may be excluded.
(v)Reduced compliance with treatment because of the possibility of receiving placebo treatment may arise.
(vi)Using standard treatments in the trial may sometimes be in a sense artificial and may have little relevance to clinical practice. Treatment within the context of a controlled trial may have to be precisely specified at the outset, which inhibits a more flexible patient-centred approach. The trial may therefore not be a true test of the therapy as used in clinical practice and the needs of the patient may conflict with the requirements of research.
(vii)Individual variations in response are often ignored in an analysis that only considers average group responses. Patients who are made worse by the treatment may not be given enough attention in reports…
(viii)Ethical problems may arise in a variety of contexts, particularly where placebo treatments are involved, or the patient or clinician has a marked preference for one treatment option over another. These concerns increase when the disease is potentially disabling or life-threatening.
(ix)The main outcome measure, based on clinical assessment and objective tests, may not reflect the patients' perspectives of what constitutes an important and beneficial change. Patients may be more concerned with the quality of their lives, which may not be closely linked with changes in biochemical parameters or other disease indicators. However, quality-of-life measures are now much more widely used.
(x)The concern with eliminating the placebo effect when assessing a treatment in relation to a comparable placebo may mean that important psychological variables are neglected."

All these methodological issues apply to both conventional and CAM treatment trials. Therefore CAM is not necessarily a special case requiring radically new methodologies.