Primary Care Physicians and Complementary-Alternative Medicine

Primary Care Physicians and Complementary-Alternative Medicine: Training, Attitudes, and Practice Patterns

Brian M. Berman, MD, Betsy B. Singh, PhD, Division of Complementary Medicine, Department of Family Medicine, University of Maryland at Baltimore; Susan M. Hartnoll, Behavioral Research Group, Baltimore; B. Krishna Singh, MB, PhD, Research and Statistical Consultant Group, Baltimore;D. Reilly, MB, FRCP, MRCGP, Research and Statistical Consultant Group, Scotland

[J Am Board Fam Pract 11 (4):272-281, 1998 1998 American Board of Family Practice]

Abstract

Background: Physician interest in complementary medicine is widely documented in many Western countries. The extent of level of training, attitudes toward legitimacy, and use of complementary therapies by US primary care physicians has not been extensively surveyed. We conducted a national mail survey of primary care physicians to explore these issues.
Methods: Primary care specialties represented were family and general practice, internal medicine, and pediatrics. A total of 783 physicians responded to the survey. For the multivariate analysis, sample weights were assigned based on specialty. Assessments were done for training, attitudes, and usage for complementary medicine. Additional data collected included years in practice, specialty, and type of medical degree.
Results: Biofeedback and relaxation, counseling and psychotherapy, behavioral medicine, and diet and exercise were the therapies in which physicians most frequently indicated training, regarded as legitimate medical practice, and have used or would use in practice. Traditional Oriental medicine, Native American medicine, and electromagnetic applications were least accepted and used by physicians.
Conclusions: Many psychobehavioral and lifestyle therapies appear to have become accepted as part of mainstream medicine, with physicians in this study having training in and using them. Such therapies as chiropractic and acupuncture appear to be gaining in acceptance despite low training levels among physicians. Those in practice more than 22 years had the least positive attitudes toward and use of complementary therapies. Osteopathic physicians were more open than medical physicians to therapies that required administering medication or a procedural technique. In the multivariate analysis, attitude and training were the best predictors of use.

Introduction

Therapies currently not taught or used in Western or US medical schools or institutions are grouped within the general classification of complementary and alternative medicine.[1,2] A widely divergent group of more than 150 different practices representing a "hodgepodge of beliefs and treatments"[3] falls within this heading. In one of the most extensive efforts to map the field of complementary or alternative medicine, the Office of Alternative Medicine, in a recent report to the National Institutes of Health (NIH), grouped these practices into seven broad categories.[4]

Physicians' attitudes toward alternative treatments vary among countries, suggesting that the distinction between alternative and conventional medicine is not always clear-cut and that many therapies previously considered fringe have become more accepted and used. Studies in many Western countries[5-17] indicate that physician interest in the use of complementary or alternative therapies appears substantial, but scientific evidence does not appear to be the basis for their interest.[11] In the United States, a survey of primary care physicians in the Chesapeake Bay area found physicians to be not only open to using or referring patients for certain complementary therapies but also interested in receiving training in many complementary therapies.[16] Other studies have found between 55 and 94 percent of physicians to be willing to refer their patients for a complementary therapy, although fewer (11 to 36 percent) were practicing some form of complementary medicine themselves.[12,17] These studies have been limited either geographically or in size, and more comprehensive surveys are needed regarding US primary care physicians' attitudes toward the use of complementary or alternative medicine practices.

To gauge such attitudes toward complementary medicine, we conducted a nationwide survey of primary care physicians during late 1994 through 1995. Survey questions addressed (1) which complementary therapies physicians considered to be legitimate medical practices, (2) in which therapies physicians had been trained, and (3) whether physicians personally practiced various complementary therapies. Based on the results of an earlier regional study,[16] we hypothesized that the greater the knowledge of complementary medical practices (as measured through training), the more positive the attitudes toward such practices, and the more likely physicians would utilize such practices for their patients. It was also hypothesized that knowledge of, attitudes toward, and practice of complementary medicine would vary based on physician specialty, type of medical degree, and number of years in practice.

Methods

Population

The survey sample was drawn from the 1994 American Medical Association (AMA) membership list of family practice, general practice, internal medicine, and pediatric physicians. Researchers requested name, address, degree, and specialty information for members describing themselves as direct patient care providers in the areas mentioned above. Of the 150,012 physicians meeting the criteria, 65,177 (43.5 percent) were general or family practitioners, 55,537 (37 percent) were specialists in internal medicine, and 30,264 (20.2 percent) were pediatricians. The sampling frame was a random, hierarchical, stratified sample selected proportionally by specialty and then by state or territory

It should be noted that AMA membership does not include all primary care physicians practicing in the United States, and thus any generalizations are limited to those who are members of the AMA. Nevertheless, the AMA listing is the largest available data source.

Sampling

Poor physician response rates to surveys have been repeatedly documented.[18-21] Although recommendations for overcoming this problem include multiple mailings[22] with a Total Design Method proposed by Dillman,[23] researchers have found that physician response rates can remain poor (approximately 16 percent) even with the multiple-wave data-collection method described by Dillman and others.[24,25] Even though data from general public surveys are mixed in terms of the differences in early and late responders,[26,27] Leslie[28] found that surveys of homogeneous groups did not require high response rates for generalizability.

Because physicians are a relatively homogeneous group compared with the general public, they might not require large samples to ensure external validity of the data.[29] Many researchers have found no differences in early and late responders on demographic characteristics.[30,31] When Sobal and Ferentz[32] tested Leslie's contention that physicians are a homogeneous group and that high response rates are not necessarily required for generalizability, they found that the additional responses received in the second wave of their test mailing did not alter the representativeness of the sample or change the results markedly. In a second study Sobal et al[33] found that physician samples, particularly within specialty groups, might not require extensive follow-up efforts, nor did data lack external validity with low response rates.

Realizing that response rates from physicians might be poor in spite of a full-field effort, that each specialty sample would be homogeneous, and that the larger the population from which the sample is drawn, the fewer respondents are necessary for representativeness if the sample is chosen randomly,[34] the researchers determined to calculate a 4 percent error rate for sample size by oversampling based on an estimated response rate of approximately 20 percent.[35]

Response Rate

A 4 percent margin of error rate required the return of approximately 602 questionnaires; 783 were returned. Sample weights were assigned for multivariate analyses based on proportions of each specialty in the sample and subsequent questionnaire returns. Among the three different specialty groups, the response rates were family physicians and general practitioners, 10.6 percent; internal medicine, 13.7 percent; and pediatricians, 31.7 percent. Each specialty sample was weighted in proportion to the total population, which was the basis for the initial sampling. The sample weights used for the multivariate analyses were as follows: general and family practice 1.54, internal medicine 1.13, and pediatrics 0.53.

Data Collection

We prepared a cover letter, revised and refined the survey instrument previously used in an earlier regional survey, and provided a mailer to facilitate easy return. Databases were formatted and maintained to track respondent feedback (completion, refusal, change of address, retirement, and so forth). Three mailing waves were followed by prompts. A closure card requested the noncompliant survey members to please return the survey instrument, request another one, or indicate specifically why compliance was not possible.

Assessment Variables

The three primary variables in this study were (1) training, (2) attitudes, and (3) usage. Training was defined as any training in complementary or alternative practices and was used as an indicator of knowledge derived from a formalized training process. Attitudes were defined as the extent to which these physicians considered legitimate each of the listed complementary and alternative medicine practices. Usage was divided into "actual use" and "would use in practice"; for the purposes of the multivariate analysis, however, actual use and would use in practice were combined. Throughout the questionnaire, 19 specific complementary and alternative practices were listed along with an "other" category. These practices were chosen based on earlier surveys conducted in Britain and the United States,[7,16] and because they represented each of the main complementary and alternative medicine categories[36] and some of the main complementary or alternative practices used in the United States and other Western countries.

Analytical Approach

Whereas most of the analyses were descriptive and bivariate, we used a multivariate model in the last section of the results to explore the effects of a set of variables on overall use of complementary or alternative practices. For the multivariate analyses, the training and attitudes variables were summated across the 19 complementary and alternative practices. Regression analyses were performed to test a two-stage independent variable model to predict physician use of complementary or alternative practices. The model was block recursive; that is, specialty, type of degree, and years of practice were entered first, then training and attitudes.

Results

Profiles of Responding Physicians

The mean age of respondents was 48 years. Most of the physicians were male (74 percent), white (89 percent), doctors of medicine (91.6 percent), involved in direct patient care (94 percent), and associated with a group or clinic-based practice (53 percent). Most spent a mean of nearly 46 hours per week in clinical care and a mean of 20 years in practice. Because response rates were expected to be low (indeed, they confirmed previously stated assertions that physicians responding to mail surveys are likely not to respond in reasonable proportions), an analysis was conducted to determine any differences based on age, sex, geographic region, and type of specialty (ie, variables available from the total AMA membership database). The results indicated that there did not appear to be any bias based on these variables between the population and the responding sample, though proportionately more pediatricians responded than any other specialty. For this reason, weights were assigned for the multivariate analysis based on specialty, followed by missing responses as discussed in the methods section.

Training

Training was measured by the question, "Have you had specific courses or training in any of the following?" The four choices for each complementary and alternative practice were none, some, a lot, and advanced. For purposes of this analysis, none was dichotomized with use (some, a lot, and advanced).

As shown in Table 1, areas where the most training was reported were diet and exercise, counseling and psychotherapy, behavioral medicine, and biofeedback and relaxation, with a range of 84.2 to 58.4 percent. Between 19.1 and 11.0 percent of physicians reported training in chiropractic, acupuncture, acupressure, homeopathic medicine, herbal medicine, and art therapy. Less than 9 percent reported training in traditional Oriental medicine, electromagnetic applications, and Native American medicine.

Using chi-square analysis, we were able to explore variations in training based on the specialty of the primary care physicians. Only training in art therapy, traditional Oriental medicine, and Native American medicine was not significantly different among specialties; prayer and homeopathic medicine were different at P < 0.05, and training in all other practices was significant at P < 0.01. Overall, fewer pediatricians and more family physicians and general practitioners had training in most of the complementary or alternative practices.

Those relatively few significant differences among the respondents were not uniform based on years in practice (diet and exercise, behavioral medicine, vegetarianism, and meditation, P < 0.05). Physicians who had osteopathic degrees were more likely than those with degrees in medicine to have training in massage therapy, therapeutic touch, chiropractic, acupressure, homeopathic medicine (P < 0.01), Native American medicine, and megavitamin therapy (P < 0.05).

Attitudes

Attitudes were measured by asking respondents to indicate one of the three following responses toward each of the specific complementary and alternative practices: (1) I can't say; I know very little about it; (2) it is a legitimate medical practice; and (3) it belongs outside medicine. For the purposes of analysis, responses 1 and 3 were collapsed into a single response category.

As evident in Table 2, those complementary and alternative therapies with the highest level of acceptance as legitimate medical practices were diet and exercise, counseling and psychotherapy, biofeedback and relaxation, and behavioral medicine (98.0 to 85.8 percent). Between 57.0 and 33.3 percent of the respondents indicated the following practices to be legitimate: hypnotherapy, massage and therapeutic touch, acupuncture, vegetarianism, meditation, chiropractic, and prayer and spirituality. Less than 30 percent of the respondents indicated that art therapy, acupressure, herbal medicine, megavitamins, and homeopathic medicine were legitimate medical practices, and less than 15 percent indicated that traditional Oriental medicine, electromagnetic applications, and Native American medicine were legitimate.

Using chi-square analysis, diet and exercise, counseling and psychotherapy, biofeedback and relaxation, acupuncture, vegetarianism, herbal medicine, traditional Oriental medicine, and Native American medicine did not show variations in attitude based on medical specialty. All others showed significant differences of at least P < 0.05.

Physicians in practice for more than 22 years were the least likely to perceive biofeedback and relaxation (P < 0.05), meditation (P < 0.05), herbal medicine (P < 0.01), chiropractic (P < 0.01), acupuncture (P < 0.01), and acupressure (P < 0.01) as legitimate medical practice; yet they were the most accepting of electromagnetic applications (P < 0.05). Herbal medicine, homeopathy, acupressure, chiropractic, and massage and therapeutic touch were more likely to be perceived as legitimate by those with osteopathic degrees than those with medical degrees (P < 0.01).

Usage

Usage was measured by asking respondents to indicate expertise in the listed complementary and alternative practices along three dimensions: (1) have used, (2) would consider using, and (3) would not consider using.

Have Used in Own Practice

Table 3 shows that 92.3 percent of the physicians reported using diet and exercise and 71.2 percent have used counseling and psychotherapy in their practice. Behavioral medicine and biofeedback and relaxation were therapies practiced by 47.3 and 44.1 percent of the reporting physicians, respectively. Massage and therapeutic touch, prayer and spirituality, vegetarianism, and meditation were used by 33.7 to 24.0 percent of the sample. Between 19.9 and 10.1 percent reported using hypnotherapy, chiropractic, megavitamins, acupuncture, and acupressure in their own practices. Less than 9.0 percent used herbal medicine, homeopathic medicine, art therapy, electromagnetic applications, Native American medicine, and traditional Oriental medicine.

Would Use in Own Practice

Respondents additionally indicated that they would be willing to use the following therapies: diet and exercise (6.5 percent), counseling and psychotherapy (24.5 percent), behavioral medicine (43.8 percent), and biofeedback and relaxation (47.6 percent). Taken together, the percentage of respondents who have used or would use these four complementary therapies ranges between 98.8 and 91.1 percent. Physicians willing to use vegetarianism (39.0 percent), prayer and spirituality (32.4 percent), meditation (42.8 percent), acupuncture (48.7 percent), hypnotherapy (48.0 percent), and massage and therapeutic touch (30.9 percent) brought the total range of physicians who have used or would use these therapies to between 60.4 to 67.9 percent. Less than 40 percent of the physicians were willing to use art therapy (39.5 percent), acupressure (36.5 percent), herbal medicine (34.3 percent), traditional Oriental medicine (33.5 percent), Native American medicine (29.8 percent), chiropractic (29.0 percent), homeopathic medicine (27.9 percent), megavitamin (21.3 percent), and electromagnetic applications (20.6 percent) in their practice. Combining those who would use with those who have used these therapies produces a range of 48.2 to 24.4 percent.

Using chi-squared analysis, we found that, where usage patterns of complementary and alternative practices varied by specialty (Table 3), pediatricians were less likely to use complementary and alternative therapies in their own practice (vegetarianism, chiropractic, and herbal medicine, P < 0.05; massage and therapeutic touch, megavitamin, meditation, acupressure, acupuncture, homeopathic medicine, P < 0.01). Family physicians and general practitioners were most likely to use these therapies.

Those in practice for more than 22 years were least likely to use complementary and alternative therapies in their professional role. Osteopathic physicians were more likely than medical physicians to use such therapies as massage and therapeutic touch, acupressure, chiropractic, and herbal medicine (P < 0.01), and homeopathy and megavitamins (P < 0.05).

Multivariate Model

We built several regression equations to test a two-stage independent variable model that would predict physician use of complementary and alternative practices. In stage 1 of the regression analysis, use of complementary and alternative therapies was predicted from three precursor variables (ie, specialty [dummy coded], type of degree [dummy coded], and years in practice).

The results indicated that, of the three precursor variables, only length of practice was significant (P < 0. 05). When these variables were entered in conjunction with training and attitudes, none of the precursor variables was found to be significant. Attitudes toward complementary and alternative practices were the most unique predictors (P < 0.001, R2 = 0.291), followed by training in complementary and alternative practices (P < 0.001, R2 = 0.120). Strong collinearity between training and attitudes existed. The total explained variance for the model with all precursor variables was R2 = 0.431.

Discussion

The four areas of complementary and alternative medicine in which a majority of the physicians in this study had training, that they used in practice, and that they clearly considered to be a part of mainstream medical practice were diet and exercise, counseling and psychotherapy, behavioral medicine, and biofeedback and relaxation. This group of practices, often categorized in complementary and alternative medicine publications under the rubric of mind-body therapies,[4] conforms to the structural definition of complementary and alternative medicine as therapies not generally taught in US medical schools and institutions. Nevertheless, these practices are perhaps a good example of therapies that have moved with time from the fringe toward the mainstream. Blumberg et al[17] have also found that US physicians use or refer patients most often for relaxation and lifestyle and diet therapies. The high percentage of physicians reporting training in and usage of these therapies is interesting, because at best only 53 to 58 percent of US medical schools include required or elective courses in nutrition or relaxation.[37-39] Indeed, in 1994-95 only 22 percent of US medical schools required nutrition courses, and the curricula of such courses have been described as chaotic and haphazard.[37,39]

Other therapies that focus on psychobehavioral and lifestyle change, such as vegetarianism, meditation, and hypnotherapy, were considered legitimate by a moderate percentage of the physicians. Similarly, a moderate percentage had had training in these therapies and had used or were willing to use these therapies in practice.

At the other end of the spectrum were ethnomedicines, such as traditional Oriental medicine and Native American medicine, as well as practices that involve administration of some type of medication, such as megavitamins, homeopathy, and herbal medicine. The lowest percentage of physicians had training in these complementary and alternative practices, regarded them as legitimate medical practices, and used them. Although these therapies seem typically not to be accepted or used by physicians in the United States,[12,16,17] homeopathy and herbal medicine are two of the most popular complementary practices among primary care physicians in such countries as Germany and the Netherlands.[15,40,41] It is interesting to note that acceptance of chiropractic and acupuncture by physicians in this study appears to be quite high despite low training levels. Although the rate of use of these therapies is low, physician acceptance would appear to be higher when rates of attitudes toward legitimacy and rates of have used and would use are considered. This finding is consistent with acceptance rates in other countries.[7-10,13,14,40-44]

When the results of the study were analyzed for differences based on specialty, years in practice, and degree type, some interesting patterns emerged. Compared with the other specialists, far fewer pediatricians appeared to be knowledgeable about and open to complementary and alternative practices. Complementary and alternative therapies tended to be used and accepted least by physicians who had been in practice more than 22 years.

A significantly greater percentage of osteopathic physicians than medical physicians were open to two general groupings of complementary and alternative practices: (1) therapies involving administration of a medication, and (2) practices using procedural techniques. In general, osteopathic physicians had more training in these therapies, were more likely to consider them legitimate, and were more likely to have used them. It is perhaps not surprising that this second grouping of procedural techniques was more accepted and used by osteopathic physicians given that manipulative therapy is considered the chief point of departure of osteopathic medicine from orthodox medical practice.

Overall, nearly 20 percent or more of the physicians in this study had used 9 of the 19 listed therapies and one third or more were open to using 17 of them (have used or would use combined). From 44 percent to 96 percent of physicians had referred at least 1 patient for one or more of nine of the therapies. Training and attitudes were the strongest predictors of usage as suggested by the multivariate model.

Our finding that knowledge of a therapy (as measured in this study through training) best predicts its acceptance and usage mirrors the results of our earlier Chesapeake Bay region study,[16] suggesting once again that familiarity with, not necessarily scientific evidence of, a therapy plays a major role in its acceptance. White[45] has estimated that in conventional medical practice only 10 to 20 percent of all procedures have been shown to be efficacious by controlled trials; therefore, it is not surprising that physicians rely on experiential knowledge.[46] Although solid empirical studies of complementary and alternative therapies are few and use a variety of scientific methodologies, the body of research about complementary therapies that does exist merits review as a building block to progressive scientific rigor.[47-51] Efforts in this regard are being undertaken by the recently established NIH Office of Alternative Medicine and by NIH-funded university centers for research in complementary medicine.

Some limitations to this study need to be addressed. First, this group of primary care physicians included a very limited proportion of physicians trained as osteopathic physicians. This difference is an artifact of the number of osteopathic physicians belonging to the AMA and thus included in the membership list of primary care physician groups selected for the sample. Because some significant differences were found in use of complementary and alternative practices by type of degree, it might be interesting to examine these findings further. Second, this survey was limited to primary care physicians for reasons of costs. Complementary and alternative practices are frequently used for treatment of arthritis, cancer, substance abuse, pain, and many other diseases[2,52-55] managed by other medical specialists. A more extensive survey of physicians might further clarify the actual usage of complementary and alternative medicine across all accepted specialties. Finally, only 19 of more than 150 complementary and alternative practices were included in this survey. Future researchers might choose to eliminate those therapies included in this survey that are now considered mainstream and offer a wider array of other complementary and alternative therapies as well as investigate and study parameters within disease specific models.

Primary care physicians in the United States appear to be accepting and using many therapies previously considered unorthodox and are open to others that until recently have been unheard of or shunned by the medical establishment. The results of this study indicate that acceptance and usage of complementary and alternative practices are strongly predicted by a physician's knowledge of and attitudes toward a therapy. Knowledge of and familiarity with any therapy is a necessary prerequisite for sound clinical judgments when caring for patients. In light of the increasing interest among physicians and acceptance of complementary medicine among the general public, research is needed to evaluate these therapies. When educational opportunities are provided to physicians to assist them with practice and treatment decisions, the best interests of their patients will be served.

Submitted, revised, 7 October 1997.

Address reprint requests to Brian M. Berman, MD, Division of Complementary Medicine, Department of Family Medicine, University of Maryland at Baltimore, Kernan Hospital Mansion, 2200 Kernan Ave, Baltimore, MD 21207.

This study was supported by a grant from the Boiron Homeopathic Foundation, The American Academy of Family Physicians, and the Laing Foundation, Thera Trust, and the National Institutes of Health Office of Alternative Medicine.

References

  1. Morgan D. editor. Complementary medicine: new approaches to good practice. London: Oxford University Press, 1993.
  2. Eisenberg DM, Kessler RC, Foster C, Norlock FE, Calkins DR, Delbanco TL. Unconventional medicine in the United States. Prevalence, costs, and patterns of use. N Engl J Med 1993;328:246-52.
  3. Murray RH, Rubel AJ. Physicians and healers -- unwitting partners in health care. N Engl J Med 1992;326:61-4.
  4. Berman BM, Larsen D, editors. Alternative medicine: expanding medical horizons. A report to the National Institutes of Health on alternative medical systems and practices in the United States. Bethesda, Md: National Institutes of Health, Office of Alternative Medicine, 1994. (NIH publication no. 94-066.)
  5. Bouchayer F. Alternative medicines: a general approach to the French situation. Compl Med Res 1990;4(2):4-8.
  6. Visser J. Alternative medicine in the Netherlands. Compl Med Res 1990;4(2):28-31.
  7. Reilly DT. Young doctors' views on alternative medicine. Br Med J Clin Res Ed 1983;287:337-9.
  8. Wharton R, Lewith G. Complementary medicine and the general practitioner. Br Med J Clin Res Ed 1986;292:1498-500.
  9. Perkin MR, Pearcy RM, Fraser JS. A comparison of the attitudes shown by general practitioners, hospital doctors and medical students towards alternative medicine. J R Soc Med 1994;87:523-5.
  10. Anderson E, Anderson P. General practitioners and alternative medicine. J R Coll Gen Pract 1987;37:52-5.
  11. Schachter L, Weingarten MA, Kahan EE. Attitudes of family physicians to nonconventional therapies. A challenge to science as the basis of therapeutics. Arch Fam Med 1993;2:1268-70.
  12. Borkan J, Neher JO, Anson O, Smoker B. Referrals for alternative therapies. J Fam Pract 1994;39:545-50.
  13. Hadley CM. Complementary medicine and the general practitioner: a survey of general practitioners in the Wellington area. N Z Med J 1988;101:766-8.
  14. Marshall RJ, Gee R, Israel M, Neave D, Edwards F, Dumble J, et al. The use of alternative therapies by Auckland general practitioners. N Z Med J 1990;103:213-5.
  15. Himmel W, Schulte M, Kochen MM. Complementary medicine: are patients' expectations being met by their general practitioners? Br J Gen Pract 1993;43:232-5.
  16. Berman BM, Singh BK, Lao L, Singh BB, Ferentz KS, Hartnoll SM. Physicians' attitudes toward complementary or alternative medicine: a regional survey. J Am Board Fam Pract 1995;8:361-6.
  17. Blumberg DL, Grant WD, Hendricks SR, Kamps CA, Dewan MJ. The physician and unconventional medicine. Alter Ther 1995;1(3):31-4.
  18. Cartwright A. Professionals as respondents: variation in and effects to mailed questionnaires and telephone interviews in a mixed mode health survey. Am J Public Health 1978;75:1338-40.
  19. Deutscher I. Physicians' reaction to a mailed questionnaire: a study in resistance. Public Opinion Q 1956;20:599-604.
  20. Martin B. Don't survey physicians! Chicago: American Medical Association, 1974.
  21. Sudman S. Mailed survey of reluctant professionals. Eval Rev 1985;9:349-60.
  22. Shostek H, Fairweather WR. Physician response rates to mail and personal interview surveys. Public Opinion Q 1979;27:276-86.
  23. Dillman DA. Mail and telephone surveys: the total design method. New York: John Wiley & Sons, 1978.
  24. Babbie E. The practice of social research. 7th ed. Belmont, Calif: Wadsworth Press, 1995.
  25. Evans RW, Evans RI, Sharp MJ. The physicians survey of post-concussion and whiplash syndromes. Headache 1994;34:268-74.
  26. Brambilla DJ, McKinlay SM. A comparison of responses to mailed questionnaires and telephone interviews in a mixed mode health survey. Am J Epidemiol 1987;126:962-71.
  27. Koenig D, Martin G, Seiler L. Response rates and quality of data: a reexamination of the mail questionnaire. Can Rev Sociol Anthropol 1977;14:432-8.
  28. Leslie L. Are high response rates essential to valid surveys? Soc Sci Res 1972;1:323-34.
  29. Berk ML. Interviewing physicians: the effect of improved response rate. Am J Public Health 1985;75:1338-40.
  30. Goodman L, Jensen L. Economic surveys of medical practice: AMA's periodic survey of physicians, 1966-1978. In: Proceedings of the Third Biennial Conference of Health Services Research. Rockville, Md: 1981:56-65
  31. Berk M, Meyers S. Reason for nonresponse on the physicians' practice survey. In: Proceedings of the Social Statistical Association, Alexandria, Va, summer, 1980.
  32. Sobal J, Ferentz KS. Comparing physicians' responses to the first and second mailings of a questionnaire. Eval Health Profession 1989;12:329-39.
  33. Sobal J, DeForge B, Ferentz KS. Physician's response to multiply questionnaire mailings. Eval Rev 1990;14:711-22.
  34. Bernard HR. Research methods in anthropology: qualitative and quantitative approaches. 2nd ed. Thousand Oaks, Calif: Sage Publications, 1994.
  35. General social survey. Chicago: National Opinion Research Center, University of Chicago, 1994.
  36. National Institutes of Health, Office of Alternative Medicine workshop: State of the art of major areas of alternative medicine. Chantilly, Va, 14-16 September 1992.
  37. Feldman EB. Networks for medical nutrition -- a review of the US experience and future prospects. Am J Clin Nutr 1995;62:512-7.
  38. Friedman R, Zuttermeister P, Benson H. Unconventional medicine. N Engl J Med 1993;329:1201.
  39. McLaren DS. Nutrition in medical schools: a case of mistaken identity. Am J Clin Nutr 1994;59:960-3.
  40. Knipschild P, Kleijnen J, ter Riet G. Belief in the efficacy of alternative medicine among general practitioners in the Netherlands. Soc Sci Med 1990;31:625-6.
  41. Visser J. Alternative medicine in the Netherlands. Compl Med Res 1990;4(2):28-31.
  42. Wardwell WI. Alternative medicine in the United States. Soc Sci Med 1994;38:1061-8.
  43. Cherkin D, MacCornack FA, Berg AO. Family physicians' views of chiropractors: hostile or hospitable? Am J Public Health 1989;79:636-7.
  44. Vaskilampi T. The role of alternative medicine: the Finnish experience. Compl Med Res 1990;4(2):23-27.
  45. White KL. International comparisons of health care systems. Milbank Mem Fund Q 1968;46:117.
  46. Assessing the efficacy and safety of medical technologies. Washington, DC: Office of Technology Assessment, 1978.
  47. Richardson PH, Vincent CA. Acupuncture for the treatment of pain: a review of evaluative research. Pain 1986;24:15-40.
  48. Anderson R, Meeker WC, Wirick BE, Mootz RD, Kirk DH, Adams A. A meta-analysis of clinical trials of spinal manipulation. J Manipulative Physiol Ther 1992;15:181-94.
  49. Assendelft WJ, Koes BW, van der Heijden GJ, Bouter LM. The efficacy of chiropractic manipulation for back pain: blinded review of relevant randomized clinical trials. J Manipulative Physiol Ther 1992;15:487-94.
  50. Kleijnen J, Knipschild P, ter Riet G. Clinical trials of homeopathy. BMJ 1991;302:316-23.
  51. Patel M, Gutzwiller F, Paccaud F, Marazzi A. A meta-analysis of acupuncture for chronic pain. Int J of Epidemiol 1989;18:900-6.
  52. Visser GJ, Peters L, Rasker JJ. Rheumatologists and their patients who seek alternative care: an agreement to disagree. Br J Rheumatol 1992;31:485-90.
  53. Cassileth BR, Lusk EJ, Strouse TB, Bodenheimer BJ. Contemporary unorthodox treatment in cancer medicine. A study of patients, treatments, and practitioners. Ann Intern Med 1984;101:105-12 .
  54. Cronan TA, Kaplan RM, Posner L, Blumberg E, Kozin F. Prevalence of the use of unconventional remedies for arthritis in a metropolitan community. Arthritis Rheum 1989;32:1604-7.
  55. Kronenfeld JJ, Wasner C. The use of unorthodox therapies and marginal practitioners. Soc Sci Med 1982;16:1119-25.