J Manipulative Physiol Ther 2003 (Oct); 26 (8): 493–501 ~ FULL TEXT
Robert A Leach, DC, Patrick L Parker, Paul S Veal, DC
OBJECTIVE: To provide an entry-level, new technology reliability assessment of the PulStar computer-assisted, differential compliance spinal instrument.
SUBJECTS: Eighteen college students (9 male and 9 female) were recruited by announcements and personal contacts.
METHODS: Following approval of the consent process by the Institutional Review Board of Mississippi State University, a PulStar Function Recording and Analysis System (PulStarFRAS) device was evaluated for clinical reliability. Two examiners, blinded from data collection, used the instrument on individual subjects in random order (lying prone with their backs exposed) to administer light impulses (approximately equal to .9 J which produced a 3- to 4-lb force) at each segmental level throughout the cervical, dorsal, and lumbar spine using probe tips spaced 3 cm apart, straddling the spinous processes, while a computer recorded the findings (resistance on a scale of 0 to 25.5 lb force). Data were analyzed by Exploratory Data Analysis (EDA) with analysis of variance (ANOVA) testing and by use of the intraclass correlation coefficient (ICC). In addition, a mean test (ANOVA) was conducted to determine if a trend in variation occurred as a result of repeated light thrusts to the spine, independent of variance explained by different examiners.
RESULTS: Using EDA analysis and ANOVA, intraexaminer reliability for the 2 practitioners was very high but not perfect. This was confirmed by ICC statistics demonstrating good to excellent reliability for both practitioners (0.89 for the experienced practitioner, 0.78 for the newly trained practitioner). Interexaminer reliability of PulStar was similarly very high but not perfect based on EDA/ANOVA analysis and good to excellent (ICC = 0.87).
CONCLUSION: The PulStar mechanical adjusting device set to analysis mode appears to have good to excellent reliability when used by either an experienced or a novice (but trained) examiner. In addition, as a measure for resistance to a light thrust or spinal compliance, reliability was similarly good to excellent between the 2 doctors using the PulStar instrument.
From the Full-Text Article:
Of all the differential spinal compliance measures made with the PulStar unit, significant variance between 2 examinations by the same doctor occurred only at the occiput and C3 and only for the novice examiner, while significant variance between examiners occurred only at occiput and C4. The measurement at occiput involved placing the probes over the occiput in such a manner that they straddled the external occipital protuberance. It is quite possible that despite stabilizing the probes by resting them against the side of the first finger of the examiner's free hand, some sliding of the probes during discharge might have created unacceptable reliability at this level. Others using PulStar place the probes directly over the atlanto-occipital joints for the occipital measurement, and this may prove to be a more satisfactory arrangement. Further research will be needed to verify this finding and to determine whether an occipital site is even necessary. Obviously, no other vertebral segment that we tested posed the unique anatomy found at the occiput.
We were only mildly surprised to find poorer agreement at the C3 and C4 levels, after the results of our earlier pilot investigation had revealed poorest reproducibility at the C6 level. Only late in the pilot investigation did we begin having the patient flex the neck, and those were the very subjects that showed improved test-retest reliability. In the present inquiry, all subjects fully flexed the neck while lying prone, which certainly seems to improve palpation of the lower cervical spine, relatively increasing the difficulty in locating the shorter C3 and C4 spinous processes. Others have proposed that midcervical palpation is most difficult as well,  and ultrasound has been used to image the spine,  as well as “indentation” testing (ie, not a quick pulsed thrust like the PulStar uses, but rather pressure applied at a rate of 2.5 mm/s until a load of 1 N is attained, by use of a flat, rigid, 3 × 3 cm surface) combined with ultrasound imaging to improve examiner palpatory reliability.  Whether ultrasonic or other imaging is needed to perform more reliable midcervical PulStar compliance measurements remains to be determined. It may be that PulStar used as a more global measure of cervical compliance (rather than differentiating exactly which 2 segments are less compliant) would provide clinically relevant information, even without palpatory determination of the exact locations of the C3 and C4 vertebrae.
We may also speculate that the amount of neck flexion in our subjects might have varied significantly from examination to examination, producing a confounding variable that affected reliability of the C3–4 measure. In this regard, although we are unaware of studies on the cervical spine, certainly there are recent observations on the lumbar spine by Caling and Lee  that suggest posteroanterior stiffness varies significantly with the direction of applied force. Further investigations might want to control for this variable by monitoring the degree of cervical flexion during PulStar testing by use of electrogoniometry; however, up to 9° of measurement error in flexion/extension may still be a confounder.  Researchers might also utilize an instrument that establishes a 90° perpendicular to the spine, to determine if that is a source of error.
It is worthy to note that in the present study rules for use of the instrument were more stringent than those used in clinical practice and might have actually led to underestimation of clinical reliability. Hence, while researchers generally agree that establishing clinical reliability is easier in an experimental setting following strict protocols than in a busy practice where compensation depends on volume of services and not necessarily quality of care rendered, in the present investigation doctors were not allowed to repeat their examination even if they thought the PulStar probe had slipped, was at an angle other than 90° perpendicular to the spinous, or because they had miscounted the level of spinous that they were checking. In each of these cases, a practicing clinician can do a reexamination to check the data; in contrast, since this was a blinded investigation, doctors were not allowed the opportunity to recheck their work. Further research of PulStar reliability should include the possibility of the clinician repeating his examination and suggesting which examination should be used (while still blinded from data collection). In this way, we might know whether repeat trials, such as would be available to the clinician in private practice, would enhance the reliability of the procedure.
Two experienced clinicians (as opposed to 1 novice clinician and 1 experienced clinician) might not have difficulty with interexaminer reliability using the PulStar in the midcervical spine. However, despite some evidence of fatigue for the novice examiner, whose variance between trials increased slightly (0.4 lb force from the 1st to the 18th subject), his overall rate of variance ranged only from 0.8 to 1.2 lb force, not dissimilar from the more constant variance rate of 1.0 lb force observed by the experienced examiner. From the standpoint of clinical reliability, this is a wash; despite some differences, it appears that for both examiners an error range of 1.0 lb force might be expected between any 2 trials. The present investigation then revealed no clinically meaningful difference in reliability between the experienced and novice, but trained, investigator.
Finally, using the EDA graphic analysis, we provide a preliminary view of normal spinal compliance using the PulStar instrument, which may guide further research aimed at developing norms for specific populations. It is noteworthy that averaged data on 18 subjects from all 4 trials (2 doctors × 2 trials) indicate that spinal compliance was greatest in the lower cervical and lumbar spines and lowest over the occiput (control site) and upper dorsal spine. Certainly, further research will be necessary to confirm and extend these preliminary observations, comparing normal populations to patients in pain, for example. Also, it should be understood that this report did not measure the validity of the differential compliance analysis, a computerized analysis which measures the difference in spinal compliance between vertebral segments and triggers the PulStar to provide more pulsed adjustments to areas of fixation or poor compliance. Only further research of trial validity can determine the significance and clinical meaningfulness of the computerized analysis and of the computer-guided PulStar adjustment itself.
It is uncommon, if not rare, to find either spinal fixation or chiropractic subluxation detection strategies that have a high degree of intraexaminer and interexaminer reliability,  and this has prompted some to suggest abandoning research in this area altogether. [3, 26–27] Since no individual or panel of chiropractic experts to date has been able to agree on an operational definition or so-called gold standard dependent variable to measure subluxation, [28–31] we here make the assumption that if there are subluxation-free spines, it is more likely that we will find them in younger, pain-free individuals, whose spines have not yet been subject to decades of postural and physical insults. While we concede that we cannot rule out the presence of VSC, purported to influence nerves and viscera, in the young college students in our trial (primarily because we do not yet know how to measure subluxation complex), if they did have these lesions they apparently did not adversely affect clinical reliability of the apparatus we tested. Of course, we will only learn whether the PulStar measure correlates with outcomes and whether it is capable of serving as a mediating variable of VSC (ie, becoming part of a gold standard for VSC diagnosis) if clinical research on trial validity of differential compliance is conducted. 
The results then of the present inquiry on the reliability of the PulStar instrument set to the analysis mode, a novel new chiropractic technology assessment utilizing the first patented computer-assisted device developed to measure spinal compliance and possibly fixation, are certainly promising and warrant further research. More research using different doctors and on larger numbers of subjects, including some with pain, would help determine the generalizability of these findings. Protocols we developed may also be used to conduct research aimed at establishing norms for different populations, including potentially patients with pain, obese individuals, and otherwise normal subjects, to verify and extend our initial findings on these healthy college students. Now that there is initial evidence of good to excellent clinical reliability of the PulStar spinal compliance measure, trials should also be designed and implemented to determine what this phenomenon means in terms of chiropractic patient care and outcomes (ie, trial and construct validity).
The PulStar mechanical adjusting device set to analysis mode appears to have good to excellent reliability when used by either an experienced or a novice (but trained) examiner. In addition, as a measure of spinal resistance to a light pulsed thrust or spinal compliance, reliability was similarly good to excellent between the 2 doctors using the PulStar instrument. Preliminary results indicate spinal compliance in normal subjects is greatest in the lower cervical and lumbar spines and lowest at the upper cervical and upper dorsal levels. This initial study does not address the validity or clinical significance of the measurement method. Further research will be necessary using greater numbers and a wider variety of subjects and more diverse examiners, to verify these findings and fully understand the generalizability of these results.