Mechanically evoked pain, and in particular the

Transcription

1 ORIGINAL ARTICLE Interrater Reliability of Algometry in Measuring Pressure Pain Thresholds in Healthy Humans, Using Multiple Raters Linda S. Chesterton, PhD,* Julius Sim, PhD,* Christine C. Wright, BSc,w and Nadine E. Foster, DPhil* Objective: To evaluate interrater reliability using 5 newly trained observers in the assessment of pressure pain threshold (PPT) using a fixed-angle algometer. Methods: The study design comprised 2 phases. Phase 1: 5 undergraduate physical therapists were trained in algometry at a predefined angle, at a rate of 5 Newtons (N)/s, to the first dorsal interosseous muscle. Each observer then underwent a competency test of the application speed. The aim was to achieve repeated applications at 5 N/s without visual feedback from the algometer. Phase 2: the 5 observers measured PPT of 13 healthy volunteers, at the first dorsal interosseous muscle. The sequence of observer measurements for each participant was randomized. Mean PPT values for each observer were analyzed using repeated measures analysis of variance, intraclass correlation coefficient (ICC 2,1 ), and standard error of measurement, with 95% confidence intervals (CIs). Results: No significant differences between observers mean values were found (P = 0.094), suggesting no bias. The ICC was 0.91 (95% CI 0.82, 0.97). The standard error of measurement value was 6.27 N/cm 2 (95% CI 5.35, 7.59). Differences in PPT measurements of more than N/cm 2 (1.77 kg/cm 2 ) are likely to exceed the magnitude of measurement error, and could be used to indicate true change. This margin of error is, however, somewhat larger than a previously proposed minimum clinically important difference in PPT of N/cm 2 (1.5 kg/cm 2 ). Discussion: This study provides new evidence that trained observers can apply an algometer at a consistent rate and provide highly reliable measures of PPT in healthy humans, when PPT is calculated as the mean of 3 trials. Key Words: reliability, multiple raters, algometry, pressure pain threshold, quantitative sensory testing (Clin J Pain 2007;23: ) Received for publication June 19, 2006; revised July 11, 2007; accepted July 15, From the *Primary Care Musculoskeletal Research Centre, Keele University, Staffordshire; and wschool of Health Sciences, University of Birmingham, Edgbaston, UK. Reprints: Julius Sim, PhD, Primary Care Musculoskeletal Research Centre, Keele University, Keele, Staffordshire ST5 5BG, UK ( j.sim@keele.ac.uk). Copyright r 2007 by Lippincott Williams & Wilkins Mechanically evoked pain, and in particular the pressure pain threshold (PPT), is a popular model for inducing acute, experimental pain. Algometry is a useful technique in determining PPT measures, and has been used widely in both clinical and laboratory settings. 1 8 Pressure algometry is predominantly a manual procedure that requires a perceptual response from the participant or patient. The reliability of PPT data is, therefore, dependent upon not only the application technique of the observer, but also the ability of the patient or participant to provide a consistent verbal indication of the PPT level. Variability seen in PPT measurements could, therefore, be due to the observer (observer error), the participant (participant error), or the measurement error (the difference between each individual rating and its true value), and these are, therefore, always confounded. 9 An important source of measurement error has been identified as the rate of pressure exerted through the algometer. Indeed, maintaining a constant rate has been cited as one of the most difficult aspects of algometry. 10 Another source of error is the angle of application of the algometer to the body part, which, if varied, will also affect the reliability of the verbal report. 11 Several studies using both clinical and healthy populations have used fixed-angle algometry techniques to overcome this source of variation Results from previous PPT reliability studies suggest, in the main, high levels of interrater reliability. 10,17 26 Where poor interrater agreement has been found, 27,28 this has largely been attributed to systematic error from unskilled observers. Despite this seemingly strong body of evidence regarding interrater reliability, there are several issues of concern in relation to these findings. First, only one study has evaluated PPT measurements using more than 2 observers. 26 Yet, on occasions it may be necessary to use a larger number of observers, for example, in a study in which there are a large number of participants, 29 or in a multicenter study. In such cases, it is important to know the likely reliability of measurements taken by multiple observers. Second, many previous studies of the reliability of PPT measurement have adopted Pearson correlation coefficient for statistical analysis. 18,19,21,24,25,27 This statistical approach has been criticized as a basis for establishing interrater reliability because it represents an index of the degree of association, not of the similarity of measures or true agreement. 30 It has been argued that a 760 Clin J Pain Volume 23, Number 9, November/December 2007

2 Clin J Pain Volume 23, Number 9, November/December 2007 Interrater Reliability of Algometry more appropriate way to analyze interrater reliability of interval or ratio data is to use intraclass correlation coefficients (ICCs) The calculation of 95% confidence intervals (CIs), to identify the precision of the estimate, has also been recommended, 34,35 accompanied by the standard error of measurement (SEM), which is a measure of the (im)precision of the measurements themselves. 35 To the best of our knowledge, only 2 previous studies have reported both ICCs and estimates of error within the statistical analysis, 10,17 and from these it is not clear that observers were trained to control the application speed at a fixed level. Finally, fixed algometers, increasingly used in laboratory and some clinical experimental studies to standardize application angles, have not been investigated. The purpose of this study was, therefore, to determine the interrater reliability of PPT measurements in healthy volunteers using 5 inexperienced but specifically trained observers, at a single anatomic site. MATERIALS AND METHODS Design This study was executed in 2 phases. The first phase comprised basic training and assessment of 5 observers in algometry and PPT measurement techniques. The primary purpose of this was to ensure a controlled rate of algometry application. The second phase was an interrater reliability study of the algometry technique using the same 5 trained observers. Before the study, University Ethics Committee approval was obtained. Instrumentation PPT was induced using a pressure algometer (Salter Abbey Weighing Machines Ltd, England) with a flat circular metal probe dressed in several layers of lint and measuring 1 cm in diameter. Force was displayed digitally in increments of 0.1 N/cm 2. The algometer was mounted vertically on a purpose-built stand to enable force to be applied perpendicular to the measurement site (Fig. 1); there was no automated control or restriction of application rate, which was controlled entirely by the observer. Phase 1-training and Testing of Observers Five female volunteer observers were recruited from a group of final year undergraduate physical therapy students (mean age 23, range: 20 to 34). We selected this population to enable the results to be generalized to observers in subsequent experimental studies investigating physical therapy interventions and to reflect the basic level of skill of a population of clinical physical therapists. Before the study, we gave each observer a full written explanation of the purpose of the experiment and the training protocol, as recommended by Goulet et al. 28 After a further verbal briefing, all observers provided written consent to take part. Observers were trained by the same trainer during a single day, in the pressure algometer technique described below, so that repeated applications at the standardized rate of 5 N/s could be FIGURE 1. Fixed-angle pressure algometer. achieved. 16,36 Practice continued until observers were able to repeat applications at the required rate without reference to the digital display of the algometer. The training and protocol incorporated suggestions made by Goulet et al 28 to increase interrater reliability. These included applying the algometer at 90 degrees to a flat skin surface, standardizing and marking the area where pressure was to be applied, covering the application tip of the algometer to avoid sharp pain from the edge of the plunger, 37 and applying pressure evenly and at a constant rate. When training was complete, we tested the observers competence in using the algometer. This test required each observer to make 5 consecutive algometry applications at the prescribed rate of 5 Newtons (N)/s, 15 seconds apart. Each application lasted 10 seconds and was made at the first dorsal interosseous muscle of a volunteer participant, who played no other part in the study than to act as a model for this procedure. Observers were masked to the application rate by covering the digital display of the algometer, such that only the study coordinator could view and record the results. The test was considered successful if all 5 applications were carried out at a rate of 5 N/s (± 1.0 N/s) over a 10-second period, r 2007 Lippincott Williams & Wilkins 761

3 Chesterton et al Clin J Pain Volume 23, Number 9, November/December 2007 because PPT at this anatomic point is usually reported well within this timescale. All observers passed the test at the first attempt and progressed to take part in the reliability study. Phase 2-reliability Study Participants For this study, the sample size was calculated so as to detect a difference in the ICC between r = 0.9 and r = 0.7 at a 2-tailed 5% significance level and with 80% power, assuming that 5 observers rated each participant. 38,39 We chose r = 0.7 for the null value on the basis of its being the midpoint of the interval proposed by Shrout 40 to constitute moderate reliability (0.61 to 0.80). From this calculation, the minimum number of participants required was 13 (Table 1). Volunteers (n = 13; mean age 22) were recruited from the student population; they were all white and all but one was female (Table 1). Although age and sex are known to influence PPT between individuals, 4,5,41 we found no literature to suggest that these population characteristics influence the reliability of an individual s repeated verbal report of PPT. Before taking part, volunteers were screened for current and previous pain conditions, medication, peripheral neuropathy, neurologic pathologies, or injury to either hand. No participants were excluded. The experimental procedure was explained and written informed consent was obtained from each participant. PPT Measurement Procedure The reliability study was conducted in one laboratory at a mean temperature of 21.51C (SD 0.73). Participants were seated in a comfortable upright position. A single study coordinator marked the location of the PPT measurement site, at the midpoint of the muscle belly of the first dorsal interosseous muscle of the dominant hand. The hand was then placed beneath the algometer such that the algometer probe would be lowered perpendicular to the marked spot. The algometer setup was the same as used previously in training the observers. Two identical calibrated algometers were available. TABLE 1. Summary of Reliability Study Protocol Components No. participants (n) 13 Male: female 1:12 Mean (range) age 22 (20 29) y No. observers 5 Male: female 0:5 Mean (range) age 23 (20 34) y No. measures taken by each observer from a single 3 participant Interval between measurements for a single observer 15 s Interval between each observer per participant 10 min Total no. measurements per participant 15 Total no. measurements per observer 39 We explained and demonstrated the method of algometer application to each participant. Each participant then underwent 2 practice PPT measures using the nondominant hand and they were coached in differentiating their report of tactile and painful stimuli. PPT was taken as the amount of pressure required to elicit a sensation of pain distinct from pressure or discomfort. 42 We asked participants to say stop as soon as a discernible sensation of pain was felt; at this point the algometer pressure was immediately released and the plunger was retracted by the observer. The algometer probe was lowered at a constant rate of approximately 5 N/s until PPT was reached, as indicated by a participant s verbal report. Each observer took 3 PPT readings, from each participant, 15 seconds apart. This replicates recommended clinical practice. 10 Participants were given 10 minutes rest between measurements taken by each observer, which has previously been shown to be sufficient. 13 Neither the participants nor the observers were able to see the algometer display and all readings were recorded by the study coordinator. Thus, observers and participants were unaware of their own PPT recordings and those of other observers or participants. To avoid order effects, each observer tested participants in a computer-generated random sequence. All readings were taken on the same day. Data Analysis The mean of each observer s 3 PPT measurements per participant was calculated, and used as the data for the main analysis. For purposes of comparison, each observer s first measurement was also analyzed. Graphical analysis of each observer s ratings and repeated measures analysis of variance (ANOVA) were used to identify systematic bias (against an assumed null hypothesis of no difference between the 5 observers). The assumptions of this ANOVA model were checked before the analysis. Although the residuals showed some positive skew, this was considered to be within acceptable limits, given that the skew was largely unidirectional. 43 The ICC (model 2,1) was computed for interrater reliability, because the analysis of interest was to determine whether the observers could be interchangeable. 44 This ICC is based on a 2-way random effects model with rater variance included in the ICC denominator, so that differences in level across raters count as error. To express the precision of the reliability coefficient, and identify the limits likely to contain the true reliability in the population, 95% CIs were computed for the ICC values. The SEM provides a quantitative indicator of the measurement error in the units of the original measurements (which in this instance are the mean of each observer s 3 consecutive measurements per participant). The SEM is the SD p of the measurement errors, and is calculated as s ffiffiffiffiffiffiffiffiffiffi 1 r, where s is the SD of the observed ratings and r is the reliability coefficient (here, the ICC). A small value for the SEM indicates low measurement error, and thus high reliability. 45 We can be 95% confident that an interval of 1.96 SEM either side 762 r 2007 Lippincott Williams & Wilkins

4 Clin J Pain Volume 23, Number 9, November/December 2007 Interrater Reliability of Algometry of an individual s observed score contains that individual s true score (assuming that measurements are not systematically biased). The SEM can also be used to identify the minimum difference in the measure that must be exceeded to demonstrate true change, as opposed to random measurement fluctuation. Thus, for example, we can be p 95% ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi confident that a difference of greater than 1:96 2SEM 2 between 2 measurements represents true change. 35 We also calculated CIs for the SEM, via the formula for the CI of a variance. 46 Spearman rho (r S ) was used to assess the correlation between observers mean ratings and the order in which they were taken. Data were analyzed using SPSS version 14. We took Pr0.05 (2-tailed) as the cut-off for statistical significance. RESULTS The data for the reliability analysis (ie, mean values of the 3 measurements taken from each participant by each of the 5 observers) are given in Table 2. Figure 2 provides a plot of these values. This plot suggests that agreement between observers is generally high, being slightly lower for high PPT values (eg, participants 1, 6, and 10). The ICC was very high (ICC = 0.91; 95% CI 0.82, 0.97); this value is significantly larger than the null value of 0.7 (P<0.0005), and the narrow CI indicates the high precision of the estimate. The SEM was calculated as 6.27 N/cm 2 (95% CI 5.35, 7.59). On the basis of a SEM of this magnitude, we can be 95% confident that (1) an interval of N/cm 2 (1.25 kg/cm 2 ) either side of the observed PPT measurement contains the true value, and (2) a difference between 2 observers PPT measurements of more than N/cm 2 (1.77 kg/cm 2 ) represents a true difference. The ICC of 0.91 for the mean of the 3 measurements for each observer was somewhat higher than that obtained when only the observers first measurement was analyzed; for the latter analysis, ICC = 0.90 (95% CI 0.79, 0.96), and P = for the statistical test against the null value of 0.7. The SEM calculated on the observers first measurement was 6.66 N/cm 2 (95% CI 5.68, 8.06), such that we can be 95% confident that a difference between 2 observers first PPT measurements of more than N/cm 2 (1.88 kg/cm 2 ) represents a true difference. The bottom row in Table 2 shows the means for each observer across all 13 participants. Repeated measures ANOVA showed no statistically significant difference across these values (F 4,48 = 1.000, P = 0.417), which suggests that there was no bias in ratings across observers. Additionally, there was a nonsignificant correlation between the means of the 15 ratings and the sequence in which they were performed (r S = 0.343, P = 0.211), which suggests that there was no systematic change in the participants PPT values over time. DISCUSSION The value of most outcome measures and assessment techniques is in part dependent upon high intrarater and interrater reliability. The purpose of this study was to establish the interrater reliability of PPT measurement made using a pressure algometer, and with multiple observers trained in applications at a consistent rate. The results showed that the training of observers in the algometry application technique was successful, with all 5 observers passing the competency test at the first attempt and within a single day. This addresses the observer error introduced by variable rates of application and is, therefore, an important feature of the study. Most previous studies have not reported whether, or how, such testing of observers performance has taken place. The interrater ICC for the mean measurements was excellent at 0.91, with a narrow CI. These results are consistent with those of Antonaci et al, 17 Delaney and McKee, 20 and Nussbaum and Downes 10 ; notwithstanding the use of different study protocols, these authors also showed high interrater reliability (with ICCs between 0.75 and 0.88), in TABLE 2. PPT Measurements in N/cm 2 for Each Participant by Observer (Data in the Cells are the Mean of 3 Measurements Per Participant by Each Observer) Observer Participant Mean (SD) (9.6) (0.7) (5.3) (2.0) (4.4) (9.3) (6.9) (6.9) (10.3) (7.0) (5.4) (3.2) (4.8) Mean (SD) 37.7 (21.8) 34.8 (22.7) 39.7 (21.5) 38.1 (17.9) 37.0 (24.6) r 2007 Lippincott Williams & Wilkins 763

5 Chesterton et al Clin J Pain Volume 23, Number 9, November/December 2007 FIGURE 2. Mean score of 3 readings from n = 13 participants for each of 5 observers (PPT in N/cm 2 ). healthy humans. The current study, however, used a larger number of observers and reports the statistical conclusions with a stated power. Not unexpectedly, reliability was somewhat greater for the observers mean values than for their first ratings, as has been reported previously. 10 It is advisable, therefore, that mean rather than single measurements should be used in multiple-observer studies of PPT. From the SEM of 6.27 N/cm 2 (0.64 kg/cm 2 ) obtained in this study, it is highly likely that measurement error will be no greater than N/cm 2 (1.25 kg/cm 2 ), and that a difference between 2 observers measurements of PPT of more than N/cm 2 (1.77 kg/cm 2 ) will indicate a true difference, when using multiple trained raters. This value for the interobserver SEM is higher than the figure of 5.74 N/cm 2 (0.59 kg/cm 2 ) calculable for the data reported by Antonaci et al, 17 and that of 5.06 N/cm 2 (0.52 kg/cm 2 ) calculable for the data reported by Nussbaum and Downes 10 despite their reporting lower ICCs. The SDs of the PPT measurements were smaller in these studies than in our study, which is likely to have constrained the values they obtained for the ICC, but would not have affected the SEM. Fischer 47 suggested meaningful clinical differences in PPT measures would range from 1.5 to 2 kg/cm 2 (14.71 to N/cm 2 ). This suggestion was based on the comparison of PPTs in healthy tissue with that of pathologic (tender) tissue on the corresponding and contralateral anatomic site in 50 volunteers (tender spots correlated as hot spots when viewed thermographically). The magnitude of measurement error of N/cm 2 (1.77 kg/cm 2 ) in our study, using mean measurements, is therefore larger than the lower boundary of Fischer s estimate of a minimum clinically important difference. Therefore, despite the high ICC, we cannot be highly confident that a clinically important change will not be masked by measurement error, when using multiple observers and mean measurements. Further, findings from the current study show that participants are reliable in making a consistent verbal report of their own PPT with 15 repeated measures over a 1-hour period, which is consistent with findings of Chesterton et al 13 and Jensen et al, 48 but represents new information because ratings are from multiple observers. There was also evidence of heteroscedasticity (where larger differences between raters were recorded for participants showing higher PPT thresholds). This finding is of interest, and possible explanations are that some examiners were unable to maintain pressure of application at higher levels of PPT, or that participants were unable to discern reliably the threshold point at high levels of pressure and tissue distention. However, the change in angle (or possible movement/wobble) of the probe that may be produced with high pressure manual algometry clearly plays no part. The role of structured training and competency testing may also have played a part in the observed differences with previous studies, but this is difficult to quantify because Nussbaum and Downes 10 provided training but did not undertake a competency test and Antonaci et al 17 did not document any training. Hence, the consistency of application rate cannot be commented upon. The comparability in terms of developed skill in the technique is, therefore, difficult to determine. Within the data from the present study, there is a small trend toward lower PPT ratings on second assessments compared with the first and third by each observer. Variations in individual readings have been noted by other authors, 10,22,41 and mean data are, therefore, likely to give better estimates. Order effects for observers were counteracted by the random order in which observers assessed participants. The findings of this study relate to PPT measurements taken at the first dorsal interosseus muscle of the hand, using a fixed angle (via a stand). They may not necessarily be generalizable to other measurement sites, or to situations in which a purely manual method of measurement is used. With reference to the statistical analysis used in this study, some authors maintain that the ICC statistic may produce misleading results through underestimating or overestimating reliability. 30,49 Heterogeneity in individual levels of PPT was observed between participants, and this is a well documented phenomenon in PPT measurement. 4,5,48 In common with the Pearson correlation coefficient, a wide versus a narrow range of PPT measures (ie, heterogeneous population) will lead to relatively higher ICCs (though researchers should not be tempted to select a sample that is more heterogeneous than the population of interest for the sake of larger ICC values). The large interindividual differences demonstrated in the present study may, therefore, have influenced the magnitude of the ICCs obtained. For this reason, the 764 r 2007 Lippincott Williams & Wilkins

6 Clin J Pain Volume 23, Number 9, November/December 2007 Interrater Reliability of Algometry ICCs should be interpreted alongside the SEMs, 30 as the latter are not affected by the range of the measurements, and thus generalizable across samples with differing degrees of variability. An approach to analyzing agreement through bias and limits of agreement has been proposed as the method of choice, 50 and was used by Nussbaum and Downes. 10 This method, which relies on graphic display and analysis of paired differences, is impractical in the case of multiple observers; there are, for example, 10 pairwise analyses possible between the observers in this study. We feel the approach to analysis in the current study is, therefore, a useful combination of methods. The familiar skewness of PPT measures can, however, present difficulties for reliability studies. Our study provides new evidence that multiple, trained observers using a fixed-angle algometer, demonstrate high interrater reliability in the measurement of PPT in healthy humans. The goal of the training was to ensure a consistent and constant speed of application, which can be achieved through basic instruction and practice. This level of competency is easily achieved with this algometer and does not require specialist training; in this respect our findings are generalizable. Also, as our observers were undergraduate students, they can be assumed to represent a basic level of skill in the measurement of PPT; therefore, our findings may well underestimate the reliability of measurements taken by more experienced clinicians. The way in which this algometer is applied should not differ between patients and healthy humans, and it is reasonable to assume that our findings would apply to both healthy and clinical populations in the research situation. It must be acknowledged, however, that the algometer stand used in this study is limited in its application to areas that can be accommodated under the algometer probe, such as the hand or forearm. This apparatus could not be used to measure PPT in regions such as the neck or back, and our findings should not, therefore, be extrapolated to these measurement situations. We conclude that multiple trained examiners and this algometry technique have the potential to achieve highly reliable measurements of PPT in multiobserver trials. In this study, changes of greater than N/cm 2 (1.77 kg/cm 2 ) can be confidently regarded as representing true change. Notwithstanding the high value of the ICC, the measurement error estimated in this study could mask a clinically important difference. This highlights the importance of calculating measures of both reliability and measurement error. REFERENCES 1. Fischer AA. Pressure threshold measurement for diagnosis of myofascial pain and evaluation of treatment results. Clin J Pain. 1987;2: Isselee H, De Laat A, Bogaerts K, et al. Short-term reproducibility of pressure pain thresholds in masticatory muscles measured with a new algometer. J Orofac Pain. 1998;12: Isselee H, De Laat A, Bogaerts K, et al. Long-term fluctuations of pressure pain thresholds in healthy men, normally menstruating women and oral contraceptive users. Eur J Pain. 2001;5: Jensen R, Rasmussen BK, Pedersen B, et al. Cephalic muscle tenderness and pressure pain threshold in a general population. Pain. 1992;48: Jensen R, Rasmussen BK, Pedersen B, et al. Muscle tenderness and pressure pain thresholds in headache. A population study. Pain. 1993;52: Kosek E, Ekholm J, Hansson P. Pressure pain thresholds in different tissues in one body region. The influence of skin sensitivity in pressure algometry. Scand J Rehabil Med. 1999;31: Kosek E, Ordeberg G. Abnormalities of somatosensory perception in patients with painful osteoarthritis normalize following successful treatment. Eur J Pain. 2000;4: Walsh DM, Lowe AS, McCormack K, et al. Transcutaneous electrical nerve stimulation: effect on peripheral nerve conduction, mechanical pain threshold, and tactile threshold in humans. Arch Phys Med Rehabil. 1998;79: Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 3rd ed. Oxford: Oxford University Press; Nussbaum EL, Downes L. Reliability of clinical pressure-pain algometric measurements obtained on consecutive days. Phys Ther. 1998;78: Greenspan JD, McGillis SL. Thresholds for the perception of pressure, sharpness, and mechanically evoked cutaneous pain: effects of laterality and repeated testing. Somatosens Mot Res. 1994; 11: Chesterton L, Foster N, Wright C, et al. Effects of TENS frequency, intensity and stimulation site parameter manipulation on pressure pain thresholds in healthy human subjects. Pain. 2003;106: Chesterton LS, Barlas P, Foster NE, et al. Gender differences in pressure pain threshold in healthy humans. Pain. 2003;101: MacDonald W, Atkins RM. An algometer for the automated measurement of pain threshold. Br J Rheumatol. 1990;29: Vatine JJ, Shapira SC, Magora F, et al. Electronic pressure algometry of deep pain in healthy volunteers. Arch Phys Med Rehabil. 1993;74: Walsh DM, Liggett C, Baxter D, et al. A double-blind investigation of the hypoalgesic effects of transcutaneous electrical nerve stimulation upon experimentally induced ischaemic pain. Pain. 1995;61: Antonaci F, Sand T, Lucas GA. Pressure algometry in healthy subjects: inter-examiner variability. Scand J Rehabil Med. 1998; 30: Buchanan HM, Midgley JA. Evaluation of pain threshold using a simple pressure algometer. Clin Rheumatol. 1987;6: Chung SC, Kim JH, Kim HS. Reliability and validity of the pressure pain thresholds (PPT) in the TMJ capsules by electronic algometer. Cranio. 1993;11: Delaney GA, McKee AC. Inter- and intra-rater reliability of the pressure threshold meter in measurement of myofascial trigger point sensitivity. Am J Phys Med Rehabil. 1993;72: Hogeweg JA, Langereis MJ, Bernards AT, et al. Algometry. Measuring pain threshold, method and characteristics in healthy subjects. Scand J Rehabil Med. 1992;24: Ohrbach R, Gale EN. Pressure pain thresholds in normal muscles: reliability, measurement effects and topographic differences. Pain. 1989;37: Persson AL, Brogårdh C, Sjölund B. Tender or not tender: testretest repeatability of pressure pain thresholds in the trapezius and deltoid muscles of healthy women. J Rehabil Med. 2004;36: Reeves JL, Jaeger B, Graff-Radford SB. Reliability of the pressure algometer as a measure of myofascial trigger point sensitivity. Pain. 1986;24: Takala EP. Pressure pain threshold on upper trapezius and levator scapulae muscles. Repeatability and relation to subjective symptoms in a working population. Scand J Rehabil Med. 1990;22: r 2007 Lippincott Williams & Wilkins 765

7 Chesterton et al Clin J Pain Volume 23, Number 9, November/December Tunks E, McCain GA, Hart LE, et al. The reliability of examination for tenderness in patients with myofascial pain, chronic fibromyalgia and controls. J Rheumatol. 1995;22: Chung SC, Um BY, Kim HS. Evaluation of pressure pain threshold in head and neck muscles by electronic algometer: intrarater and interrater reliability. Cranio. 1992;10: Goulet JP, Clark GT, Flack VF. Reproducibility of examiner performance for muscle and joint palpation in the temporomandibular system following training and calibration. Commun Dent Oral Epidemiol. 1993;21: Lacey RJ, Lewis M, Jordan K, et al Interrater reliability of scoring of pain drawings in a self-report health survey. Spine. 2005;30:E455 E Haas M. The reliability of reliability. J Manipulative Physiol Ther. 1991;14: Fleiss JL. The Design and Analysis of Clinical Experiments. New York: Wiley; Haas M. Statistical methodology for reliability studies. J Manipulative Physiol Ther. 1991;14: Rankin G, Stokes M. Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clin Rehabil. 1998;12: Batterham AM, George KP. Reliability in evidence based clinical practice: a primer for allied health professionals. Phys Ther Sport. 2000;1: Eliasziw M, Young SL, Woodbury MG, et al. Statistical methodology for the concurrent assessment of interrater and intrarater reliability: using goniometric measurements as an example. Phys Ther. 1994;74: McDowell BC, McCormack K, Walsh DM, et al. Comparative analgesic effects of H-wave therapy and transcutaneous electrical nerve stimulation on pain threshold in humans. Arch Phys Med Rehabil. 1999;80: Greenspan JD, McGillis SL. Stimulus features relevant to the perception of sharpness and mechanically evoked cutaneous pain. Somatosens Mot Res. 1991;8: Donner A, Eliasziw M. Sample size requirements for reliability studies. Stat Med. 1987;6: Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Stat Med. 1998;17: Shrout PE. Measurement reliability and agreement in psychiatry. Stat Methods Med Res. 1998;7: Brennum J, Kjeldsen M, Jensen K, et al. Measurements of human pressure-pain thresholds on fingers and toes. Pain. 1989;38: Fischer AA. Pressure algometry over normal muscles. Standard values, validity and reproducibility of pressure threshold. Pain. 1987;30: Kirk RE. Experimental Design: Procedures for the Behavioral Sciences. 3rd ed. Pacific Grove: Brooks/Cole; Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86: Denegar CR, Ball DW. Assessing reliability and precision of measurement: an introduction to intraclass correlation and standard error of measurement. J Sports Rehabil. 1993;2: Miller I, Miller M. John E Freund s Mathematical Statistics. 6th ed. Upper Saddle River: Prentice Hall; Fischer AA. Application of pressure algometry in manual medicine. J Man Med. 1990;5: Jensen K, Andersen HO, Olesen J, et al. Pressure-pain threshold in human temporal region. Evaluation of a new pressure algometer. Pain. 1986;25: Bland JM, Altman DG. A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement. Comput Biol Med. 1990;20: Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1: r 2007 Lippincott Williams & Wilkins