The serious but commonplace problem of pain among. Pain in the Elderly. Validity of Facial Expression Components of Observational Measures

Transcription

1 ORIGINAL ARTICLE Validity of Facial Expression Components of Observational Measures Esther Sheu, BA,* Judith Versloot, PhD,*w Rami Nader, PhD,* Deborah Kerr, PhD,*z and Kenneth D. Craig, PhD* Objectives: Assessing pain in elderly persons, who have diminished capacity to communicate verbally, requires use of observational scales that focus upon nonverbal behavior. Facial expression has been recognized as providing the most specific and sensitive nonverbal cues for pain. This study examined the validity of facial expression components of 6 widely used pain assessment scales developed for elders with dementia. Descriptions of the facial expression of pain vary widely on these scales. Methods: The detailed, anatomically based, objectively coded, and validated Facial Action Coding System was used as a criterion index to provide a definitive description of the facial expression of pain. Thirty elderly inpatients with clinically significant pain in the back or hip, the majority of whom had cognitive impairments, provided videotaped reactions to physical activities. Participants facial expressions were videotaped during 4 randomly ordered physical activities and coded by a qualified Facial Action Coding System coder. Three 6-second clips indicative of mild, moderate, and severe pain intensities were selected for study for each participant. The 90 clips were coded by 5 raters using the facial expression components of the following observational scales: Doloplus-2, Mahoney, Abbey, pain assessment checklist for seniors with limited ability to communicate, noncommunicative patient s Pain Assessment Instrument, and Pain Assessment in Advanced Dementia. Results: Overall, scales that provided specific descriptions using the empirically displayed facial actions associated with pain yielded greater sensitivity, interjudge reliability, and validity as indices of pain. Discussion: Facial expression items on observational scales for assessing pain in the elderly benefit from adherence to empirically derived descriptions. Those using the scales should receive specific direction concerning cues to be assessed. Observational scales that provide descriptors that correspond to how people actually display facial expressions of pain perform better at differentiating intensities of pain. Key Words: elderly, observational scales, facial expression, pain, assessment (Clin J Pain 2011;27: ) The serious but commonplace problem of pain among older adults deserves careful development of pain assessment tools. People aged 60 and above are twice as Received for publication July 30, 2010; revised December 10, 2010; accepted January 11, From the *Department of Psychology, University of British Columbia; zvancouver Coastal Health; and wthe University of Toronto, Toronto, Canada. Supported by grants from the Social Sciences and Humanities Research Council of Canada and Vancouver Coastal Health, Ottawa, Canada. The authors declare no conflict of interest. Reprints: Kenneth D. Craig, PhD, Department of Psychology, University of British Columbia, Vancouver, British Columbia V6T 1Z4 Canada ( kcraig@psych.ubc.ca). Copyright r 2011 by Lippincott Williams & Wilkins likely to experience pain as younger people, 1 with almost half of the elderly population have some form of chronic pain. 2 A significant number of the elderly reside in nursing home settings. 1,2 Fully 67.4% of all nursing home patients are reported to have dementia with verbal limitations compounding the challenges of pain assessment. 3 Substantial evidence shows that seniors with dementia are significantly undertreated for their pain compared with those without cognitive impairments, 1,4,5 although they do not differ in the prevalence of acute or chronic health conditions associated with pain. 2,6,7 Core symptoms of dementia are gradual deterioration in memory and other cognitive deficits in language and abstract thinking. 1,3,7 Impairment in communicative ability, self-report in particular, has been recognized as the primary reason that these elders fail to receive adequate pain management. 1,3,4,7 It becomes imperative that nonverbal capacities for communication are effectively assessed using observational scales. 2 In the domain of spontaneous, nonverbal, behavioral expressions, facial expressions have been recognized as providing the most sensitive and specific information. 5,8 10 The American Geriatrics Society describes facial expression as 1 of the 6 main types of observable behavioral indicators of persistent pain in the elderly. 7 Facial grimaces expressed during painful experiences display a discrete and stereotypical structure that correlates with observer global judgments of pain and verbal reports of pain intensity among both healthy research participants and those burdened by injuries or diseases across the life span, 2,8,9 although situational demand can lead to discordance among measures. 11,12 As facial expressions are readily accessible and essentially available continuously, they typically are included in nonverbal assessment instruments. 13 Research on the facial expression of pain was spurred by development of the Facial Action Coding System (FACS). 8,14 The FACS is an objective, anatomically based measure that comprehensively describes facial activity in detail. The 44 facial action units (AUs) are based on discrete movements of specific muscles or groups of muscles of the face and describe any facial display without inference of subjective psychological states. 9 The measure consistently distinguishes a core set of facial actions that occur in response to different types of clinical pain (eg, acute pain, exacerbations of chronic pain) and experimentally induced noxious stimulation (eg, heat or pressure stimulation) in both healthy participants and patients and is sensitive to intensity of painful stimulation. 8 The FACS has been shown to provide valid assessment of pain in both healthy and demented elderly research participants who are verbally compromised. 2,13,15 However, the FACS has limited clinical use, as it requires videotaped records, extensive coder training, and is time consuming to score. 8 Consequenly, less detailed and demanding observational tools have become the preferred Clin J Pain Volume 27, Number 7, September

2 Sheu et al Clin J Pain Volume 27, Number 7, September 2011 choice for pain assessment in elderly persons who are verbally compromised. 2 These nonverbal measures typically are multidimensional and focus on a wide range of behavioral activities presumed to be associated with pain, including changes in behavior and functioning, such as sleep, appetite, physical activity, interpersonal interactions, verbalizations or vocalizations, facial or body language, and mental status. 1,16 However, evaluation of the validity of items putatively indicative of pain is rarely undertaken. Careful inspection of facial expression items on many scales indicates that the accounts of facial activity allegedly indicative of pain vary substantially from one tool to another and often do not accord with empirical descriptions. 8 For example, some scales ask observers to attend to negative emotional states, such as a tense, frowning, or frightened face. 17 These expressions often occur independently of pain, are not invariably features of pain expression, and fear has its own stereotypic structure that differentiates it from the facial expression of pain. 18 Other scales ask for identification of facial activity that is inconsistent with pain, such as an unusual, permanent blank look or staring face. 19 It is interesting that other scales use precisely those characteristics as representing an absence of pain. 20 Furthermore, many scales use vague and often misleading terms such as grimace, frown, and restlessness to indicate pain. 8 Inconsistent and inappropriate descriptions of pain likely lead to failures in distinguishing pain from no pain or to inferring pain when it is absent. Vague descriptions may lead to poor inter-rater judgment reliability. 9 The purpose of this study was to examine and compare the psychometric properties of facial activity components of a selection of behavioral observational scales developed for pain assessment in the elderly with cognitive impairments to those of the FACS. Specifically, how valid and reliable were facial expression items on these scales in detecting and assessing pain and its intensity, in comparison to each other and to the FACS? Scales using explicit descriptors corresponding to empirically based accounts of the facial expression of pain, such as criteria for judgment, were expected to (1) yield higher levels of intercoder reliability, (2) perform better in differentiating intensities of pain, and (3) correlate more strongly with scores from the FACS. In addition, reliability was expected to be better when there was a greater likelihood of vigorous facial activity (ie, more severe pain intensity). The facial components of 6 widely used and recommended nonverbal observational behavioral pain tools were the focus of this study. They were the Doloplus-2 scale, 19 the Mahoney Pain Scale (MPS), 20 the Abbey Pain Scale, 17 the Noncommunicative Patient s Pain Assessment instrument (NOPAIN), 21 the Pain Assessment Checklist for Seniors with Limited Ability to Communicate (PACSLAC), 22 and the Pain Assessment in Advanced Dementia (PAINAD). 23 These scales were selected because they are actively used clinically, they have been described as having desirable psychometric properties, and facial expression figures prominently in use of the scale. METHOD Patient Sample The study was approved by the Behavioural Research Ethics Board at the University of British Columbia. Recruitment of patients was as follows: Hospital staff informed patients /or their families about the study. Those who expressed an interest were briefed on particulars of the study. Patients either provided informed consent, or, when attending staff considered the patient to be unable to provide consent, a family member gave consent for the patient to participate. All patients for whom there was an expression of interest in participating were included. Facial reactions of pain during physical activity were videotaped for 60 elderly inpatients with clinically significant pain in the hip or back. The patients were 49 females and 11 males aged 65 years or older (ages ranged from 65 to 96 y). The average age was 84 years. Individuals who received a diagnosis of a serious psychiatric condition, except for dementia, were excluded from participation. Individuals in advanced stages of Parkinson disease were also excluded, as the disease influences facial muscular movements. All patients showed a capacity to comprehend and communicate in English, so as to cooperate with instructions. The study required physical movement and participants had to be able to ambulate independently or with the assistance of another person. Patients had been admitted to hospital due to pain in the hip or back. Seventy-three percent of the group had been admitted due to hip fracture, 10% had a compression fracture of the spine, 3% suffered from hip pain, 7% from back pain, and another 7% from other pain problems. Moreover, 72% had undergone hip surgery before the assessment. Only 1 patient was not on any pain medication. Of the group, about 53% were taking acetaminophen, 8% were taking an opioid, 35% were taking both acetaminophen and an opioid, and 2% were taking another type of pain medication. The patients cognitive status was assessed using the Cognistat. 23 The Cognistat is a brief bedside screening tool for detecting and characterizing cognitive dysfunction; it evaluates 10 cognitive domains to establish the person s level of cognitive status. Results from the Cognistat showed that approximately 32% of the patients fell within the no impairment range, 22% were mildly impaired, 25% moderately impaired, and 22% displayed severe impairment. Video Recordings of Elderly Patients and Facial Action Coding Each of the 60 patients was videotaped, specifically targeting facial expression, when engaged in physical activities during a 30-minute pain and cognitive assessment session at their bedside. During the recording session, patients were asked to voluntarily perform 4 routine activities for a period of 1 minute each: lying, sitting, standing, and walking. The order of the activities was randomized for each patient. A physical therapy aide was available at all times to provide physical assistance if needed, and patients used crutches, walkers, or canes if desired. Facial expression of the patients was coded using the FACS by a trained and qualified FACS coder. Using slowmotion and stop-action video-editing equipment, the FACS coder applied explicit criteria to identify 44 separate AUs. The intensity of each AU identified was coded on a 6-point intensity scale, ranging from no action through maximum action. Exceptions occurred when AUs can be only present or absent and do not show variation in intensity (eg, eyes closed). In this case, the intensity of the AU was coded dichotomously, with no reaction coded zero intensity and presence coded 3 on the 6-point intensity scale. Ten second segments were coded for the following: 2 randomly selected r 2011 Lippincott Williams & Wilkins

3 Clin J Pain Volume 27, Number 7, September second segments during each of the 4-videotaped activities, and one 10-second segment during the transitions between each activity (eg, lying/sitting, sitting/standing, standing/sitting, sitting/lying). Thus, 12 segments of facial activity were coded for each patient. Twenty percent of the segments were coded by a second trained FACS coder to determine inter-rater reliability. Inter-rater coding reliability was calculated by assessing the proportion of agreement on actions recorded by 2 coders relative to the total number of actions coded as occurring to each coder. 14 Inter-rater reliability was very high at 0.94 for frequency of the AUs and similarly good at 0.84 for intensity. Selection of Mild, Moderate, and Severe Pain Clips As the FACS is the most sensitive and comprehensive tool for the characterization of the facial expression of pain presently available, the coded FACS data were used to select segments of mild, moderate, and severe pain intensities for each patient. Using AU intensity ratings, 3 video clips of mild, moderate, and severe intensities were selected from the 12 that were available for each patient, and subsequently used for coding with the 6 tools. As the first 6 seconds captured the onset, peak, and decay of the facial reaction, this duration of time was used. Pain intensity scores were the sum of the intensity scores for the 4 facial AUs established as most typical of pain expression : AU 4 (brow lower), AU 6 (cheek raiser), AU 10 (upper-lip raiser), and AU 43 (eyes closed) occurring during the 6-second segment. 9 Given the substantial individual differences in pain expression across patients, the clips representing mild, moderate, and high pain were selected for each patient. Mild and severe clips were the 6-second segments that scored the lowest and highest, respectively. The clip corresponding most closely to the mean score of the 12 clips, excluding those that were scored as zero, was selected to represent moderate pain. The order of presentation of the clips was randomized using a table of random numbers. Judgment Study Sample Selection For the purposes of this study, 30 elderly patients were randomly selected using a table of random numbers from the original group of 60 recordings available, as a sample of 30 provided adequate power for judges ratings. The a priori sample size calculation was based on an expected medium effect size of 0.5, a of 0.05 and 1-b (power) of A sample size of 27 participants was determined to be sufficient. 27 As FACS data were available for the 30 participants, a post hoc analysis was performed based on a paired sample t-test using the means and standard deviations of the FACS scores at the mild and moderate intensity levels. This calculation showed a large effect size of 1.8. Therefore a sample size of 30 participants who provided facial expressions of pain was confirmed to provide sufficient power for this study. Measures The facial expression components from the Doloplus-2, MPS, Abbey, Noncommunicative Patient s Pain Assessment instrument (NOPPAIN), Pain Assessment Checklist for Seniors with Limited Ability to Communicate (PACSLAC), and Pain Assessment in Advanced Dementia (PAINAD) were selectively extracted for use in the coding of pain. Doloplus-2 Scale The Doloplus-2 is a multidimensional behavioral pain assessment instrument developed for use with nonverbal elder persons. It was originally developed in French and later adapted for use in English. 28 The Doloplus-2 evaluates the presence of 10 pain-related behaviors. They are verbal complaints, facial expressions, protective body postures, protection of sore areas, disturbed sleep, behavioral problems, functional impairments in mobility and washing and dressing, and changes in communication and social life. 28 The available literature shows mixed findings for the scale s inter-rater reliability and convergent validity, and there is limited support for its test-retest reliability. 1,7,28 It has been suggested that there is insufficient research addressing the psychometric properties of the English adaptation. 7 Facial expression on the Doloplus-2 is scored on 4 levels from 0 to 3, wherein 0 represents usual expression, 1 indicates expression showing pain when approached, 2 specifies expression showing pain even without being approached, and a score of 3 represents a permanent and unusually blank look (voiceless, staring, looking blank). Coders judge correspondence of these descriptors to facial activity expressed. Note that a score of 2 was not applicable in this study, as the elderly patients were all videotaped in proximity to a physical therapy aide, made available for physical assistance when needed, and therefore, all patients were being approached by someone. Mahoney Pain Scale The MPS was developed to assess the presence and severity of pain in people in advanced stages of dementia. 20 Eight behavioral items are rated on a scale from 0 to 3, with higher ratings indicating higher pain intensity, including facial expression, breathing, vocalization, and body language. Developers of the tool reported adequate inter-rater reliability and internal consistency between scale items, and indicated there is support for its construct and concurrent validity. However, support for the MPS is limited as evidence from only 1 study could be located. The MPS associates unique facial descriptions with different levels of pain. A score of 0 is given if a pleasant, relaxed, or blank expression is identified. A score of 1 corresponds to a sad (eyes down, frowning, teary) or anxious (eyes wide and alarmed) look, 2 is given if an occasional or slight grimace (eyes narrowed, brow furrowed, lips parted) is identified, and 3 by a frequent or definite grimace (eyes narrowed or closed, brow furrowed, lips clearly parted). Abbey Pain Scale The APS was developed to measure pain intensity in people with end-stage dementia. 1 It addresses a total of 6 items including vocalization, facial expression, changes in body language, functional behavioral change, physiological change, and physical changes. 7 Each item is scored on a 4-point intensity scale from 0, representing absence of pain, to 3 representing severe pain, with these summed to provide a score interpreted as a total pain score. 7 Available research indicates adequate internal consistency for the scale, but moderate construct validity and relatively low inter-rater reliability. There is little research at present with regard to the tool s test-retest reliability. 1 Hence, the tool requires further psychometric assessment. The facial expression r 2011 Lippincott Williams & Wilkins 595

4 Sheu et al Clin J Pain Volume 27, Number 7, September 2011 component of the APS directs the coder to look for the presence of a tense, frowning, grimacing, and/or frightened facial expression, to be rated using the 0 to 3 intensity scale. NOPPAIN Scale The rater initially uses a checklist of daily care activities to indicate whether pain has been observed. Then intensity of pain is rated on 6 categories of pain behavior during these activities on a 6-point Likert-type scale, ranging from lowest to highest possible intensity. 3 The 6 pain responses include pain words, pain faces, bracing, noises, rubbing, and restlessness. No criteria are provided to establish the intensity of the pain behaviors. 7 There is preliminary evidence for its potential in clinical use, however, as the scale has been described as showing reasonably good inter-rater reliability and construct validity in naturalistic settings. 1,3,7 The NOPPAIN facial expression component requires the coder to establish the presence or absence of a pain face, as indicated by grimaces, winces, and furrowed brow. If these facial features were present, the coder then proceeds to making a judgment on the intensity of the pain face on a scale of 0 to 5, with 0 representing the lowest possible intensity and 5 the highest possible intensity. PACSLAC Scale The PACSLAC is a comprehensive checklist of observable pain behaviors. Subscales examine facial expressions, body movements, social activity, physiological activity, sleeping changes, and vocal behaviors. 22 Sixty items are scored as present or absent. Scores from the subscales are summed to characterize painful, calm, or distressing events. 7 The PACSLAC is reported to have high internal consistency and criterion-related validity, but the tool requires further psychometric evaluation of inter-rater and test-retest reliability. 7 The facial expression component of the PACSLAC calls for judgments of presence or absence of 13 facial behavioral items, with the intensity of pain determined by the number of items observed. The 13 items follow: grimacing, sad look, tighter face, dirty look, change in eyes (squinting, dull, bright, increased movement), frowning, pain expression, grim face, clenching teeth, wincing, opening mouth, creasing forehead, and screwing up nose. PAINAD Scale The PAINAD was intended to assess pain in elders with advanced dementia. 16 The tool consists of 5 items: breathing, vocalization, facial expression, body language, and consolability. Each item is scored on a scale of 0 to 2, according to specific descriptions provided for each level. The internal consistency of the PAINAD was found to be moderate, although it received good support to inter-rater reliability. 1,7,22 The facial expression component of the PAINAD requires a score of 0 to be used for a smiling or inexpressive face, 1 for a sad, frightened, or frowning face, and 2 to facial grimacing. Summary of Facial Expression Items on the Scales There was variable agreement between the descriptions of the facial expression of pain on the observational scales with the account provided by empirical studies of facial displays of pain coded using FACS. The core actions associated with pain were described by Prkachin as 30 : brow lowering, orbit tightening, upper-lip raising/nose wrinkling, and eye narrowing or closure, with variability relating to the severity of pain being suffered, the source of the distress, and personal and situational variables. 8 One also may observe horizontal lip corner pulling, oblique pulling at the corner of the lips, mouth open through to vertical stretching of the mouth, and other facial actions unique to the individual. When addressing the most severe level of pain, the Doloplus-2 description of a permanent and unusually blank look (voiceless, staring, looking blank) differed the most substantially. Note that a blank expression is associated with no pain on the MPS. The MPS used some terms corresponding to the FACS-derived account. The APS requires judgments on whether tense, frowning, grimacing, or frightened faces were displayed, thereby providing little explicit advice and directing attention to facial activity associated with emotions not necessarily observed during pain. The NOPPAIN scale again succeeds at least partially in directing attention to some features of the facial display of pain, requiring the presence of the following for the most severe judgment: grimaces, winces, and furrowed brow. The PACSLAC facial expression component directs attention to the 13 descriptors of facial activity noted above, some of which are observed during pain, and some of which are observed during non-noxious but aversive emotions. The PAINAD is relatively nonspecific in its requirement that the rater identify expressions such as sad, frightened, frown, or facial grimacing with sad, frightened, and frown expressions not always associated with pain and contributing to false positives. Procedure Video Coding and Data Collection Advertisements for recruitment of coders were posted on campus, with 5 coders, 3 male and 2 female, hired on a first-come-first-serve basis. Literature on k coefficients show that adequate inter-rater agreement (k=0.51, 67%) can be achieved between 2 raters on an ordinal scale with 4 levels. 31 The percentage of inter-rater agreement increases over chance as the number of raters increases; thus, a total of five raters for the study provided sufficient power to reliably examine inter-rater reliability. Rosenthal study 32 on estimating effective reliability R, or the overall reliability among a group of judges, indicates that the reliability of the mean of a group of judges ratings exceeds agreement obtained between pairs of raters. Further, the effective reliability increases with more judges because of greater cancellation of random errors between the different raters in rating. Given an estimated mean reliability of r=0.51, a sample of 5 raters would produce a high effective reliability at Therefore, ratings from 5 judges provided sufficient power to achieve sound effective reliability. Coders were required to be 19 years of age or older and to have healthy vision. All coders completed their ratings individually in the laboratory. They were orally briefed and provided with an instruction sheet describing the standardized procedure to follow for coding the clips. Coders were kept blind as to the intensities of pain represented in the clips, and coded each of the 6 instruments in a randomized order to guard against order and practice effects. Coders coded all 90 clips on one tool before moving on to the next tool. They did not have access to earlier ratings once a scale had been completed. They were aware that the full set of ratings would require 4 to 6 hours r 2011 Lippincott Williams & Wilkins

5 Clin J Pain Volume 27, Number 7, September 2011 Coders completed the task in multiple 1 to 2 hour sessions over 1 week. Coders paced themselves and took breaks whenever they wished to do so. RESULTS We examined inter-rater reliability, construct (discriminant) validity, and concurrent validity of the facial expression components of the 6 nonverbal observational assessments. Inter-rater Reliability The extent to which the 5 raters agreed with each other on each specific scale was determined for pairs of raters. k statistics indicating agreement between pairs of raters at each pain intensity level were calculated for the 6 scales. The range and mean of k values at each intensity level for each of the 6 scales are reported in Table 1. k could not be computed in cases when 2 raters failed to use overlapping levels on an observational scale at least once. This was observed in many of the calculations, particularly with the NOPPAIN and PACSLAC. In general, ranges of k were large and the means were relatively low, indicating that the level of agreement was not particularly sound, although greater agreement in the means was observed as pain intensity increased. Overall, using standard criteria, 31 the mean k values indicated fair agreement between the raters at the mild intensity level, and moderate agreement at the moderate and severe intensity levels. Rosenthal approach to estimating effective reliability allowed an estimate of the overall mean reliability of the 5 coders ratings. 32 On the basis of the mean k value calculated across the different scales for each intensity and the number of judges, the effective reliability for mild, moderate, and severe pain intensity was 0.47, 0.62, and 0.68, respectively. Hence, the reliability of the integrated judgments across the 5 coders was satisfactory, thereby justifying contrasts across scales and intensities. As the range of scores available for the PACSLAC was considerable (0 to 13), Pearson product-moment correlation coefficient was used as a method of second preference to evaluate the scale s inter-rater reliability. Analyses indicated that correlations for mild intensity ranged across pairs of raters from r=0.00 to r=0.68, with a mean of r=0.25. The range for moderate intensity was r=0.33 to TABLE 1. Range and Mean of k Values for Each Scale at Mild, Moderate, and Severe Pain Intensities Observational Pain Scale Mild Moderate Severe NOPPAIN 0.23 PAINAD , M= , M= , M=0.42 Abbey Pain Scale , M = , M= , M=0.31 Mahoney Pain Scale , M= , M= , M=0.32 DOLOPLUS , M= , M= , M=0.38 PACSLAC 0.02 k statistics that could not be computed were omitted. NOPPAIN indicates Noncommunicative Patient s Pain Assessment instrument; PACSLAC, Pain Assessment Checklist for Seniors with Limited Ability to Communicate; PAINAD, Pain Assessment in Advanced Dementia. r=0.74, and the mean r=0.55. Finally, the range for severe intensity was r=0.48 to r=0.63, and the mean of r=0.53. The ranges of coefficients became narrower as the intensity increased, and the mean levels of agreement were higher at moderate and severe intensities. This indicates greater agreement of scores where more pain was observed, which was consistent with the general trend observed in the k analysis. As a result of differences in the metrics of the scales (potential ranges varied between 0 to 2 and 0 to 13), scores from each scale were arithmetically converted into a standardized range of 0 to 10 to enable comparison across scales. The mean scores for the 6 scales for each of the raters at each intensity level are graphically displayed in Figures 1, 2 and 3. The more dispersion between the lines, the less agreement there was among the pain scores of the 5 raters. The relatively large dispersions displayed on the graphs indicate considerable disagreement and correspond to the wide ranges of k values. Construct Validity of Scales Construct validity was evaluated by examining whether scales differentiated the 3 levels of the facial expression of pain. We examined whether mean ratings for scales differed from each other, and whether scale mean scores and raters differed from each other, and interactions using a mixed model ANOVA, with a priori pain intensity (mild, moderate, and severe) and the 6 scales as withinpatient variables and raters as a between-patient variable. There was a main effect for intensities, F(2,28)=9.00, P= Overall, the mean ratings for the 3 intensities significantly differed from each other, M=1.79, SD=0.91 at mild, M=2.57, SD=0.32 at moderate, and M=3.84, SD=0.54 at severe. Furthermore, there was a main effect for scales, F(5,25)=22.36, P< Pairwise comparisons showed no significant differences between the NOPPAIN and Mahoney, the PAINAD and APS, the PAINAD and Doloplus-2, and the APS and MPS scales. There was not a significant overall effect in pain ratings between the different raters; however, there was a significant interaction between rater and intensity, F(8,22)=2.45, P=0.046, and rater and scale, F(20,10)=3.70, P=0.019, with the interaction between intensity and scale approaching significance, F(10,20)=2.08, P= The significant variability in the scores of individual raters evident in Figures 1, 2, and 3 represents the significant interactions between rater and intensity and rater and scale. To determine sources of the almost significant interaction between intensity and scale, we performed pairwise comparisons of the means between each intensity level for each of the 6 scales. This identified which pain scale differentiated the 3 pain intensity levels (Fig. 4). Owing to the large number of comparisons, significance was reported at the 0.01 level to control for the potential inflation of type I error. The means and standard deviations of each scale at the different intensities are reported in Table 2. Results of tests of pairwise comparisons appear in Table 3. The APS, MPS, and PACSLAC facial items showed significant differences in mean scores across all 3 intensity levels. There were no significant differences in mean scores between mild and moderate intensities on the NOPPAIN scale, and between moderate and severe on the PAINAD. The Doloplus-2 did not distinguish the 3 levels of pain. A similar set of comparisons was undertaken across the means for all scales at each level of pain intensity. Three multivariate analyses were performed, 1 for each of pain r 2011 Lippincott Williams & Wilkins 597

6 Sheu et al Clin J Pain Volume 27, Number 7, September 2011 FIGURE 1. Mean scores for each rater using each pain scale at mild pain intensities. levels mild, moderate, and severe, to investigate if differences between the pain scores obtained by the comparison scales were significant. A significant difference was found between the different scales for the mild clips, F(5,25)=17.46, P< Subsequent post hoc analysis using paired sample t tests was conducted to determine which scales differed from each other. Findings show that scores from the PACSLAC were significantly lower than scores from all other scales, and scores from the Doloplus-2 were significantly higher than the rest. There was also a significant difference between the pain levels for the moderate clip, F(5,25)=8.66, P< In addition, the PACSLAC had a significantly lower score compared with all other scales. The NOPPAIN scored significantly below the PAINAD, APS and Doloplus-2, and the PAINAD scored higher than the APS and MPS. Finally, scores for the severe clips also differed significantly from each other, F(5,25)=11.97, P< The PACSLAC once again showed a significantly lower score than all other scales, and the NOPPAIN scored significantly below the APS. Concurrent Validity of Scales Comparison of scores on the individual observational scales with FACS scores was undertaken, as FACS provided an objectively coded criterion measure. Pearson bivariate correlations between mean pain scores for each comparison scale and the FACS score are reported for the 3 intensities in Table 4. Correlations were substantially greater when judgments were made of moderate and severe expressions of pain in contrast to judging mild pain. No significant correlations were observed between scores on the Doloplus-2 and those on the FACS at any of the pain intensities. Only the PAINAD and MPS showed significant correlations with the FACS at all 3 intensity levels. The NOPPAIN, APS, and PACSLAC had significant correlations at the moderate and severe but not the mild intensities. The FACS/Doloplus-2 correlation coefficient was negative at the mild intensity level, indicating an inverse relationship between scores on the 2 scales. DISCUSSION Overall, the results of this study were consistent with the expectation that observational scales that provide descriptors that correspond to how people display facial expressions of pain would perform better at differentiating intensities of pain and have better psychometric properties. Empirically based definitions of criteria for coder judgments improved reliability. The measure of inter-rater reliability indicated that there often was substantial disagreement FIGURE 2. Mean scores for each rater using each pain scale at moderate pain intensities r 2011 Lippincott Williams & Wilkins

7 Clin J Pain Volume 27, Number 7, September 2011 FIGURE 3. Mean scores for each rater using each pain scale at severe pain intensities. among raters. Although the effective reliability of the mean ratings across the 5 raters was stable and justified comparisons across scales, there was relatively low agreement between pairs of raters. Mean k values reflected fair agreement between pairs of raters at the mild intensity level, and only moderate agreement was present at the moderate and severe levels. 32 The inadequate inter-rater reliability observed seemed attributable to the subjectivity involved in the scoring criteria, for example requiring global judgments of being frightened, frowning, or grimacing. If facial characteristics are not empirically described, different raters can hold contrasting opinions on what is understood by the same terminology, which leads to poor consistency in rating. Nevertheless, raters were in better agreement as pain intensity increased, as overt behavior display became more vigorous. k statistics could not be computed for the PACSLAC and NOPPAIN because of the absence of overlap in the range of rating levels used by individual raters. These scales offered opportunities for greater differences in ratings, as PACSLAC scores could range from 0 to 13 and NOPPAIN scores across 6 levels. Correlational analyses of PACSLAC scores did indicate greater agreement as intensity increased, which supported the general pattern of results. The same analyses could not be used with the NOPPAIN, as the range of scores was not large enough to allow for reliable computations. In terms of construct validity, the APS, MPS, and PACSLAC significantly differentiated among the 3 levels of pain. The NOPPAIN and PAINAD scales were less satisfactory, as the former failed to distinguish between mild and moderate intensities, and the latter between moderate and severe. Pain judgments on the NOPPAIN are guided by directions to attend to grimaces, winces, and furrowed brow. This combines vague description with partial, but incomplete direction. A further challenge to the rater is the requirement to assess intensity with no observable descriptors provided for different levels of intensity. Raters could easily differ as to the pain severity being depicted. The PAINAD would seem to perform relatively poorer because the behavioral descriptors provided confound emotions not always associated with pain with response to noxious events. Although a smiling or inexpressive face could accord with a face showing little or an absence of pain, the pleasure associated with smiling does not need to be present when pain is absent, and it is not unusual for patients to be smiling even when they are in pain. 33 The second level on the PAINAD scale requires judges to decide whether they have viewed a face that is sad, frightened, or a frown. These refer to emotional states sometimes present during pain, but often also present when pain is absent. Treating patients for pain when non-noxious, but aversive emotions were present would be inappropriate. In addition, there are no descriptors that would assist the rater in specifically identifying moderate pain. Finally, the highest rating, 3, is to be assigned when facial grimacing is evident, but no specific behavioral descriptors are FIGURE 4. Mean scores of each pain scale at mild, moderate, and severe intensities of pain. r 2011 Lippincott Williams & Wilkins 599

8 Sheu et al Clin J Pain Volume 27, Number 7, September 2011 TABLE 2. Means (Using the Standardized 0 to 10 Scale) and Standard Deviations for Each Pain Scale at Mild, Moderate, and Severe Pain Intensities (n=30) Observational Pain Scale Pain Severity Mean Standard Deviation NOPPAIN** Mild Moderate Severe PAINAD*** Mild Moderate Severe Abbey* Mild Moderate Severe Mahoney* Mild Moderate Severe Doloplus-2 Mild Moderate Severe PACSLAC* Mild Moderate Severe *Refers to significant difference between all 3 intensity levels at P<0.01. **Refers to significant difference between severe and the other 2 intensities at P<0.01. ***Refers to significant difference between mild and the other 2 intensities at P<0.01. NOPPAIN indicates Noncommunicative Patient s Pain Assessment instrument; PACSLAC, Pain Assessment Checklist for Seniors with Limited Ability to Communicate; PAINAD, Pain Assessment in Advanced Dementia. provided to identify this state. Thus, misdirection and the lack of empirical descriptors characterize the PAINAD. The only scale that failed to detect differences between the 3 levels of pain depicted and did not correlate with the FACS measure was the Doloplus-2. The Doloplus-2 provides misleading and vaguely defined characterizations in its scoring criteria. The scale provides no explicit defining characteristics for a usual expression to be used in identifying the absence of pain. The highest rating, 3, called for observation of a permanent and unusually blank look (voiceless, staring, looking blank). This account of facial expressions during severe pain has no empirical support in the extensive number of studies in the current literature. 8 The various scales differed in the magnitudes of pain reported. When rating the most severe intensity, the highest levels of pain were reported on the PAINAD and the lowest on the PACSLAC. The score usually interpreted as signifying clinically important pain has been a rating greater than 3 on a 0to10scale. 34 All scales exceeded this for the most severe level of pain observed. It seems reasonable that the scales should risk false positives when identifying pain, as there are widespread tendencies to under estimate pain in other persons The inclusion of items on the PACSLAC facial expression checklist describing states other than pain, including those that are not necessarily present during the experience of pain (sad look, tighter face, dirty look, change in eyes to dull or bright, frowning, or clenching teeth) seems to account for the relatively low mean pain scores on this scale, as many were not observed. Correlations of pain scores with the FACS indicated the MPS and PAINAD outperformed the NOPPAIN, APS, and PACSLAC. Correlations between the NOPPAIN, Abbey, and PACSLAC and the FACS were found to be more consistent at the moderate and severe levels, but much weaker at mild. This indicated that these scales were less sensitive in their assessment when there is little or no pain. On the whole, the correlational analyses show that the MPS achieved above the other comparison scales. It is the only scale that significantly differentiated pain levels and was consistently correlated with the FACS at each level of pain. This was the only scale that consistently provided behaviorally defined visual cues for which observers could search, thereby increasing the likelihood of reliable and accurate assessment. However, there are features of the behavioral cues provided on the MPS that detract from its accuracy. Although anxiety may be present during a painful experience, the characterization of an anxious face (eyes wide and alarmed) is inconsistent with a display of pain. It is important to identify and treat fear, but fear should not be characterized as low-level pain. Similarly, reference to a sad expression, and its characterization including frowning or teary similarly misleads and diverts from detecting pain expression, albeit people in pain are often sad. This is important because interventions devoted to mood states differ from those addressing pain. There is considerable support for the validity of the PACSLAC in the existing literature. 38,39 The scale includes numerous facial items that are sensitive to pain. However, many items are imprecisely defined and require subjective judgments (eg, dirty look, pain expression). In summary, when considering facial expression items, the Mahoney seemed to be the most promising scale among the 6 compared. The APS, PACSLAC, and PAINAD all showed some potential for clinical use. The NOPPAIN may benefit from further improvements and developments, and the Doloplus-2 was found to be the least valid and reliable scale. In conclusion, the results of the study suggest that there is adequate construct and concurrent validity for the facial components of some commonly used observational measures of pain at present. This is especially true with regard to the MPS, followed by the APS, PACSLAC, and TABLE 3. Pairwise Comparisons Between Means at Mild, Moderate, and Severe Pain Intensities Observational Pain Scale Mild-Moderate Mild-Severe Moderate-Severe NOPPAIN 0.74, P= * 1.56* PAINAD 1.47* 2.67* 1.20, P=0.045 Abbey 1.09* 2.71* 1.62* Mahoney 0.98* 2.73* 1.76* Doloplus , P= , P= , P=0.481 PACSLAC 0.78* 1.86* 1.08* *P<0.01. NOPPAIN indicates Noncommunicative Patient s Pain Assessment instrument; PACSLAC, Pain Assessment Checklist for Seniors with Limited Ability to Communicate; PAINAD, Pain Assessment in Advanced Dementia r 2011 Lippincott Williams & Wilkins

9 Clin J Pain Volume 27, Number 7, September 2011 TABLE 4. Pearson Correlation Coefficients Between Each Pain Scale and the FACS at Mild, Moderate, and Severe Pain Intensities Observational Pain Scale Mild Moderate Severe NOPPAIN ** 0.592** PAINAD 0.412* 0.582** 0.552** Abbey ** 0.631** Mahoney 0.450* 0.675** 0.593** Doloplus PACSLAC ** 0.647** *P<0.05, 2 tailed. **P<0.01, 2 tailed. NOPPAIN indicates Noncommunicative Patient s Pain Assessment instrument; PACSLAC, Pain Assessment Checklist for Seniors with Limited Ability to Communicate; PAINAD, Pain Assessment in Advanced Dementia. PAINAD. They have good potential for clinical utility. However, most scales have limitations as identified here. Although reliability of the scales is insufficient at present, it is expected that both validity and reliability would benefit from further developments to improve the specificity and precision of their constructs. This can be achieved by adopting more objectively and anatomically defined characterizations of the facial expression of pain. REFERENCES 1. Zwakhalen SMG, Hamers JPH, Abu-Saad HH, et al. Pain in elderly people with severe dementia: a systematic review of behavioral pain assessment tools. BMC Geriatr. 2006;6:3. 2. Hadjistavropoulos T, Herr K, Turk DC, et al. An interdisciplinary expert consensus statement on assessment of pain in older persons. Clin J Pain. 2007;23: Snow AL, Weber JB, O Malley KJ, et al. NOPPAIN: a nursing assistant-administered pain assessment instrument for use in dementia. Dement Geriatr Cogn Disord. 2004;17: Tsai P, Chang JY. Assessment of pain in elders with dementia. MedSurg Nurs. 2004;13: Herk R, Dijk M, Baar FP, et al. Observational scales for pain assessment in older adults with cognitive impairments or communication difficulties. Nurs Res. 2007;56: Smith M. Pain assessment in nonverbal older adults with advanced dementia. Perspect Psychiatr Care. 2005;41: Bjoro K, Herr K, Decker S. Tools for assessment of pain in nonverbal older adults with dementia: a state-of-science review. J Pain Symptom Manage. 2006;3: Craig KD, Prkachin KM, Grunau RVE. The facial expression of pain. In: Turk DC, Melzack R, eds. Handbook of Pain Assessment. 3rd ed. New York: Guilford Press; 2011: Prkachin KM, Solomon PE. The structure, reliability and validity of pain expression: evidence from patients with shoulder pain. Pain. 2008;139: LeResche L. Facial behaviors related to pain in the elderly. Gerodontology. 1984;3: Kunz M, Mylius V, Schepelmann K, et al. On the relationship between self-report and facial expression of pain. J Pain. 2004;5: Labus JS, Keefe FJ, Jensen MP. Self-reports of pain intensity and direct observation of pain behavior: when are they correlated? Pain. 2003;102: Lints-Martindale AC, Hadjistavropoulos T, Barber B, et al. A psychophysical investigation of the facial action coding system as an index of pain variability among older adults with and without Alzheimer s disease. Pain Med. 2007;8: Ekman P, Friesen WV. Facial Action Coding System: a Technique for the Measurement of Facial Movement. Palo Alto, CA: Consulting Psychologists Press; Kunz M, Scharmann S, Hemmeter U, et al. The facial expression of pain in patients with dementia. Pain. 2007;133: Herr K, Coyne PJ, Key T, et al. Pain assessment in the nonverbal patient: position statement with clinical practice recommendations. Pain Manag Nurs. 2006;7: Abbey J, Piller N, Bellis A, et al. The Abbey pain scale: a 1- minute numerical indicator for people with end stage dementia. Int J Palliat Nurs. 2004;10: Simon D, Craig KD, Gosselin F, et al. Recognition and discrimination of prototypical dynamic expressions of pain and emotions. Pain. 2008;135: Lefebvre-Chapiro S. The DOLOPLUS 2 Scale: evaluating pain in the elderly. Eur J Palliat Care. 2001;8: Mahoney AEJ, Peters L. The Mahoney pain scale: examining pain and agitation in advanced dementia. Am J of Alzheimer s Dis Other Dem. 2008;23: Snow AL, O Malley KJ, Cody M, et al. A conceptual model of pain assessment for noncommunicative persons with dementia. Gerontologist. 2004;44: Fuchs-Lachelle S, Hadjistavropoulos T. Development and preliminary validation of the Pain Assessment Checklist for Seniors with Limited Ability to Communicate (PACSLAC). Pain Manag Nurs. 2004;5: Warden V, Hurley AC, Volicer L. Development and psychometric evaluation of the Pain Assessment in Advanced Dementia (PAINAD) scale. J Am Med Dir Assoc. 2003;4: Mcaulay C, Battista M, Lebby PC, et al. Geriatric performance on the neurobehavioral cognitive status examination (Cognistat): what is normal? Arch Clin Neuropsychol. 2002;18: Prkachin KM, Mercer SR. Pain expression in patients with shoulder pathology: validity, properties and relationship to sickness impact. Pain. 1990;39: Defrin R, Lotan M, Pick CG. The evaluation of acute pain in individuals with cognitive impairment: a differential effect of the level of impairment. Pain. 2006;124: Hadjistavropoulos T, LaChapelle DL, MacLeod FK, et al. Measuring movement-exacerbated pain in cognitively impaired frail elders. Clin J Pain. 2000;16: Faul F, Erdfelder E, Lang AG, et al. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39: Holen JC, Saltvedt I, Fayers PM, et al. Doloplus-2, a valid tool for behavioral pain assessment? BMC Geriatr. 2007;7: Prkachin KM. The consistency of facial expressions of pain: a comparison across modalities. Pain. 1992;51: Haley SM, Osberg JS. Kappa coefficient calculation using multiple ratings per subject: a special communication. Phys Ther. 1989;69: Rosenthal R. Estimating effective reliabilities in studies that employ judges ratings. J Clin Psychol. 1973;29: Kunz M, Prkachin K, Lautenbacher S. The smile of pain. Pain. 2009;145: Falanga IJ, Lafrenaye S, Mayer SK, et al. Management of acute pain in children: safety and efficacy of a nurse-controlled algorithm for pain relief. Acute Pain. 2006;8: Chambers CT, Reid GJ, Craig KD, et al. Agreement between child and parent reports of pain. Clin J Pain. 1998;14: Prkachin KM, Solomon PA, Ross AJ. The underestimation of pain among health-care providers. Can J Nurs Res. 2007;39: Kappesser J, Williams AC, Prkachin KM. Testing two accounts of pain underestimation. Pain. 2006;124: Zwakhalen SMG, Hamers JPH, Berger MP. The psychometric quality and clinical usefulness of three pain assessment tools for elderly people with dementia. Pain. 2006;126: Van Nispen tot Pannerden SC, Candel MJ, Zwakhalen SM, et al. An item response theory-based assessment of the pain assessment checklist for seniors with limited ability to communicate (PACSLAC). J Pain. 2009;10: r 2011 Lippincott Williams & Wilkins 601