Psychometric Evaluation of the Revised American Pain Society Patient Outcome Questionnaire (APS-POQ-R) in Postoperative Patients

Transcription

1 Psychometric Evaluation of the Revised American Pain Society Patient Outcome Questionnaire (APS-POQ-R) in Postoperative Patients Hui Wang 1, Gwen Sherwood 2, Zhi-yi Gong 3, Hua-ping Liu 1 1.School of Nursing, Peking Union Medical College Hospital, University of North Carolina at Chapel Hill 3.Department of Anesthesiology, Peking Union Medical College Hospital, Abstract Background: Poor postoperative pain management has posed a pervasive clinical problem to professionals in medical practice. The effective measurement of pain management outcome is one of the essential parts to improve the pain management quality. The Revised American Pain Society Patient Outcome Questionnaire (APS-POQ-R) fully reflects the pain management quality, while its reliability and validity has not been verified in the operation. Objectives: To verify the reliability and validity of APS-POQ-R in postoperative patients and to discuss the relationship between pain degree and other outcomes of pain management. Methods: To randomly select 244 hospitalized adult patients of 5 surgical departments in a hospital in North Carolina, US. To measure the pain management result of the patients with the use of APS-POQ-R in the first postoperative day and the average pain degree in postoperative 24 hours respectively. Meanwhile, it needs to get the general demographic information and surgical information of the patients by the means of checking the cases or the patients self-reports. Results: The total internal consistency of the questionnaire (Cronbach s alpha) is Factor analysis testifies the five subscales of the original chart: effects of pain on the sleep, effects of pain on emotions, adverse effects of pain treatment, effects of pain on activities and the perception of pain service, accounting for 61.41% of the total variation. The total internal consistency in the perception of pain service is much low (0.510), which probably related with the sample homogeneity; the total internal consistency of other dimensions is According to the regression analysis, the average and largest pain degree could explain effects of pain on activities is 18.6 of the variation, the percentage of time and average pain degree in severe pain respectively explain 28.9% and 16.8% of the variation of pain on sleep and emotions. The 17.9% variation of patients satisfaction with pain treatment can be predicted by pain relief degree and the lightest pain degree. Conclusions: This study testified the reliability and validity of APS-POQ-R in the patients, which shows that it is feasible to apply APS-POQ-R to measure patient outcomes in pain management for internal consistency. The items (the lightest, heaviest pain degree and the percentage of time in severe pain) of pain degree are the important predicted factors in other pain management outcomes. As for the effects of average pain on pain outcome, it is much reasonable to integrate them into APS-POQ-R. Key Words: APS-POQ-R; Pain management outcome; Reliability and validity; Psychometric evaluation Corresponding Author: Hua-ping Liu, huapingliu@vip.126.com Introduction Postoperative pain management has posed a pervasive clinical problem to professionals in medical practice. [4,10,13,24] Poor postoperative pain management may ause series of negative physiological and psychological consequences, including painful suffering, and delayed recovery. [3] What s worse, severe acute postoperative pain might be associated with risk of developing persistent pain subsequently. [3,4,12,28] Pain management has become routine task under evidence based guidelines in many countries, [3,4,10,13,16] but it is often overlooked by clinical professionals in China. [24] Valid and reliable measurement is essential to explore effective pain treatment strategies and evaluate quality improvement in pain management. [5,11] With the development of medicine and society, the concept and measurement of pain management quality is changing. [10,11] In 1991, the American Pain Society (APS) published its first patient Satisfaction survey questionnaire as part of quality assurance standards for pain management, including acute and cancer pain. [17] In 1995, the APS revised quality assurance Laboratory and Clinical Investigation 285 FAM 2013 August Vol.20 Issue 4

2 standards as well as Satisfaction survey questionnaire, which was renamed as Patient Outcome Questionnaire, (APS-POQ). [1] The APS-POQ was composed of items from previously validated questionnaires, including pain intensity, interference with function, patient satisfaction, and barriers to pain management. In 1998, McNeill modified the APS-POQ (APS-POQ-Modified) by increasing an open question asking why patients were satisfied with pain management and querying pain management approaches. [18] The APS-POQ or APS- POQ-Modified was widely used in many studies, [18,19] as well as different countries, including Taiwan. [7,8,15,22] Psychometric characteristics were verified except the satisfaction scale in APS-POQ or APS-POQ-Modified. [7,8,18,19,20] Patients satisfaction is consistently skewed toward the high end even in patients with severe pain. [7,8,18,20] (Dihle, Helseth, Kongsgaard, Paul, & Miaskowski, 2006; DihleHelseth & Christophersen, 2008; McNeill, Sherwood, Starck, & Nieto, 2001; McNeill, Sherwood, Starck, & T hompson, 1998) Most of the studies were conducted in a mixed sample of patients with postoperative pain, cancer related pain, or other painful conditions. [15,18,19,20] The questionnaire was also used in patients with cancer or surgery separately. [8,22] In a d d i t ion, to a d d re s s d i s p a r i t y i n t reating postoperative pain in Hispanics, McNeill et al. developed a pain outcome instrument in Spanish, the Houston Patient Outcome Instrument (HPOI) based on the prototype APS-POQ and refined through qualitative interviews. [26] The HPOI is composed of 5 subscales, pain intensity, interference with emotions, interference with activities, satisfaction with pain treatment, and satisfaction with information about pain management. HPOI was widely used in America and has been tested in Chinese postoperative patients in mainland. [24,25] Similarly, patient satisfaction is also highly reported even in patients with high pain intensity. [24] In the year 2005, the APS revised the recommendation for high quality of pain management based on amount of evidence. [10] High quality pain management has multiple levels: including appropriate assessment; interdisciplinary and collaborative planning; effective, cost efficient, culturally and developmentally appropriate, and safe; and access to specialty care if needed. [10] Additionally, measurement of patient satisfaction with pain management and patient beliefs about pain was not recommended.10 Accordingly, a revised version of APS-POQ (APS-POQ-R) was developed in 2010 to measures 6 aspects of pain management quality, including pain severity and relief, impact of pain on function and emotions, adverse effects of treatment, helpfulness of information, participation in decision making and nonpharmacological methods of pain management. [11] Only one item asking patients satisfaction about results of pain treatment was included in the APS-POQ-R and located in the subscale of perception of pain care in factor analysis. APS-POQ-R was verified to Table1 Characteristics of Sample Variable (N=244) Mean SD Age N % Gender Male Female Education Primary school and lower 2.8 Middle school high school/ged College and higher Marital Unmarried Married Divorced Widowed Others Missing Religions None Christian Others History of surgery No Yes Location of surgery Head/neck Thorax Abdomen/pelvis Extremities and joints Spine Others Missing Category of surgery Endoscopic Non-endoscopic Both/Endoscopic transfer to Missing Type of diagnosis Non-cancer/tumor Benign tumor 3 Cancer Unknown Analgesia after surgery None 1.4 Non-PCA PCA Both Laboratory and Clinical Investigation 286 FAM 2013 August Vol.20 Issue 4

3 have excellent psychometric characteristics and patients Table2 Descriptive statistics of items of APS-POQ N Minimum Maximum Mean SD Least pain Worst pain Percentage of time in severe pain activities in bed activities out of bed falling asleep staying asleep anxious depressed frighten helpless Nausea Drowsiness Itching Dizziness Pain relief Allowed to participate in decisions about pain treatment? How satisfied are you with the results of your pain treatment? Did you receive any information about your pain treatment option? No % Yes % Missing 1 0.4% How helpful the information was Did you use any non-medicine method to relieve your pain? No % Yes % Non-medicine method used N percent% Percent of cases Cold pack % 27.4% Meditation % 11.0% Deep breathing % 43.2% Listen to music % 13.0% Distraction % 40.4% Prayer % 47.9% Heat % 11.6% Relaxation % 28.8% Imagery or visualization 8 2.0% 5.5% Walking % 21.2% Massage % 14.4% Total % 275.3% How often did a nurse or doctor encourage you to use non-medication methods? Never % Sometimes % Often % Missing 2 0.8% satisfaction can be predicted by variety of items in the questionnaire very well. [11] But its validity and reliability haven t been confirmed in other countries or postoperative patients only. Materials and Methods Sample A convenience sampling method was used in the study. During the period from January 04, 2012 to May 18, 2012, 244 adult patients on the first postoperative day were recruited from 5 units in a teaching hospital in North Carolina, US. The study protocol was reviewed and approved by the institutional review boards in School of Nursing in University of North Carolina (UNC) and administrative department of the participating hospital. Data Analysis The data was analyzed using SPSS version Descriptive analyses were used to describe the characteristics of the sample and pain management outcomes. The psychometric characteristics of the questionnaire were tested using internal consistency reliability, exploratory factor analysis, ANOVA or student t test among contrasting groups. And stepwise linear regression was used to predict satisfaction of pain management. Table3 Internal consistency reliability of APS-POQ-R Cronbach s Alpha of total Corrected Cronbach's Item-Total Alpha if Item Correlation Deleted Least pain Worst pain Percentage of time in severe pain activities in bed activities out of bed falling asleep staying asleep anxious depressed frighten helpless Nausea Drowsiness Itching Dizziness Pain relief Allowed to participate in decisions about pain treatment? How satisfied are you with the results of your pain treatment? Laboratory and Clinical Investigation 287 FAM 2013 August Vol.20 Issue 4

4 Results Sample Characteristics Sample characteristics are summarized in table 1. Participants were 50.85± (range from18 to 84) years old in average and composed of more female participants (male 38.5%, female61.5%). The majority of the participants were diagnosed as non-cancer/tumor (n=183, 75.0%) and had history of surgery (n=215, 88.1%) before. Most of the surgery was non-endoscopic (n=174, 71.3%) and located in abdomen/pelvic (n=114, 46.7%). Only 0.4% (n=1) of the participants didn t use any analgesics to treat pain on the first day after surgery. 41.8% (n=102) used non-pca, 4.9% (n=12) PCA, and 52.9% (n=129) both. Description of the APS-POQ-R Items There were 18 items in which the response was measured by 0 to 10 NRS and treated as continuous scale. The means, standard deviations, minimum and maximum scores for the continuous items and answers for information items are shown in the table2. Only 74.2% (n=181) had received information about pain management and helpfulness was rated 8.63±2.255 in average. 59.8% (n=146) had used nonmedicine method to manage postoperative pain, of whom 47.9% used prayer, 43.2% used deep breath and 40.4% used distract. 56.1% of the participants reported that they were never encouraged to use non-medicine methods by nurses or doctors. Internal Consistency Reliability of APS-POQ-R Overall Cronbach s alpha for 18 items was Absolute value of inter item correlations ranged from to 0.675, lower than 0.7 indicating that no items highly correlated. 11 Item to total correlation and changes in Cronbach s Alpha if item deleted from the questionnaire are shown in table3. Construct Validity of APS-POQ-R and Subscales Exploratory factor analysis was used to evaluate construct validity of the questionnaire. During the procedure, principal components and Equamax rotations were adopted excluding cases pairwise. Five factors were extracted with eigenvalues over 1(1.119 to 5.208) and % of the total variance was explained cumulatively. Table 4 of rotated component matrix shows five factors extracted is similar to the original APS-POQ except the item inching. The first factor of Pain severity and sleep interference subscale accounted for14.291% of the total variance. Factor 2 is related to emotional responses of pain naming Affective Table 4 Rotated Component Matrix Component Subscale Pain Severity and Sleep Interference Affective Adverse effects Activity interference Perception of care Variance explained (61.407% total) % % % % % Least pain Worst pain Percentage of time in severe pain activities in bed activities out of bed falling asleep staying asleep anxious depressed frighten helpless Nausea Drowsiness Itching Dizziness Pain relief Allowed to participate in decisions about pain treatment? How satisfied are you with the results of your pain treatment? Extraction Method: Principal Component Analysis. Rotation Method: Equamax with Kaiser Normalization. a. Rotation converged in 7 iterations. Laboratory and Clinical Investigation 288 FAM 2013 August Vol.20 Issue 4

5 subscale, including anxious, depressed, frightened, and helpless, accounting for % of the variance. Side effects of nausea, drowsiness, itching and dizziness are belonged to the third factor and explain % of the total variance % of variance is contributed by interference with activities in and out of bed. Subscale of Perception of care is defined by pain relief, participation in decision and satisfaction with pain treatment (10.503% of the variance). The item to total correlations and Cronbach s Alpha of subscales were calculated and showed in the table 5. Cronbach s Alphas of the subscales were acceptable (affective in 0.791, activity interference 0.772, pain severity and sleep interference 0.797, adverse effects0.647) except perception of care (0.510). Inter correlations of subscales were calculated and shown in table 6. No subscale was highly correlated with others and most of them were significantly correlated with each other as expected, except for adverse effect and perception of care. Relationship between pain severity and other outcomes To find relationship between pain severity and other outcomes, stepwise regression was used. The results showed that average and worst pain predicted 18.6% of the variance of activity interference. Time in severe pain and average pain accounted up to 28.9% of the sleep interference and 16.8% of affective interference. 17.9%.of the patient satisfaction was explained by degree of pain relief and least pain. Discussion APS-POQ-R is the latest version of questionnaires that measure patient outcomes in pain management for quality improvement. Factor analysis verified the structure validity of APS-POQ-R, which is composed of five subscales, Pain severity and sleep interference, Affective, Adverse effects, Activity interference, and Perception of care. Internal consistency reliability of overall questionnaire and subscales was good except for the Perception of care subscale. Overall Cronbach s Alpha for 18 items (0.770) was lower than that in original study tested in American inpatients sampled from both surgical and medical service units (0.86). [11] but still acceptable suggested by Nunnally and Bernsterin (1994). [21] The general internal consistence is also comparable to that of HPOI in Mexican (0.76) and Chinese patients (0.77). [23,24,25] Internal consistency reliability of three subscales in the study was similar to those obtained by original re s e a rchers, A f f e c t ive ( v s ), Ac t iv i t y Table 5 Subscale Item to Total Correlations and Cronbach s Alpha Cronbach s Alpha Subscales (variance) and Items Of subscale Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted Pain severity and sleep interference subscale (14.291%) Least pain Worst pain Percentage of time in severe pain falling asleep staying asleep Affective subscale (14.159%) anxious depressed frighten helpless Adverse effect subscale (11.271%) Nausea Drowsiness Itching Dizziness Activity interference subscale (11.183%) activities in bed.632.a activities out of bed.632.a Perception of care subscale (10.503%) Pain relief Allowed to participate in decisions about pain treatment? How satisfied are you with the results of your pain treatment? Laboratory and Clinical Investigation 289 FAM 2013 August Vol.20 Issue 4

6 interference (0.812 vs 0.82), and Adverse effects (0.636 vs 0.63); while lower for two other subscales, Pain severity and sleep interference (0.773 vs 0.83) and Perception of care (0.492 vs 0.70). Most of them are similar to the internal consistence of subscales of APS-POQ-Modified, ranging 0.68 to 0.82, [18] except for Perception of care. There are several reasons for unsatisfactory reliability of Perception of care. Mostly, reliability might be lower if sample is more homogenous, which is also founded in the study evaluating Norwegian version of APS-POQ in orthopaedic surgical patients, whose internal consistency reliability for satisfaction subscale is [7] Additionally, the number of the items also influenced the internal reliability. Non-medicine methods are important component in pain management and recommended in many guidelines in different organizations and countries. [4,6,9,10,16] Patient satisfaction is an important but elusive outcome indicator, [27] making it significant but difficult to predict. In this study, satisfaction with pain treatment was largely predicted by degree of pain relief, instead of pain intensity. As shown previously, no straightforward relationship between satisfaction and pain severity was found. [27] But in the study by Gordon et al., percent time spent in severe pain was included in the predicting model. [11] In the predicting model, items of Pain severity are important predictors for other outcomes. The pain severity items in the APS-POQ-R, the least, worst pain level, and time in severe pain are very important predictors of other pain outcomes. Average pain level is another very important predictor of the patient outcomes. Limitations Several limitations of this study need improvements. The sample was convenience recruited as from one hospital, which might be not representative to the population of postoperative patients. Further studies ought to be performed using more representative samples in other countries. Postoperative acute pain management quality was target of the questionnaire in this study. But we can t avoid patients reporting chronic or other acute pain they have other than pain related to surgery. Since recalling pain intensity in different time scope might lead to different results, time to interview patients might impact the psychometric validity and reliability of the questionnaire. Participants were approached on the first postoperative day in this survey, in which might miss patients in severe pain having difficulty to respond. More studies should be done at different time after surgery. Conclusion This study verified acceptable reliability and validity of APS-POQ-R in a postoperative sample, which implies that universal measurement of pain management quality in medical and surgical patients is reasonable. Five important outcomes in pain management can be Table 6 Inter Correlation Matrix of subscales Pain severity and sleep interference Affective Activity interference Adverse effect Perception of care Pain severity and sleep interference Affective Activity interference Adverse effect Perception of care Pearson Correlation 1.513**.466**.234** -.211** Sig. (2-tailed) N Pearson Correlation.513** 1.408**.270** -.192** Sig. (2-tailed) N Pearson Correlation.466**.408** 1.263** -.132* Sig. (2-tailed) N Pearson Correlation.234**.270**.263** Sig. (2-tailed) N Pearson Correlation -.211** -.192** -.132* Sig. (2-tailed) N **. Correlation is significant at the 0.01 level (2-tailed). *. Correlation is significant at the 0.05 level (2-tailed). Laboratory and Clinical Investigation 290 FAM 2013 August Vol.20 Issue 4

7 measured, pain severity and interference with sleep, activity interference, negative emotional responses of pain, side effects of pain medicine, and perception of pain management. Internal consistence reliability was acceptable except for the subscale of Perception of pain care. The pain severity items in the APS-POQ-R, the least, worst pain level, and time in severe pain are very important predictors of other pain outcomes. It is recommended that average pain level as another important predictor of the patient outcomes should be added to the questionnaire. REFERENCES [1] American Pain Society Quality of Care Committee. Quality Improvement Guidelines for the treatment of acute pain and cancer pain. JAMA. December 20, 1995; 274(23): [2] American S O A. Practice guidelines for acute pain management in the perioperative Setting: An updated report by the American Society of Anesthesiologists task force on acute pain management [J]. Anesthesiology, 2004, 100: [3] Berry P H, Dahl J L. The new JCAHO pain standards: implications for pain management nurses [J]. Pain Management Nursing, 2000, 1(1):3-12. [4] Breivik H, Stubhaug A. Management of acute postoperative pain: Still a long way to go! [J]. Pain, 2008, 137: [5] Clark M E, Gironda R J, Young R W. Development and validation of the pain outcomes questionnaire-va[j]. J Rehabil Res Dev, 2003, 40(5): [6] Devine E C, Bevsek S A, Brubakken K, et al. AHCPR clinical practice guideline on surgical pain management: adoption and outcomes [J]. Research In Nursing & Health, 1999, 22: [7] Dihle A, Helseth S, Christophersen K. The Norwegian version of the American pain society patient outcome questionnaire: reliability and validity of three subscales [J]. J Clin Nurs, 2008, 17: [8] Dihle A, Helseth S, Kongsgaard U E, et al. Using the American pain society s patient outcome questionnaire to evaluate the quality of postoperative pain management in a sample of Norwegian patients[j]. J Pain, 2006, 7(4): [9] Dod V. VHA/DoD clinical practice guideline for the management of postoperative pain[r], [10] Gordon D B, Dahl J L, Miaskowski C, et al. American pain society recommendations for improving the quality of acute and cancer pain management [J]. Arch Intern Med, 2005, 165: [11] Gordon D, Polomano R, Pellino T, et al. Revised American Pain Society Patient Outcome Questionnaire (APS-POQ-R) for Quality Improvement of Pain Management in Hospitalized Adults: Preliminary Psychometric Evaluation [J]. J Pain, 2010:1-15. [12] Hickey O T, Burke S M, Hafeez P, et al. Severity of acute pain after breast surgery is associated with the likelihood of subsequently developing persistent pain [J]. Clin J Pain, 2010, 26(7): [13] Kumar N. WHO normative guidelines on pain management [J] [14] Lichter I. Communication in cancer care. Edinburgh: Churchill Livingstone, [15] Lin C. Applying the American Pain Society's QA standards to evaluate the quality of pain management among surgical, oncology, and hospice inpatients in Taiwan [J]. Pain, 2000, 87(1): [16] Macintyre P E, Scott D A, Schug S A, et al. Acute pain management: scientific evidence [J] [17] Max MB.American Pain Society quality assurance standards for relief of acute pain and cancer pain.in: Bond MR, Charlton JE, Woolf CJ, eds. Proceeding of the Sixth World Congress on Pain. New York:Elsevier, [18] McNeill J A, Sherwood G D, Starck P L, et al. Assessming clinical outcomes: Patient satisfaction with pain management[j]. J Pain Symptom Manage, 1998, 16: [19] McNeill J A, Sherwood G D, Starck P L. The hidden error of mismanaged pain: a systems approach [J]. J Pain Symptom Manag, 2004, 28(1): [20] McNeill J A, Sherwood G D, Starck P L, et al. Pain management outcomes for hospitalized Hispanic patients [J]. Pain Manag Nurs, 2001, 2(1): [21] Nunnally JC, Bernstein IH: Psychometric Theory. New York, McGraw-Hill, [22] Panteli V, Patistea E. Assessing patients' satisfaction and intensity of pain as outcomes in the management of cacer-related pain [J]. European Journal of Oncology Nursing, 2007, 11: [23] Shen, Q, Li, Z. The reliability and validity for Houston Pain Outcome Instrument [in Chinese]. Chinese Journal of Nursing, 41, 2006, [24] Shen Q, Sherwood G D, Mcneill J A, et al. Postoperative pain management outcome in Chinese inpatients [J]. Western Journal of Nursing Research, 2008, 30(8): [25] Sherwood, G. D., McNeill, J. A., Hernandez, L., & Penarrieta, I. (2005). A multinational study of pain management among Hispanics: An evidence-based approach. Journal of Research in Nursing, 10, [26] Sherwood GD, McNeill JA, Palos G, Starck P. Perspectives on pain: A qualitative analysis of the Hispanic pain experience Nursing Times Research September : [27] Sherwood G D, Mcneill J A, Starck P L, et al. Changing acute pain management outcomes in surgical patients [J]. AORN J, 2003, 77(2): [28] Werner M U, Mjobo H N, Nielsen P R, et al. Prediction of postoperative pain: A Systematic review of predictive experimental pain studies [J]. Anesthesiology, 2010, 112(6): Laboratory and Clinical Investigation 291 FAM 2013 August Vol.20 Issue 4