PROMIS User Manual (Version 1.1)

Transcription

1 USER MANUAL PATIENT REPORTED OUTCOMES MEASUREMENT INFORMATION SYSTEM (PROMIS) Version. October, 008 COPYRIGHT NOTICE This is an unpublished manual. From October 5, 008 through March 5, 009, please cite as follows: PROMIS Cooperative Group. Unpublished Manual for the Patient Reported Outcomes Measurement Information System (PROMIS) Version.. October, 008: After March 5, 009, please refer to for proper citation(s) Revised on 0/4/008

2 TABLE OF CONTENTS I. Overview... 3 II. PROMIS Item Bank Development... 3 Statistical Evaluation of Archival Data Sets... 3 Qualitative Item Review... 4 Wave I Testing... 4 III. PROMIS INSTRUMENTS... 8 Available Formats... 8 PROMIS Domains... 8 Instrument Properties... Selecting the Right Instrument for your Study... 3 Instrument Statistics... 5 Scoring... Raw Score to T score Conversion Tables... IV. PROMIS Item and Instrument Copyright Statement and Users Agreement... 9 V. Assessment Center SM Summary Feature List URL... 3 Future Features... 3 VI. Validity and Interpretation Tables... 3 VII. References VIII. Appendices Appendix A: Patient Reported Outcomes Measurement Information System (PROMIS) Domain Framework... 6 Appendix B: COPYRIGHT NOTICE AND CLAIMS OF COPYRIGHT INFRINGEMENT... 6 Revised on 0/4/008

3 I. Overview In late 004, a group of outcomes scientists from seven institutions and the National Institutes of Health (NIH) formed a cooperative group funded under the NIH Roadmap for Medical Research Initiative to re engineer the clinical research enterprise ( This initiative the Patient Reported Outcomes Measurement Information System (PROMIS) aims to revolutionize the way patient reported outcome tools are selected and employed in clinical research and practice evaluation. It will also establish a national resource for accurate and efficient measurement of patient reported symptoms and other health outcomes in clinical practice. As part of the NIH Roadmap, NIH funded six primary research sites and a statistical coordinating center as the PROMIS cooperative group network. PROMIS aims to develop ways to measure patient reported symptoms, such as pain and fatigue, and aspects of health related quality of life across a wide variety of chronic diseases and conditions. "There is a pressing need to better quantify clinically important symptoms and outcomes that are now difficult to measure," said NIH Director Elias A. Zerhouni, MD. "Our clinical research communities would benefit greatly from efficient, consistent, well validated approaches to measuring these and other subjective outcomes." Clinical outcome measures, such as x rays and lab tests, have minimally immediate relevance to the day to day functioning of patients with chronic diseases such as arthritis, multiple sclerosis, and asthma, as well as chronic pain conditions. Often, the best way patients can judge the effectiveness of treatments is by perceived changes in symptoms. One main goal of the PROMIS initiative is to develop a set of publicly available computerized adaptive tests for the clinical research community. The NIH Roadmap is a series of far reaching initiatives designed to transform the nation's medical research capabilities and speed the movement of research discoveries from the bench to the bedside. It provides a framework of the priorities NIH must address to optimize its entire research portfolio, and it lays out a vision for a more efficient and productive system of medical research. For more information about the NIH Roadmap, please visit the Web site at For more information about PROMIS, visit The PROMIS User Manual provides information on the development of PROMIS instruments as well as instructions for their use. This includes () description of the development of the PROMIS instruments, () review of the properties of PROMIS instruments, (3) information about administration and scoring, and (4) discussion of validity and interpretation of PROMIS instruments. Information concerning the navigation of Assessment Center SM is available in the Assessment Center SM User Manual (see II. PROMIS Item Bank Development Statistical Evaluation of Archival Data Sets To maximize information already available about measuring PROs, the Statistical Coordinating Center (SCC) identified several extant data sets with self report data on the five PROMIS core domains: physical functioning, fatigue, pain, emotional distress, and social role participation. The SCC identified and reviewed large data sets that included patient reported outcomes and selected for initial Revised on 0/4/008 3

4 analysis. Working groups identified items representing the PROMIS domains. Items were then subjected to IRT analyses by PROMIS psychometricians. All results were reviewed collectively by the SCC analysis team and summaries were presented to the appropriate domain working group. Our primary goal was to use these archival data to better understand the dimensional structure of items that tap one of the five selected domains. Secondarily, we aimed to inform the revision of items in the item library, inform the identification of the most useful response sets, and guide new item construction in preparation for the first wave of PROMIS testing. These posted reports (see Data Analysis on summarize the contribution of archival data analyses to the production of PROMIS items for network testing. Qualitative Item Review One of the PROMIS network s primary goals is the development of a comprehensive item bank for patient reported outcomes of chronic diseases. For its first set of item banks, PROMIS chose to focus on pain, fatigue, emotional distress, physical function, and social function. An essential step for the development of an item pool is the identification, evaluation, and revision of extant questionnaire items for the core item pool. In this work we also describe the systematic process wherein items are classified for subsequent statistical processing by the PROMIS investigators. Six phases of item development are documented: identification of existing items, item classification and selection, item review and revision, focus group input on domain coverage, cognitive interviews with individual items, and final revision before field testing. Identification of items refers to the systematic search for existing items in currently available scales. Expert item review and revision was conducted by trained professionals who reviewed the wording of each item and revised as appropriate for conventions adopted by the PROMIS network. Focus groups were used to confirm domain definitions, and to identify new areas of item development for future PROMIS item banks. Cognitive interviews were used to examine individual items. Items successfully screened through this process were sent to field testing and will be subjected to innovative scale construction procedures (DeWalt et al, 007). Wave I Testing From July 006 to March 007, data were collected from the U.S. general population and multiple disease populations. The sampling plan was developed for collecting responses to the candidate items from the targeted PROMIS domains and was designed to accommodate multiple objectives: () create item calibrations for each domain; () estimate profile scores for various disease populations; (3) create linking metrics to legacy questionnaires (e.g., SF 36); (4) confirm the factor structure of the domains; and (5) conduct item and bank analyses. Because of the large total number of items (> 000), it was not possible for participants to respond to the entire pool. Based on an estimate of 4 questions per minute, the number of items administered to any respondent was limited to about 50 (37 minutes). Two data collection designs ( full bank and block administration ) were used. Some individuals were administered full banks of 56 items for a subset of the PROMIS wave domains while others were administered 4 blocks of 7 items selected from the 4 item banks (3 physical functioning banks, anxiety, depression, anger, substance abuse, fatigue impact, fatigue experience, social role performance, social role satisfaction, pain interference, pain quality, pain behavior). In addition to the PROMIS items and appropriate legacy items (items from widely used fixed length measures such as the SF 36) completed by those administered full banks of items, participants were administered approximately auxiliary items consisting of global health rating items and sociodemographic Revised on 0/4/008 4

5 variables including age, income, number of hospitalizations, disability days, whether they take prescribed medicines, body mass index, gender, race/ethnicity, whether the person was married or living with someone, educational attainment, and whether the person was working full time, a full time student, or not working full time or going to school full time. There were also a series of questions about the presence and degree of limitations related to 5 chronic medical conditions: hypertension, angina, CAD, heart failure, heart attack, stroke or TIA, liver disease, kidney disease, arthritis or rheumatism, osteoarthritis, migraines, asthma, COPD, diabetes, cancer, depression, anxiety, alcohol or drug problems, sleep disorder, HIV/AIDS, spinal cord injury, multiple sclerosis, Parkinson s disease, epilepsy, and amyotrophic lateral sclerosis. The full bank administration allows for evaluation of dimensionality and calibration within item banks (domains). The block administration permits an evaluation of associations between domains. The ability to calibrate items on a common metric (item linking) is possible because of the administration of blocks of items from the full bank to general population and clinical samples. Each item was administered to at least 900 respondents from the general population and 500 respondents with a chronic medical condition. Data were collected primarily by YouGovPolimetrix ( also see a polling firm based in Palo Alto, CA. YouGovPolimetrix operates PollingPoint.com, a centralized portal that allows interested individuals to provide their views about public policy and other current issues. The respondents for a typical YouGovPolimetrix Internet survey are selected from the PollingPoint panel, a panel of over one million respondents who have provided YouGovPolimetrix with their names, physical addresses, addresses, and other information, and who regularly participate in online surveys. YouGovPolimetrix uses a sample matching procedure to select a representative sample when a listing of the population is available but it is difficult to draw a random sample from the population frame. The sample matching methodology starts with a listing of all respondents in the desired or target population. Next, a random sample of the desired size is selected from the population listing (the target sample ). Third, for each element of the target sample, the closest match is selected from the PollingPoint panel. This method has been shown to give accurate results in a wide variety of applications, even for groups significantly underrepresented on the Internet. Its validity depends upon the panel of available respondents being sufficiently large and diverse, not upon Internet usage or other types of behavior. For PROMIS, we specified targets in terms of gender (50% female), age (0% in each of 5 age groups: 8 9, 30 44, 45 59, 60 74, 75+), race/ethnicity (0% black and Hispanic), and education (0% less than high school graduate). Panelists have been recruited by a variety of methods, such as e random digit dialing, invitations via web newsletters, and Internet poll based recruitment where panelists have opted to participate in a survey advertised on the World Wide Web. The topics of these surveys vary greatly from politics to polls on popular entertainment. All of the PollingPoint panelists have provided their e mail so that they may receive survey invitations to participate in future surveys. Panel members receive minimal compensation. This methodology was developed by Stanford Political Science Professor Douglas Rivers and was recently evaluated using the 005 California Special Election. The matched samples performed as well or better than conventional random digit dial (RDD) telephone surveys, correctly predicting the outcomes for all of the propositions and having an average absolute error of 3 percent (consistent with normal sampling error). Revised on 0/4/008 5

6 In general, YouGovPolimetrix has reported a base response rate of 67% for general population surveys (similar to the general population component of the PROMIS Wave I data collection) and rates at or exceeding 80% with more specialized, clinical samples such as the ones in PROMIS. They also utilize sample matching methodologies for selection of representative samples from nonrandomly selected pools of respondents, which is ideally suited for Web access panels. The PROMIS wave sample included,33 respondents, with,53 recruited from primary research sites associated with PROMIS network sites and the vast majority (9,60) from YouGovPolimetrix s panel sample. YouGovPolimetrix sample data was collected using their website on a secure server. The PROMIS network site data was collection using the PROMIS Assessment System. The Statistical Coordinating Center based at CORE received de identified datasets from YouGovPolimetrix. Each of the 7,005 individuals (6,676 from YouGovPolimetrix, 36 from UNC, and 93 from Stanford) assigned to administration of full item banks was administered a pair of the 4 item banks. The 7 item blocks for each of the 4 banks were administered to 4,8 individuals (6,45 general population, 7,883 clinical samples). The clinical samples included persons with heart disease (n =,56), cancer (n =,754), rheumatoid arthritis (n = 557), osteoarthritis (n = 98), psychiatric illness (n =,93), chronic obstructive pulmonary disease (n =,4), spinal cord injury (n = 53), and other conditions (n = 560). The clinical population supplied by YouGovPolimetrix was identified through a pre survey of 50,000 YouGovPolimetrix panel members. These respondents were asked to complete the PROMIS clinical form. Persons who self reported currently being diagnosed with a given condition were eligible for inclusion in the Wave I clinical sample associated with that condition. It should be noted that the general population sample included people with and without diseases (general population respondents were also administered the clinical form but their responses did not impact their participation in the general population sample). Overall Sample The overall sample (n =,33) was 5% female. The median age was approximately 50 years with % 8 9, % 30 39, 6% 40 49, 3% 50 64, and 8% 65 and greater years old. Eighty two percent were white, 9% Black, 8% multi racial, and % other (Asian/Pacific Islanders and Native Americans). Nine percent of the sample was Hispanic or Latino. Educational attainment ranged from less than high school (3%) to advanced degree (9%) with 4% with a college degree, 39% some college, and 6% a high school diploma. The overall sample was used primarily for calibrating item parameters. Scale Setting Sub Sample The distribution of gender, age, race, and education in the 000 census was: 5% female; % 8 9, 3% 30 44, 4% 45 59, 4% and 8% 75 and greater years old; 74% White, % Black, % Hispanic/Latino, and 4% Other; and 5% more than high school. A subset of the PROMIS general population sample (includes respondents administered full bank and item blocks) was created with the purpose of representing the marginal distributions of race/ethnicity (White, Black/Hispanic/Other) and education (High School or less versus more than high school) in the census. The distribution of characteristics for the PROMIS scale setting sub sample (n = 5,39) was: 57% female; 5% 8 9, % 30 44, 8% 45 59, % and 3% 75 and greater years old; 74% White, 0% Black, % Hispanic/Latino, and 4% Other; 5% had more than a high school education. The scale setting subsample was primarily used to establish U.S. population norms. Revised on 0/4/008 6

7 Figure: PROMIS Wave Sample Site (n=39) Demo. short form Global health General population (n=39) UNC=36 Full Bank Testing (N=7,005) Forms A G Legacy Polimetrix (n=6,676) Stanford=93 Demo. short form Global health General population (n=6,676) Forms A G Legacy Co morbidity Site (n=,03) General population (n=400) UNC=304 Stanford=96 Block Testing (N=4,8) Demo. extended form Global health Forms H W Co morbidity Polimetrix (n=,95) Demo. extended form Global health Forms H W Co morbidity Clinical sample (n=803) General population (n=5,845) Clinical sample (n=7,080) Pittsburgh=5 Stanford=50 Duke=40 Additional Chronic Pain Sample In order to increase the number of participants experiencing pain in the calibration sample, individuals with chronic pain were recruited through the American Chronic Pain Association (ACPA). An invitation to complete the PROMIS pain survey was posted on the ACPA website. To be eligible, participants had to be years of age or older and have at least one chronic pain condition for at least 3 months prior to participating in the survey. Those who met eligibility criteria were asked to complete an online informed consent form. After obtaining informed consent, participants immediately began the survey. The survey was posted on the website of the ACPA from September 007 to March 008. There were 967 participants who responded to 47 pain impact, 4 pain behavior, and 4 pain quality items, and one global average pain intensity item through online administration. The average age was 48. years (SD =.). Eighty one percent of the respondents were female, and 9% were Caucasian,.5% were African American, and 5% were of Hispanic origin. Eighty one percent of the participants had an education equal to or greater than high school. The data was combined with Wave full bank test data to calculate item calibrations. Revised on 0/4/008 7

8 Sleep and Wake Disturbance Sample The sample for the Sleep and Wake Disturbance banks included administration of 8 Sleep Wake Functioning items to,993 individuals from YouGovPolimetrix (,59 from general population, and 734 with self identified sleep problem) and 59 individuals with sleep disorders from Pittsburgh clinical sites. The overall sample (n =, 5) was 43.8% female. The median age was 5 years old with 0.7% 65 and older and 79.3% less than 65 years old. Eighty two percent were White,.6% Black,.7% Native American or Alaskan, 0.4% Native Hawaiian or Pacific Islander, and 5.9% other. Ten percent of the sample was Hispanic or Latino. Educational attainment ranged from high school or less (3.6%), some college (38.6%), college degree (7.9%) to advanced degree (9.9%). Item response data from the overall sample (, 5 individuals) were used for item calibration. III. PROMIS INSTRUMENTS Available Formats PROMIS instruments derive primarily from calibrated item banks that enable computerized adaptive testing and multiple short forms of varying length. The following instruments are available as of April, 008. All instruments can be accessed within Assessment Center SM ( Item Bank Short Form Domain # of Items # of Items Emotional Distress Anger 9 8 Emotional Distress Anxiety 9 7 Emotional Distress Depression 8 8 Fatigue 95 7 Pain Behavior 39 7 Pain Impact 4 6 Physical Function 5 0 Satisfaction with Discretionary Social Activities 7 Satisfaction with Social Roles 4 7 Sleep Disturbance 7 8 Wake Disturbance 6 8 Global Health 0 PROMIS Domains PROMIS investigators developed a domain framework for self reported health (see Appendix A). Domain definitions were created for global health, physical function, fatigue, pain, emotional distress (including depression, anxiety, and anger), and social health (social function and social support). Global Health Global health refers to evaluations of health in general rather than specific elements of health. The global health items include global ratings of the five primary PROMIS domains (physical function, fatigue, pain, emotional distress, social health) and general health perceptions that cut across Revised on 0/4/008 8

9 domains. Global items allows respondents to weight together different aspects of health to arrive at a bottom line indicator of their health status. Global health items have been found to be consistently predictive of important future events such as health care utilization and mortality. The PROMIS global health items include the most widely used single self rated health item (global0). Previous research has shown that the former item taps physical health and mental health about equally but it reflects physical health more than mental health, especially for those with low less of income (Bjorner et al, 005). PROMIS includes a single item that provides a pure rating of physical health (global03) and another item for mental health (global04). Also included is an overall quality of life item (global0) that is a good indicator of mental health (Lorenz et al, 006). The remaining items provide global ratings of physical function (global06), fatigue (global08), pain (global07), emotional distress (global0), and social health (global05 and global09). Physical Health Physical Function Physical function is defined as one's ability to carry out various activities that require physical capability, ranging from self care (activities of daily living) to more vigorous activities that require increasing degrees of mobility, strength, or endurance (Haley et al, 994a; 994b; Stewart & Kamberg, 99; Wilson & Cleary, 995). Physical function items, when considered as an outcome endpoint for clinical research in chronic illness, generally have a "capability" stem and a corresponding "capability" set of response items (e.g., "Are you able to...normally, with some difficulty, with moderate difficulty, with great difficulty, unable to do"), and are given in the present tense. This specifically excludes some items that may have great utility in other settings, as with "performance" items that ask whether an activity was actually conducted during a specified time frame (with a "Did you?" type of stem). Such items require capability but also opportunity and motivation. The use of capability stems also excludes the concept of satisfaction (e.g., "How satisfied are you with your current level of function?"). Such questions address subjective appraisals of oneself that incorporate concepts such as coping or adjustment. Because many persons with a chronic disease will have more than one chronic condition and cannot distinguish the fraction of a problem attributable to each one, physical function items attempt to quantitate the sum of these effects, leaving the teasing out of relative contributions to the analysis stage. Physical function is conceptually multidimensional, with four related subdomains: mobility (lower extremity function), dexterity (upper extremity function), axial (neck and back function), and ability to carry out instrumental activities of daily living (IADL). Symptoms Fatigue at its highest level is defined as an overwhelming, debilitating, and sustained sense of exhaustion that decreases one's ability to carry out daily activities, including the ability to work effectively and to function at one's usual level in family or social roles (Glaus, 998; North American Nursing Diagnostic Association, 996; Stewart, Hays & Ware, 99). Similar subjective feelings, yet fewer behavioral impacts, are associated with lower levels of fatigue. Fatigue is divided conceptually into the experience of fatigue (such as its intensity, frequency and duration), and the impact of fatigue upon physical, mental and social activities. Pain is an unpleasant sensory and emotional experience associated with actual or potential tissue damage, or described in terms of such damage (Chang, 999; Mersky & Bogduk, 994; Meuser et al, 00; Sherbourne, 99). Pain is what the patient says it is that is, the "gold standard" of pain assessment is self report. Pain is divided conceptually into components of quality (referring to the Revised on 0/4/008 9

10 nature, characteristics, intensity frequency and duration of pain), impact upon physical, mental and social activities, and behaviors one engages in to avoid, minimize or reduce pain. Sleep and Wake Disturbance Sleep and wakefulness are the two fundamental behavioral states of human beings. Sleep is a rapidly reversible, recurrent state of reduced (but not absent) awareness of and interaction with the environment. Wakefulness is a behavioral state of active engagement and interaction with the environment, including the perception and processing of stimuli and the production of cognitive, emotional, and behavioral responses. Sleep and wakefulness are both distinct from abnormal behavioral states such as delirium or coma. The generation of sleep and wakefulness is an endogenous phenomenon which is regulated by homeostatic and circadian physiological processes, but which can be influenced by internal (e.g., cognitive, emotional) and external (e.g., physical, environmental) stimuli. A considerable body of scientific data describes the neuroanatomy and neurophysiology of sleep and wakefulness. While the precise functions of sleep remain to be identified, there is little doubt that sleep is necessary for optimal mental and physical function during wakefulness. Alterations in the amount or quality of sleep have been associated with impaired alertness, cognitive and emotional function, and learning; disordered function of the central nervous system, cardiovascular, endocrine metabolic, and immune systems; and even with increased mortality. As fundamental behavioral and brain states, sleep and wakefulness can be described at several levels of organization, including the activity of individual cells, neural systems, or the entire organism. Methods of measuring sleep at the organismic level in humans include physiological recordings, functional neuroanatomic studies, and patient reported outcomes (PROs). The PROMIS Sleep Disturbance and Wake Disturbance scales are examples of the latter. Multiple types of assessments are possible within the broad domain of sleep wake PROs. For instance, some self report assessments are used to diagnose specific sleep disorders; others (sleep wake diaries or logs) are used to assess habitual sleep wake quantities and patterns; and still others measure an individual s perceptions of the quality and global experience of sleep and wakefulness. The PROMIS Sleep Disturbance and Wake Disturbance scales fall into the latter category. Both scales assess function and disturbances over a seven day time frame. Sleep Disturbances: The PROMIS Sleep Disturbances Scale focuses on perceptions of sleep quality, sleep depth, and restoration associated with sleep; perceived difficulties with getting to sleep or staying asleep; and perceptions of the adequacy of and satisfaction with sleep. The Sleep Disturbances Scale does not include symptoms of specific sleep disorders, nor does it provide subjective estimates of sleep quantities (e.g., the total amount of sleep, time to fall asleep, or amount of wakefulness during sleep). Wake Disturbances: The PROMIS Wake Disturbances Scale focuses on perceptions of alertness, sleepiness, and tiredness during usual waking hours; and on functional impairments during wakefulness that are associated with sleep problems or impaired alertness. The Wake Disturbances Scale does not directly assess cognitive, affective, or performance impairments. The Wake Disturbances scale measures the level of waking alertness, sleepiness, and function within the context of overall sleep wake function. Revised on 0/4/008 0

11 Mental Health Emotional Health Emotional distress commonly refers to unpleasant feelings or emotions that are experienced privately and, therefore, are good candidates for assessment as patient reported outcomes. Emotional distress is comprised typically of aspects of anxiety, depression, and anger. Anxiety, depression, and anger represent risk factors that have been associated with both the incidence and progression of disease. The mechanisms by which these associations arise are not well understood, but they can be organized into two general families: direct effects via physiological pathways (e.g., the association between depression and risk factors for cardiovascular disease such as blood lipids and inflammation, which may be produced by shared causal variables) and indirect effects via the impact on health related behaviors (e.g., increased use of tobacco and alcohol as a consequence of negative emotions). Given the overlap among the symptoms of anxiety, depression, and anger, a number of conceptual models have been proposed to account for the shared versus unique variance captured in measures of negative affect. PROMIS adopted a hierarchical structure to explain the relationships between self reported symptoms of anxiety, depression, and anger (Clark & Watson, 99; Watson & Clark; 99). This structure includes a second order, nonspecific factor reflecting high levels of negative affect or general distress common to all these emotions. Anger tends to have smaller loadings on the general factor than anxiety and depression, but it still is a strong marker of the dimension. In addition, Watson and Clark s model included first order factors that are specific to, and help to differentiate, these three affects. Depression: Symptoms specific to depression are those that reflect low levels of positive affect. In addition, depression is often characterized by the experience of loss and feelings of hopelessness, helplessness, and worthlessness. The PROMIS item bank for depression focuses on negative mood (e.g., sadness, guilt), decrease in positive affect (e.g., loss of interest), information processing deficits (e.g., problems in decision making), negative views of the self (e.g., self criticism, worthlessness), and negative social cognition (e.g., loneliness, interpersonal alienation). Anxiety: Symptoms that best differentiate anxiety are those that reflect autonomic arousal and the experience of threat. The PROMIS item bank for anxiety focuses on fear (e.g., fearfulness, feelings of panic), anxious misery (e.g., worry, dread), hyperarousal (e.g., tension, nervousness, restlessness), and somatic symptoms related to arousal (e.g., cardiovascular symptoms, dizziness). Anger: Anger is distinguished by attitudes of hostility and cynicism and is often associated with experiences of frustration impeding goal directed behavior. Specific components relate to verbal and nonverbal evidence of interpersonal antagonism. The PROMIS item bank for anger focuses on angry mood (e.g., irritability, reactivity), negative social cognition (e.g., interpersonal sensitivity, envy, vengefulness), verbal aggression, and efforts necessary to control angry mood. In general, our PROMIS item banks emphasize the cognitive and affective components of these concepts. Both psychometric considerations (e.g., skewed distributions for high threshold behavioral items, the need to fit item response theory models to coherent unidimensional concepts) and considerations regarding validity (e.g., potential confounding between somatic symptoms of emotional distress and markers of physical disease) have led us to this emphasis. Revised on 0/4/008

12 Social Health PROMIS User Manual (Version.) Social health is defined as perceived well being regarding social activities and relationships, including the ability to relate to individuals, groups, communities and society as a whole. The term social health is used here synonymously with "social function" and refers to a higher order domain, with measurable subdomains. Components of social functioning include understanding and communication, getting along with people, participation in society and performance of social roles. Additional conceptualizations of social functioning focus on the quality, reciprocity and size of an individual s social network (Beels et al, 984; Brekke, Long & Kay, 00). Although social role participation was the initial focus of PROMIS investigation, several other aspects of social health are noteworthy. These include social support and interpersonal attributes independent of particular roles, such as intimacy, assertiveness, sociability, submissiveness and interpersonal control (Horowitz et al, 988). The two broad patient reported outcomes under social function within the PROMIS framework are Social Function and Social Support. Social Function Social function is defined by PROMIS as involvement in, and satisfaction with, one s usual social roles in life s situations and activities. These roles may exist in marital relationships, parental responsibilities, work responsibilities and social activities (Dijkers, Whiteneck & El Jaroudi, 000; McDowell & Newell, 996). Social function has also been referred to with terms such as role participation and social adjustment (McDowell & Newell, 996). Qualitative and quantitative analysis of PROMIS data collected from have resulted in a conceptual division of social function into ability to participate and satisfaction with participation. Each of these two components has sub components that divide social roles such as work and family responsibilities, and more discretionary social activities such as leisure activity and relationships with friends. Social Support There are two broad types of social support: quantitative and qualitative (Cohen & Syme, 985; Wills, 985; Wortman & Conway, 985). Quantitative social support refers to the existence of, and interconnections between, social ties, e.g., marital status, number of relationships, frequency of contacts with friends and relatives, church membership, and volunteer participation. Qualitative social support refers to functional aspects of supportive relationships, i.e., interpersonal relationships that serve particular functions. This includes the interactive process by which emotional, instrumental, informational or motivational support is obtained from one's social network (Bowling & Farquhar, 99). It also includes feeling cared for and valued as a person, communication with others, and feelings of belonging and trust (Larson, 993, McDowell & Newell, 996). Measures of social support generally seek information about a person s perception of the availability or adequacy of resources provided by other persons (Cohen & Syme, 985). In this context, perceived social support is a subdomain of social health. Instrument Properties Recall Period Most PROMIS item banks utilize a 7 day recall period. The past 7 days is the reference period for all items in Anxiety, Anger, Depression, Fatigue, Pain Behavior, Pain Impact, Satisfaction with Discretionary Social Activities, and Satisfaction with Social Roles. Physical function items emphasize current capabilities and therefore do not employ a recall period. Item stems begin with phrases such as Does your health now limit you or Are you able to. Some global health items use a 7 day recall period while others do not employ a recall period and emphasize current status in general. Revised on 0/4/008

13 Response Options PROMIS User Manual (Version.) The term response options refers to the set of answers a respondent can choose from when responding to a question or statement. Virtually all PROMIS items employ five response options (e.g., =Not at all, =A little bit, 3=Somewhat, 4=Quite a bit, 5=Very much). Pain Behavior uses six response options to allow for respondents to endorse no pain. The 0 PROMIS Global Health items each have 5 response choices, with the exception of the common point pain intensity item ( How would you rate your pain on average with 0=No pain and 0=Worst imaginable pain). To the extent possible, the wording of response categories was kept consistent within banks, and a limited degree of variation in response options was used across banks. This was done to ease patient burden. An effort was made for items within a PROMIS domain to utilize the same or a small number of response options. Some flexibility in response choices within banks was considered important, however, to capture the range of patient experience in a domain (e.g., intensity, frequency, duration). Therefore, for example, most banks employed a common set of response options for intensity (i.e., Not at all to Very much ) and frequency (i.e., Never to Always ). The final response categories were pre tested with cognitive interviews to confirm patient comprehension, prior to field testing for item calibration. When possible, to ease responder burden, initial short forms developed from PROMIS item banks comprise items with a common set of response options. Exceptions are the 0 Global Health items which utilize 5 unique response options and the Physical Function short form which uses response options. Notes on Administration PROMIS Version.0 instruments were developed based on data collected on an internet survey platform. As such, it can safely be considered valid for internet or personal computer based applications with screen presentations of individual items. Comparability of results obtained using paper or telephone administration cannot be assumed until further testing is done. On average, respondents will answer 5 questions per minute, suggesting, for example, that a CAT administration of all 9 banks with an average of 5 items per bank will take about 0 minutes to complete. Populations As noted in the Wave Testing section, a sample reflecting Year 000 census demographics data was used for scale setting. Many participants with chronic disease (e.g., cancer, heart disease, arthritis, COPD, spinal cord injury, psychiatric conditions) were included. Selecting the Right Instrument for your Study There are two general options to consider when selecting a PROMIS instrument for your study: CAT or static short forms. Within these two general options are several specific considerations to guide your optimal selection. In all cases, when you create an assessment from a PROMIS bank, a score will be produced on the same common (Theta) metric which has been converted to a T distribution based on the United States general population. The choice you make for assessment in your study should be driven by your relative interests in precision, brevity, item content, and flexibility/portability. Precision Precision, the conceptual inverse of error, is increased by adding questions from the same item bank. Two important things to keep in mind are that the amount of increase in precision gained by adding a question decreases as the number of questions increases. Therefore, a 6 item scale is much more Revised on 0/4/008 3

14 precise than a item scale, but the increase in precision gained by adding yet another 5 questions to create an item scale is not nearly as much as going from to 6 questions. The second thing to keep in mind is that not all questions in a bank are equally informative, so it matters which question you add. This is the reason that CAT assessment will virtually always be more precise than a fixed short form of the same length. PROMIS banks using CAT can achieve precision that meets standards for individual level assessment with fewer than 6 questions on average. If precision of assessment is your main goal (such as might be the case in tracking an individual person over time to detect reliable change), then CAT (or a static short form of 0 or more items) would be your preferred choice. Brevity Both CAT and static short forms can be brief. CAT will out perform a static short form of the same length, so if brevity and precision are desired, CAT would be the better choice. However, often brevity is desired in settings where CAT is not possible or even desired. Some applications (e.g., large sample studies seeking population estimates; large sample clinical trials that plan group comparisons) do not require the precision offered by CAT or lengthy short forms. In that case, careful selection of a small number of questions per bank (even one question per bank) will produce T scores that can still be referenced to the general population sample. Although confidence in the individual score estimates derived from very short forms is low, large group averages are reliable. Using item statistics in this manual and on Assessment Center, customized short forms of any length can be created from the PROMIS item banks. We have created sample short forms for each bank, ranging in length from 6 0 items per form. Items selected for these short forms cover the measurement range with some of the more informative questions. Each of them is suited for individual assessment in the middle range of the trait being measured. Extremes on the measurement continuum (e.g., very little fatigue or extreme fatigue) are less reliably estimated. Briefer short forms can be custom made. Item Content Sometimes a researcher will prefer to determine which questions in a bank are administered. Reasons for this preference can be the clinical relevance of a desired subset of items, or the lack of relevance of a subset of questions in a given target research population. Similarly, some researchers may wish to ensure that the same questions get administered at every time point in a longitudinal design. In these cases, CAT would not be desirable. The strength of CAT is in its flexibility with regard to selection and sequence of specific questions asked in any given assessment. Generally, then, only one question in the bank (the first one) administered by the CAT engine would be guaranteed to be repeated at each assessment. In these cases (desire to determine which questions get asked or desire to ask the same questions at each administration), static short forms would be the preferred option. CAT options do exist, however, to balance content of items administered from a given bank, which is important to remember before opting out of the CAT approach. Flexibility/Portability For all practical purposes, CAT administration depends upon access to computer administration (either web based or standalone computer). Although options exist for branched assessment that approximates CAT on paper, these are not currently available for PROMIS banks. Thus, any research project that does not have the capability to electronically enter participant responses in real time should select paper (or telephone) administration of static short forms. Revised on 0/4/008 4

15 Instrument Statistics Emotional Distress Anger Item Bank v.0 Emotional Distress Anxiety Item Bank v.0 Revised on 0/4/008 5

16 Emotional Distress Depression Item Bank v.0 Fatigue Item Bank v.0 Revised on 0/4/008 6

17 Pain Behavior Item Bank v.0 PROMIS User Manual (Version.) Pain Impact Item Bank v.0 Revised on 0/4/008 7

18 Physical Function Item Bank v.0 PROMIS User Manual (Version.) Satisfaction with Discretionary Social Activities Item Bank v.0 Revised on 0/4/008 8

19 Satisfaction with Social Roles Item Bank v.0 Sleep Disturbance Item Bank v.0 Revised on 0/4/008 9

20 Wake Disturbance Item Bank v.0 PROMIS User Manual (Version.) Revised on 0/4/008 0

21 Scoring Short Forms Static short forms administered online can be scored using the same numerical scoring algorithm implemented in the PROMIS software (see the bullet on CAT in this section). The item response theory (IRT) model adopted in PROMIS (i.e., the graded response model) uses the pattern of item responses for scoring, not just the sum of individual item responses. Under paper administration of static short forms, however, it is inevitable to streamline the process and use some form of lookup tables based on summed scores, because the number of different response patterns will become unmanageably large to present in a lookup table as the test length increases beyond two or three items. Therefore, for a given collection of IRT calibrated items, we estimate the conditional distribution of each summed score as a function of the trait continuum. Taking into account the score distribution in the population of examinees from which our standardization sample was drawn, we then determine the optimal trait level for each summed score and populate a scoring table. A scoring table can be prepared in advance to map all possible summed scores to trait scores in one to one correspondence. Upon administering a static short form, the test administrator can add up the individual item responses reported by the examinee and map the summed score to a trait score using the scoring table provided for the form. All PROMIS short forms are designed to be scored from to 5 (or higher depending on the number of response options). For instance, the PROMIS Depression short form is composed of 8 items and each item has five response options (i.e., Never, Rarely, Sometimes, Often, and Always). Thus, the minimum possible raw summed score for the form is 8 (8 times ) and the maximum 40 (8 times 5). The scoring table for the form lists all possible raw summed scores ranging from 8 to 40 and the corresponding trait scores and standard errors. Because a scoring table is prepared for a fixed set of items, it can only be used when an examinee responds to all of the items in the set. One or more missing responses will render such scoring tables unusable. CAT Technical Information In computer adaptive testing (CAT), the trait score is updated as each new item is administered, and typically the estimate at the conclusion of a CAT is reported as the final trait score. The trait estimation method employed in the PROMIS CAT software is the Expected A Posteriori (EAP) estimator. Although several different trait estimation methods are available, EAP has been a popular choice for CAT. Unlike the Maximum Likelihood (ML) estimation, another popular approach, EAP does not rely on an iterative procedure and hence has computational advantages under adaptive testing. Another advantage of EAP over ML is that it provides finite estimates for the maximum and minimum possible raw scores (e.g.,,,,, and 5,5,,5). Thus, EAP can be useful especially in the early stages of a CAT where all responses to first few items can be extreme in the same direction, for which ML does not provide a finite estimate and hence requires updating the interim trait estimate arbitrarily in order to select the next item. Another advantage of EAP over ML is that it provides smaller squared errors on average at the expense of slightly shrinking trait estimates towards the center of the presumed trait distribution, which is known as the prior distribution. We employ a normal prior with a mean of 50 and a standard deviation of 0. All PROMIS banks are scaled to have a mean of 50 and a standard deviation 0 based on the standardization sample. The first item is selected to maximize the expected information over the prior distribution. That is, the first item selected is that would provide the largest amount of information on average for examinees drawn from the prior distribution. The prior distribution is then updated into a posterior distribution in conjunction with the response probability distribution of the first item. The next item will be selected to maximize the expected information over the posterior distribution. As additional items are administered, the variance of the posterior Revised on 0/4/008

22 distribution becomes progressively smaller. That is, we become more and more confident in the trait level of the examinee. The CAT stops when the standard deviation of the posterior distribution, which is a common measure of standard error under EAP, becomes smaller than a threshold value (e.g., 0.3 on the theta metric or 3 on the T score metric), and the mean (i.e., the expected value) of the posterior distribution is reported as the final trait score. The standard error after administering item is lower than one may expect. There are two reasons for this. First, PROMIS items have very high discrimination parameters. Second, the EAP trait estimator takes into account the prior distribution. EAP provides the smallest mean standard errors at the cost of regressing the trait scores toward the mean of the prior distribution. As the fist item selected is the most informative item for the most people, a small SE will be calculated if a nonextreme response option is selected. After five or six items the SE should be in the.30s for many people. Raw Score to T score Conversion Tables The following conversion tables allow a user to convert simple summed raw scores from PROMIS short forms into T score values on an individual respondent or group of respondents. In all cases, these conversions only work accurately when all questions on the short form have been answered. T score distributions are standardized such that a 50 represents the average (mean) for the US general population, and the standard deviation around that mean is 0 points. Thus, a person who has a T score of 60 lies one standard deviation higher than the average of the general population for the concept being measured. A high score always represents more of the concept being measured. Thus, for symptoms and other negatively worded concepts like pain, fatigue and anxiety, a 60 is one standard deviation worse than average. For function scores and other positively worded concepts like physical or social function, a 60 is one standard deviation better than average. The T score associated with each summed score was computed as the expected value (mean) of the sum of the likelihoods of all combinations of responses producing the given summed score. A recursive algorithm described in Lord and Wingersky (984) was used to obtain the likelihoods, which were then combined with the normal population prior distribution to compute the posterior distributions. Finally, the T Scores and associated standard errors (SEs) were computed as the expected values and standard deviations of the posterior distributions. Revised on 0/4/008