Using WHOQOL in NZ mental health and addiction services

Transcription

1 Using WHOQOL in NZ mental health and addiction services 5 th Australasian Mental Health and Outcomes Information Conference, Queenstown, 11 November 2015

2 Our names Rex Billington AUT Chris Krägeloh AUT Melissa Rowthorn Connect Sarah Andrews Emerge Aatearoa Ross Phillips Pathways Jillian Pennington Wild Bamboo Jessica Gu AUT Your names

3 Structure of the workshop: Background to the WHOQOL-BREF (Rex) Use in mental health outcomes assessment (Chris) Do the test and score it (All) Analyse the results (Rex and Chris) Implementation strategies in NGOs (Melissa, Sarah & Ross) Storing and presenting scores (Jillian. Sarah) Ground rules for use and the future in modifying the tool in mental health recovery organisations (Rex) Summary and contacts (Chris)

4 Activities of the NZ WHOQOL Group Collaboration is among AUT University, Mental Health Recovery NGOs, some DHBs and Platform Trust including Te Pou, Wild Bamboo and Lantern. We provide general support and advice on the use of the tools; provide general guidelines on scoring and interpretation. We monitor the use of the WHOQOL tools in NZ to avoid misuse and keep a current data base of users and collaborate on improvisation.

5 1. The original development of the WHOQOL tools and their purpose: WHOQOL = World Health Organisation Quality of Life Scale. Health-related QOL assessment. Developed in the 1990s cross-culturally in 14 countries. Expanded to over 35 country versions since estimate of 123 researchers in 67 centres and 39 countries involved in studies using WHOQOL instruments.

6 Rationale behind WHOQOL development Need for measurement of health beyond traditional morbidity and mortality to include impact of disease and impairment on daily activities and behaviour. Desire to find out what patients and clients felt about themselves to supplement what experts thought. Introduces a humanistic element to health care to balance mechanistic medical approaches and that positively affects clinican-patient relationship. Unsatisfactory and culturally biased translations of similar UK and North American measures.

7 The World Health Organisation has defined health as: A state of complete physical, mental and social wellbeing not merely the absence of disease and infirmity. Thus the measurement of health and the effects of health care must include not only an indication of changes in the frequency and severity of illness but also an estimation of well being.

8 Definition of QOL individuals perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns. It is a broad ranging concept affected in a complex way by the person s physical health, psychological state, personal beliefs, social relationships and their relationship to salient features of their environment (WHOQOL Group, 1995, p.1405). People can be satisfied with their QOL despite ill health or disability. And regain QOL after an accident.

9 Developmental process Experts meeting Geneva multifaceted scale agreed. Recruited 15 collaborating centres who conducted country-level focus group with consumers, patients and health experts to examine domains and develop facets ideas returned to Geneva and sorted. Next collaborating centres generated 2,500 questions. Translated/back-translated. Again sorted in Geneva.

10 Developmental process items and 29 facets in the first international trial to examine construct validity of domains and facets and to select best items. 19 centres involved in the field testing of the WHOQOL with 8,294 respondents involved. Selection of 24 facets on basis of their importance and 100 items on the basis of item analysis data.

11 Properties of the WHOQOL Comprehensive and holistic multidimensional profile. Subjective approach: Have you enough money to meet your needs? Objective approach How much money do you have? Cross-culturally developed with many languages. Standardised 5-point Likert rating scale covering 4 dimensions - intensity (how much), frequency (how often), evaluation (how satisfied), capacity (are you able).

12 Uses of the WHOQOL Medical practice Medical audit Clinical trials Research Policy making

13 Versions and features WHOQOL-100 = core generic instrument WHOQOL-BREF = abbreviated 26-item version. Additional modules = HIV, OLD, SRPB, DIS Some country versions have optional additional national items (including New Zealand with 5).

14 Domain WHOQOL-100 Domains and Facets Facets incorporated within domains Overall Quality of Life and General Health 1. Physical Health Energy and fatigue Pain and discomfort Sleep and rest 2. Psychological Bodily image and appearance Negative feelings Positive feelings Self-esteem Thinking, learning, memory and concentration 3. Level of Independence Mobility Activities of daily living Dependence on medicinal 4 items per facet substances and medical aids Work capacity

15 WHOQOL-100 Domains and Facets 4. Social Relations Personal relationships Social support Sexual activity 5. Environment Financial resources Physical safety and security Health and social care: availability and quality Home environment Opportunities for acquiring new information and skills Participation in and opportunities for 4 items per facet recreation / leisure Physical environment (pollution/ noise / traffic/ climate) Transport 6. Spirituality/Religion/ Religion / Spirituality / Personal beliefs Personal Beliefs (Single facet)

16 The WHOQOL-BREF QOL profile: Physical health Environment pain and discomfort dependence on medicinal substances and medical aids energy and fatigue mobility sleep and rest activities of daily living work capacity Psychological health positive affect spirituality thinking, learning, memory and concentration bodily image and appearance self esteem negative affect physical safety and security physical environment financial resources opportunities for acquiring new information and skills recreation and leisure activities home environment health and social care transportation Social relationships personal relationships sexual activity social support 1 item per facet

17 Originally studies in New Zealand using the WHOQOL- BREF used the Australian or English versions. In 2008, Prof. Rex Billington and colleagues founded the New Zealand WHOQOL Group. One of the early goals of this group was to validate the 26 item BREF instrument for use in New Zealand and thus develop a New Zealand version.

18 Development of the New Zealand version of the WHOQOL-BREF In 2009 to 2011 NZ National items were developed from focus groups and tested for their suitability. Data from the general population were obtained using a random sample from the national electoral roll, and analysed using confirmatory factor analysis and Rasch analysis. During the last few years, researchers at AUT and elsewhere have collected data in a large variety of settings, which have now been pooled to produce national reference values.

19 Development of New Zealand national Items: Conducted 13 focus groups with general community members, people with disabilities, and health experts. WHOQOL team wrote potential new items based on themes that had emerged from those focus groups. 24 new items were sent out with the WHOQOL-BREF to the random sample of 3,000 participants from the national electoral role (n=808).

20 Items were excluded if they exhibited the following: ceiling or floor effect low correlation with total QOL and likely parent domain decreased reliability of the domains does not discriminate between sick and well duplicates an existing core item results in unacceptable CFA fit indices if included

21 Final selected new national items are: Psychological domain: To what extent do you feel you have control over your life? To what extent are you able to manage personal difficulties? To what extent are you feel respected by others? How satisfied are you that you are able to meet the expectations placed on you? Social domain: To what extent do you have feelings of belonging?

22 Reliability and validity values:

23

24

25

26 Using the WHOQOL-BREF is easy Administer a questionnaire to an individual Either self-administered or intervieweradministered Note: A time frame of two weeks is indicated in the assessment. It asks us to think about our lives over the last two weeks.

27 Let s fill one in! Read the instructions. Time yourself. Discuss and compare ratings with a colleague. What are your thoughts about the questions?

28

29 Different measures for different purposes By service users/clients about themselves e.g. WHOQOL (subjective). By clinician about service user/client. HONOS (subjective) By management to stakeholders about service delivery. e.g. KPIs (social outcome measure - from service user information) EXPLain KPI

30 Purposes and uses of the WHOQOL-BREF in mental health services assessment Leads service users to assess themselves and their progress. Facilitates discussion between case manager and service user to review priorities, progress and goals. Aids service provider to review the service programmes through user progress. Provides feedback to stakeholders.

31 Service user level Identifies strengths Self-rated measures that give people a voice. Measure and track own wellbeing. Permits individual goal setting. Encouraged self-direction, empowerment, and taking responsibility.

32 Case manager level Helps understand service user concerns and living situation. Serves as a basis for longer term planning with the service user. Provides feedback and helps track the service user s progress. Complements the social outcome indicators now available.

33 Service and organisational level Builds evidence and contextualises the service. Complements good recovery planning practices. Service evaluations, particularly new services. Assists in reviewing strategic direction for the service provider. Shareing infromation for learning Comparisons are possible because the WHOQOL is a globally recognised and validated outcome tool.

34 Stakeholder level Aggregate data provided to stakeholders in numerical tabular form. Additional attribution information also required.

35 Items for discussion How do you define success? absolute standard: e.g. an individual reaches a certain cut-off score, such as the mean of a comparison group, such as the national reference values provide. relative standard: e.g. the score of an individual shows a positive change, regardless of the final level.

36 Items for discussion Attribution: Even if you notice change in scores, how do you know this change was due to the intervention programme or other factors? Should qualifying qualitative information accompany qualitative findings?

37 Scoring the WHOQOL-BREF Scoring individual facet scores (31 facets) Scoring domains. (4 domains)

38 NB: The two global items are not included in domain score calculations. You can include the NZ national items in domain score calculations. It will make the social domain more robust as it will now be four items instead of three. However, as you present them, please make it clear that your scores contain national items.

39 Discuss changes and decide what to do. Especially in areas targeted for change last time. Reflect on these changes and possible explanations for them. Consider what has happened in this person s life over the last 3-6 months. Look at the areas scoring low, particularly those most important to the client. Consider together anything that could be done over the next 3-6 months to improve this area of life. Look at high scores and discuss why they may have gone up. The service user might want to do this alone. The service user might want your support. The service user might want to do this with a family member. The service user might want to take this to another service to work on.

40 Service user feedback Entry scores recorded for each facet. Most important facets may be prioritized. Next administrations depends on service use. (3 or 6 months usual). Exiting the service administration. Compare changes after each administration?. Compare changes in those items of special priority to the service user.

41 domain scor Value of domain level analysis Calculate domain scores Some items are negatively worded, and are thus transformed from negatively framed questions to positively framed questions. This is done mathematically by subtracting the client s score from 6. Reverse code the three negatively-worded items: Q3, Q4, Q26 now this is easy to identify on the scoring sheet. For example: Original response Q3 = 1, thus 6-1 = 5 (recoded response) Original response Q4 = 2, thus 6-2 = 4 (recoded response) Original response Q26 = 4, thus 6-4 = 3 (recoded response)

42 Examining the response patterns: You might decide to discard assessments from participants with 20% or more missing answers. However, the reasons for missing those questions could be interesting information by itself (if you are able to access that information). As you record the data on a spreadsheet, watch out for response sets. Very obvious response patterns are easy to detect, such as all 1s or all 5s. The negative worded questions (Q3, 4, & 26) allow you to detect those: If someone really has such high QOL with lots of 5s, then the negatively worded questions should be 1s. Other response patterns are harder to detect and require statistical methods, although even those are limited. A large sample size typically takes care of this problem.

43 Missing scores Under some circumstances (such as when the sample size is very small), you may decide to impute data. In other words, the missing item values are calculated as the rounded average of the other scores on the same domain. However, you only do that if at least 50% of the items belonging to the domain were answered: Physical ( 4) Psychological ( 7) Social ( 3) Environmental ( 4)

44 Check domains to ensure there were no mistakes Ensure domain scores fall within the expected range: Physical Domain range = 7 to 35 (7 items) Psychological Domain range = 6 to 30 (6 items) NZ Psychological Domain range = 10 to 50 (10 items) Social Domain range = 3 to 15 (3 items) NZ Social Domain range = 4 to 20 (4 items) Environment Domain = 8 to 40 (8 items)

45 Check for response sets Very obvious response patterns are easy to detect, such as all 1s or all 5s. The negative worded questions (Q3, 4, & 26) allow you to detect those: If someone really has such high QOL with lots of 5s, then the negatively worded questions should be 1s. Other response patterns are harder to detect and require statistical methods, although even those are limited. A large sample size typically takes care of this problem.

46 Individualised graphs could show these changes easily 25 Mary's WHOQOL Administration points Jan April Aug Dec Mary s WHOQOL-BREF domain scores over the year

47 Outcome assessment the bigger picture organisation /service New Zealand comparisons Use it to learn and improve! QOL data can be used along with other information so we can do things like: Compare different groups Compare interventions Frequency 20 Identify influences on QOL 10 Evaluate programmes environmental 5.00 Measure outcomes

48 Estimating outcomes for each facet Entry scores recorded. Next administrations depends on service use. (3 or 6 months usual) Exiting the service administration. Compare changes annually for stakeholders. Item by item mean average changes calculated. Compare changes in those items of special priority to the service.

49 Ordinal versus interval scales WHOQOL is an ordinal scale. Measuring change is best using an interval scale. Problem of all Likert type scales used in measuring change. AUT will produce conversion tables next year. NB: these will be for domain scores only.

50 Item for discussion Response shift: Over time time in service, standards and expectations may change. If a service user s score has not increased in the expected direction, to what extent could this be a reflection of lack of change or of a shift in expectations that masks the fact that the client s circumstances have actually improved?

51 Using the WHOQOL-BREF in practice Introducing ourselves Representatives of a community of practice supported by the NZ WHOQOL Group & Platform Trust NGO MH and Addiction recovery services Providing community based support services: Service delivery is typically - strengths focussed, goal orientated, focussed on life-skill development, encouraging learning & increased insight, gaining new coping skills, community integration and social reconnection drawing on consumer informed and psychosocial models of recovery

52 Using the WHOQOL-BREF in practice Two main uses different yet inter-related As a practice tool - to increase self determination for clients As part of an organisation / service level outcome measurement suite - to increase client feedback about the impact of services on their lives Early questions, decision making, engagement The Rationale why evaluate subjective HRQOL during service delivery? Where does it sit within an outcome suite? What is the core fit with the service / organisation s mission, vision, values, service delivery ethic or primary service delivery models? Warm up to roll out consultation and collaboration clients included.

53 Using the WHOQOL-BREF in practice Roll Out Essentials Integration into strategic planning and business planning, reviews, reporting and overt demonstration of leadership support Communication information for clients / stakeholders Staff training packages with a practical emphasis Integration into service delivery frameworks - entry, exit, reviews Support for implementation champions, team meetings, coaching Monitoring uptake and addressing where it is slow / no uptake Reviews of progress and challenges imbedded in routine meetings Staged and paced in consideration of the current context Methods of feedback & follow-up - clients / staff / services / organisation Resources and IT systems requirements

54 Using the WHOQOL-BREF in practice Implementation Drivers Alignment and fit Implementation planning and goals Leadership support Empowerment [ clients, staff, services ] Staff development Evaluative Learning and CQI focus Explorative and Developmental Evaluation

55 Using the WHOQOL-BREF in practice Staff self-assessed areas of need prior to the development of an outcome informed practice training Average ratings for each question where 1= low knowledge / perceived value and 10= high knowledge / perceived value Knowledge about the history of OM in NZ Knowledge of 'outcome informed practice' Importance of giving / receiving Feedback about outcomes Confidence giving / receiving feedback about OM Importance of recognising who is obtaining recovery outcome and who is not Perceived value of OM to your role Focus points of the 7 questions asked about in the pre-training survey Perceived value of measuring outcomes to clients

56 Using the WHOQOL-BREF in practice Staff training needs some qualitative comments: I would like to be confident that the measurement tool we use is the best available and adds real value to our practice and relationships. People would need to know it is working. Peer Support needs to know that Peer Support is working More about historical measures to present outcome measures. Would like to be able to articulate more clearly to team members, the relevance of client outcomes to service delivery to support work practices. We need to have the list of DHB outcomes sought do not know what the outcomes are that are expected of us. Being able to use the information effectively on client s recovery journey. To be able to put it out in a positive manner so clients can be able to contribute well towards their goal. Cultural diversity and measurement verses mainstream outcome measurement to identify needs assessment and requirements Evidence based practice. More information on the relevance to us in the context of the changing face of mental health support work. Discussion on graphs and simplicity for peers to understand and PSW to explain to peers

57 Using the WHOQOL-BREF in practice As a practice tool staff & client observations The meaning assigned by clients to ratings Insight into their world Recognising strengths & resources Establishing client s self determined priorities for change Enabling feedback about interventions on HRQOL Translating this into personal planning What's going on for me now? What do I want the future to look like? What have I learnt? Stimulating a growth mindset. Context of a trusting relationship ++ Enhances relationship but trust a pre requisite Can be misperceived as a test / service outcome / staff performance measure Engagement and responses can depend on person s state / situation

58 Using the WHOQOL-BREF in practice Individual client results

59 Using the WHOQOL-BREF in practice Certainly evidence of people using QOL as conversation starter or opening a wondering about what s happened. I m surprised about how useful that has been in helping people focus conversations it provides a neat little structure with a number of domains. (Emerge Aotearoa Practice Leader, 2013) It touches on areas we don t usually go into with clients so we can gain some more insignts into where clients are at. It is good for gaining insights and self awareness in clients ( CSR staff 2015) Continuous use of the tool provides opportunities to compare previous results and shows whats working and what requires improvement improves support to an individual we work with because it enables them to identify their choices needs and thoughts (CSR Team Leader 2015) Client quotes: Its good, it s visual and its something you have done your self. It s not like somebody had interpreted it for you. You have had to do it yourself, so you can t say somebody else thought that, you must have thought that at the time. It's a really good thing. I was worried some answers may show a set back it s hard to be completely honest I felt I needed to keep something back I m honest with him so he can help me Reminded me of therapy The timing aspect is interesting; if you get me on a good day then its good, if you got me on a bad day then it wouldn t be so good

60 Using the WHOQOL-BREF in practice In Organisations as a CQI / Outcome Measure Service level Expected and identified patterns of change Identifying at risk individuals / those not thriving in the BAU context Used alongside other evidence sources Direct / indirect impacts on HRQOL can be explored Outcomes of service level interventions can be explored Organization level Trends over time for service population / subgroups Responses to systemic intervention and changes Commonalities and differences between services Minority groups and subpopulation CQI evaluation Health promotion role Sector learning [ future potential]

61 Using the WHOQOL-BREF in practice In Organisations Domain Level Analysis Richmond: Mean domain ratings (initial cohort) Mean % ratings 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 57% 59% 59% 58% 55% 58% 57% 58% 55% 57% 59% 58% Physical domain Psychological domain Social/ Relationships domain 62% 63% 63% 63% Environmental domain mean baseline mean 3 month mean 6 month mean 9 month

62 Using the WHOQOL-BREF in practice In Organisations Facet Level Analysis Changes in facet 19 ratings among a cohort of complete s [entry, review exit] clients at CSR Satisfaction with self on the WHOQOL BREF over time Self ratings on WHOQOL BREF From Days within CSR

63 Using the WHOQOL-BREF in practice The Attribution Dilemma Qualitative Data collection: an unresolved challenge: Exploring ways to create further meaning from the quantitative results that remain aligned to the self determination objectives of the tool s use and purpose in the outcome suite

64 Using the WHOQOL-BREF in practice Implementation challenges: Technical Problems - Difficulty accessing data - Expense of collecting information - Difficulty providing analysis and feedback in a timely manner - Dislike of paperwork Measurement problems - Lack of adequate measures - Difficulty obtaining follow-up information Organisational Issues - Preference for operating subjectively or on the basis of authority - Fear of control by management - Fear of empowerment of workers (source: Binner & Topolski, 1994)

65 Using the WHOQOL-BREF in practice Current Areas of Focus Data collection rates Data analysis service level reports collaborative NGO work with Wild Bamboo Feedback processes and use of the feedback to inform individual service planning Work in progress using the tool to its full potential as a CQI and service / organisational outcome measure

66 Using the WHOQOL-BREF in practice

67 Using the WHOQOL-BREF in practice

68 Using the WHOQOL-BREF in practice Recovery is happening when people can live well in the presence or absence of mental health problems Scottish Recovery Network

69 Computer storage and analyses Graphic outputs: For individual service users. For management and stakeholders. Possibility of presenting the BREF and collecting responses on line?

70

71 Future Continual improvement of the BREF: Additional mental health recovery items. Melissa Rowthorn. Possibly discarding some items. Updating reference values periodically. Developing change score tables. (Ordinal to interval scaling problem) Item banking. (for specificity and flexibility) Attribution studies. Collaboration with wild bamboo

72 Additional Versions Does your client think this measure is too brief - do you want more detail to work on? The WHOQOL-100 is available. Is your client particularly interested in spirituality, making sense of their recovery in terms of personal life beliefs and values? Consider the additional Spirituality module WHOQOL-SRPB. Your client s first language is not English contact us to see if we can source a version in your client s first language.

73 Items for discussion What suggestions do you have for future developments.

74

75 Purpose of this workshop 1. To give you very brief background of the original development of the WHOQOL tools and their purpose. 2. To complete the questionnaire and be able to score it. 3. To show the uses of the NZ WHOQOL-BREF in mental health recovery programmes in NZ. 4. To show you how the NZ WHOQOL scores can be recorded and analysed for different uses. 5. To discuss how the WHOQOL is introduced into service use. 6. Relevancy and future modifications for use in mental health outcomes assessment.

76 How to contact us The NZ Mental Health Recovery WHOQOL Group who meet bi-annually. Ross Philips - Pathways Phillipa Gaines Lattice/Platform Sarah Andrews Emerge Aotearoa Melissa Rowthorn -Connect Patricia Hsu WCDHB Ross Phillips Pathways James Gordon Wild Bamboo Mark Smith Te Pou And 20 Others.

77 How to contact us Adjunct Professor Rex Billington Tel.: extension Associate Professor Chris Krägeloh Tel.: extension We are based on AUT University s North Shore (Akoranga) Campus.

78 website For people who are registering for the first time, they can do that directly from the WHOQOL webpages login page: The registration request will come through to Chris K. or Rex B. for the time being. They will agree the request. Then there will be a couple of simple steps to follow which you will receive through the .

79 Thank you!