Panel 2 What Does, What Could and What Should Count as Research Use? The right kind of evidence integrating, measuring and making it count

Size: px

Start display at page:

Download "Panel 2 What Does, What Could and What Should Count as Research Use? The right kind of evidence integrating, measuring and making it count"

Beryl Harrell
5 years ago
Views:

1 Power, Politics and the Use of Health Equity Research Panel 2 What Does, What Could and What Should Count as Research Use? The right kind of evidence integrating, measuring and making it count Patricia J. Martens PhD Director, Manitoba Centre for Health Policy Professor, Dep t Community Health Sciences Faculty of Medicine, University of Manitoba CIHR/PHAC Applied Public Health Chair Toronto: February 17, 2011

2 The right kind of evidence Involving the users, uptake by decision makers, and measuring impact Using the right measures Using the right research designs for population levels

3 Right kind of evidence? We both need good tools! Researchers: best possible approaches to answer the question in the most valid and reliable way Decision makers: research which answers something relevant, real world, of high importance

4 Tool #1: Involving users, measuring impact Integrated KT MCHP s model with the provincial government The Need To Know Team Interactive workshops with decision makers Looking for the stories in the data Bringing context to numbers

MCHP: Who we are The Manitoba Centre for Health Policy Community Health Sciences, Faculty of Medicine 5-year grant relationship with Manitoba Health since 1991 but a long history pre-dates

5 MCHP: Who we are The Manitoba Centre for Health Policy Community Health Sciences, Faculty of Medicine 5-year grant relationship with Manitoba Health since 1991 but a long history pre-dates this A magnet centre Attracts the attention of other national and international research centres Constant visits from and consultations with other provinces and national/international groups

6 MCHP: What researchers do with the information - KT At the government level Deliverables (i.e., research reports); briefing of ADMs, DM, Minister of Health, other Ministers, workshops At the RHA level Annual Workshop Days (WRHA, MH, non-winnipeg RHAs), dissemination of reports, website data The Need To Know Team At the researcher level Research reports, publications, conferences etc. Concept Dictionary and Glossary, website At the public/clinician level Four-pagers; clinician one-pagers; media interviews, op eds,, responses to news, website

MCHP s involvement in influencing health policy The Need To

Manitoba Health CIHR-funded: 2001-2006 through CAHR)program;

CIHR 2005 KT Award for Regional Impact Highlighted as 1 of 5

7 MCHP s involvement in influencing health policy The Need To Know Team since 2001 A decade of collaboration: MCHP, RHAs, Manitoba Health CIHR-funded: through CAHR)program; CIHR/PHAC Applied Public Health Chair for Martens CIHR 2005 KT Award for Regional Impact Highlighted as 1 of 5 knowledge to action stories in the 2009/2010 annual report of CIHR

8 Our whole philosophy is health planning based on evidence Arlene Wilgosh, CEO of the WRHA, former DM of Health

9 Involvement and influencing health policy MCHP s Annual Workshop Days Rural & Northern RHAs, Winnipeg RHA, Manitoba Health Days Look for the STORIES! Evidence-based stories lead to evidence-informed decisionmaking

In Arabian Nights: A Caravan of Moroccan

Shah) pages 297-299 Stories are a way of

10 In Arabian Nights: A Caravan of Moroccan Dreams (T. Shah) pages Stories are a way of melting the ice, he said gently, turning it into water. They are like repackaging something changing its form so that the design of the sponge can accept it.

Lewis S, Martens PJ, Barre L. Estimating the Return on Investment for health services research: A theoretical and empirical analysis. In Canadian Academy of Health Sciences.

11 Lewis S, Martens PJ, Barre L. Estimating the Return on Investment for health services research: A theoretical and empirical analysis. In Canadian Academy of Health Sciences. Making an Impact: A preferred framework and indicators to measure returns in investment in health research. Ottawa, Canada: Canadian Academy of Health Sciences, January Appendix A (Commissioned Papers): A21-A42. Available at

12 So what works? researchers, decision makers USER INVOLVEMENT FROM START TO FINISH integrated KT INTERACTIVE FORUMS EVIDENCE-BASED STORY TELLING potentially leads to EVIDENCE-INFORMED DECISION MAKING

13 So what does it take? To develop collaborative relationships around policy relevant research and uptake: TIME and $ commitment SHARED LANGUAGE TRUST RELATIONSHIP BUILDING LETTING GO of traditional roles PATIENCE UNDERSTANDING Bowen S, Erickson T, Martens P. More than using research : the real challenges in promoting evidence informed decision making. Healthcare Policy 2009;4(3): Bowen S, Martens PJ. A model for collaborative evaluation of university community partnerships. J. Epidemiol. Community Health 2006; 60: Bowen S, Martens PJ, The Need To Know Team. Demystifying Knowledge Translation : Learning from the community. Journal of Health Services Research & Policy 2005;10(4):203

14 Roos et al Four success factors for getting research into action 1. continually pursue opportunities to dialogue with the relevant decision makers, building up understanding and trust; 2. use local data organized by some measure of SES, so that the message of inequity is strong and not easy to ignore; 3. focus on the outcome measure of educational attainment this helps develop a strong and broad constituency for taking action, and also attracts the business community into the discussions; 4. show research that the poor outcomes in the lower SES groups are not inevitable, but rather can be overcome through appropriate action.

16 Tool #2: Using the right measures Digging deep in the research tool kit to build that evidencebased story

17 Measures to illustrate equity For each of the following scenarios from a fictitious study, 1.give a one sentence message about the results to a decision maker, and 2.state whether the message is good news or not from their perspective!

18 Case Study 1 Relative Risk (RR) Relative Risk of Disease over Time (Low SES compared to High SES) Time

19 Case Study 2 Relative Risk (RR) Relative Risk of Disease over Time (Low SES compared to High SES) Time 4.00

20 Case Study 3 Relative Risk (RR) Relative Risk of Disease over Time (Low SES compared to High SES) Time 1.60

21 Case Study 4 Relative Risk of Disease over Time (Low SES compared to High SES) Relative Risk (RR) Time

22 Case Study 5 Rate of disease per thousand Time Low SES High SES

Case Study 6 Rate of disease per thousand 90 80 70 60 50 40 30 20 10 0 80

23 Case Study 6 Rate of disease per thousand Time Low SES High SES

24 Case Study 7 Rate of disease per thousand Time Low SES High SES

25 Case Study 8 Rate of disease per thousand Time Low SES High SES

26 Rate of disease per thousand Case studies 5 and Time Low SES High SES Relative Risk (RR) Relative Risk of Disease over Time (Low SES compared to High SES) Time

27 Rate of disease per thousand Case studies 6 and Time Low SES High SES Relative Risk (RR) Relative Risk of Disease over Time (Low SES compared to High SES) Time 4.00

28 Rate of disease per thousand Case studies 7 and Time Low SES High SES Relative Risk (RR) Relative Risk of Disease over Time (Low SES compared to High SES) Time 1.60

29 Rate of disease per thousand Case studies 8 and Time Low SES High SES Relative Risk of Disease over Time (Low SES compared to High SES) Relative Risk (RR) Time

30 So what does that tell us? Be VERY careful of relative measures The meaning may only be intuitive when the reference group in a time trend analysis has a rate that changes very little. If you are going to present relative measures, combine that with the real rates, and with other measures (like rate differences)

31 40 Figure 5.10: Amputations Among Residents with Diabetes Over Time by Urban Income Quintile Adjusted by (2005/ /08) age & sex, annual rate per 1,000 residents w ith diabetes (aged 19 and older) w ho had an amputation Amputations per 1, T1: 1984/ /87 T2: 1987/ /90 T3: 1990/ /93 T4: 1993/ /96 T5: 1996/ /99 T6: 1999/ /02 T7: 2002/ /05 T8: 2005/ /08 NF (Not displayed) U1 (low est income) U U U U5 (highest income) Disparity Rate Ratios (U1/U5) Disparity Rate Differences (U1-U5) Time Period (fiscal years) Comparison of Disparity Rate Ratios T8 to T1: 1.10 (95% CI 0.56, 2.15 ) NS Comparison of Disparity Rate Differences T8 to T1: 0.88, NS Martens et al Source: Manitoba Centre for Health Policy, 2010

32 Lorenz curves are intuitive If you don t dwell on the mathematics, Lorenz curves can tell stories to decision makers How bad is the inequity? In the words of Michael Marmot and proportionate universalism, what end of the spectrum do we focus upon targeted or universal? Is the inequity getting better or worse over time?

33 Lorenz curves are intuitive Cumulative Percent of Amputations Among Residents with Diabetes Figure 5.14: Adjusted Lorenz Curve for Amputations Among Residents with Diabetes in Urban Areas 2005/ / 08 Adjusted by (2005/ /08) age & sex, residents with diabetes (aged 19 and older) who had an amputation 100.0% 100% 80% 60% 40% 20% 0% 44.9% 26.0% U1 47.7% U2 64.0% 77.2% Cumulative Percent of the Population Lorenz Curve 67.9% U3 Line of Equality GINI = 85.5% U4 90.9% (95% CI 0.146, 0.277) 100.0% U5

34 Figure 5.13: Adjusted Lorenz Curve for Amputations Among Residents with Diabetes in Urban Areas 1984/ / 87 Adjusted by (2005/ /08) age & sex, residents with diabetes (aged 19 and older) who had an amputation 100% 100.0% Cumulative Percent of Amputations Among Residents with Diabetes 80% 60% 40% 20% 0% 39.2% 27.1% U1 50.5% U2 62.5% 82.2% Cumulative Percent of the Population Lorenz Curve 71.4% U3 Line of Equality 91.9% GINI = (95% CI 0.090, 0.251) 85.9% U4 45% of amputations in lowest SES, representing 26% of population Martens et al % U5 Figure 5.14: Adjusted Lorenz Curve for Amputations Among Residents with Diabetes in Urban Areas 2005/ / 08 Adjusted by (2005/ /08) age & sex, residents with diabetes (aged 19 and older) who had an amputation Source: ManitobaCentre for Health Policy, 2010 Cumulative Percent of Amputations Among Residents with Diabetes 100% 80% 60% 40% 20% 0% 39% of amputations in lowest SES, representing 27% of population 44.9% 26.0% U1 47.7% U2 64.0% 77.2% Cumulative Percent of the Population Lorenz Curve 67.9% U3 Line of Equality 85.5% U4 90.9% 100.0% GINI = (95% CI 0.146, 0.277) 100.0% U5 Source: ManitobaCentre for Health Policy, 2010

35 So use the appropriate tool for what you want to accomplish

36 Tool #3: Use the right research design for population health

38 Higher Internal validity Randomized Controlled Trials, RCTs Cluster randomization (CRTs) Quasi-experimental comparison group studies (good external validity) Time series with comparisons, many data points Time series with qualitative layers Observational (prospective, historical prospective) Case-control Lower Cross-sectional Anecdote/case study

39 100% 90% 80% 70% Figure 10.5: Trends in Non-Winnipeg Mammography Rates Age-adjusted percentage of women age receiving at least one mammogram in two years South Mid North Brandon Manitoba 60% 50% 40% 30% 20% 10% 0% RD 9.1% Mid-1990s: Beginning of notification and rural Mobile RD 47.0% Screening Program 1984/ / / / / / /981998/ / /04 Time period Source: Manitoba Centre for Health Policy, 2008 Martens et al. 2008

40 100% Figure 7.6: Trends in Winnipeg Breastfeeding Initiation Rates Maternal age adjusted percent of newborns breastfeeding at hospital discharge 90% 80% 70% 60% RD 11.7% 50% 40% 30% 20% 10% RD 16.0% Wpg Most Healthy Wpg Average Health Wpg Least Healthy Winnipeg Manitoba Canada Prenatal Nutrition Programs, Healthy Baby and Family First Significant jump at program onset, p<.003 0% 1988/ / / / / / / / / / / / / / / /04 Time Period Martens et al source: Manitoba Centre for Health Policy, 2007

41 An evidence-based story Forget s MINCOME research Randomized site Winnipeg Saturation site Dauphin (quasiexperimental comparison sites in same rural area) Different results on male highschool graduation due to social networking!

42 The Bottom line Use integrated KT Be VERY careful of relative measures give real rates! Embrace research designs that capture population synergies and networking Traditional RCTs are super for individual treatment, but may miss social networking effects Let s keep working on population-based designs for real world KT

43 M C H P Manitoba Centre for Health Policy

44 Three Cups of Tea Balti proverb: The first time you share tea, you are a stranger. The second time you take tea, you are an honored guest. The third time you share a cup of tea, you become family... Relationships!

45 Figure 10.5: Trends in Non-Winnipeg Mammography Rates Age-adjusted percentage of women age receiving at least one mammogram in two years Non-Winnipeg 100% 90% 80% South Mid North Brandon Manitoba Time Trend Analysis South - improving relative to the Manitoba time trend Mid -improving relative to the Manitoba time trend North- improving relative to the Manitoba time trend Brandon - improving relative to the Manitoba time trend 70% 60% 50% 40% 30% 20% Note: Rates for Brandon are missing for the period of 1984/86 to 1990/92 Huge gaps Reduced gaps: 57% to 66%, i.e., 9% real gap 10% 0% 1984/ / / / / / / / / /04 Time period Martens et al Source: Manitoba Centre for Health Policy, 2008

46 Figure 10.6: Trends in Winnipeg Mammography Rates Age-adjusted percentage of women age receiving at least one mammogram in two years Winnipeg 100% 90% 80% Wpg Most Healthy Wpg Average Health Wpg Least Healthy Winnipeg Manitoba Time Trend Analysis Wpg Most Healthy - not improving as fast as the Manitoba time trend Wpg Average Health - not improving as fast as the Manitoba time trend Wpg Least Healthy -not improving as fast as the Manitoba time trend 70% 60% Not as huge gaps 50% 40% 30% 20% 10% Persistent and possibly increasing gaps: 52% to 65%, i.e., 13% real gap 0% 1984/ / / / / / / / / /04 Time Period Martens et al Source: Manitoba Centre for Health Policy, 2008

4.6 Diabetes Incidence

4.6 Diabetes Incidence Definition: the average number of new cases of residents aged 19 and older with diabetes (Type I and II) per 100 person years as defined by either: at least one hospitalization with