Multiple Criteria HTA Into Practice Local HTA Program. Innovation. Presenter: Paule Poulin, PhD CREATE OR DIE!

Transcription

1 Multiple Criteria HTA Into Practice Local HTA Program Innovation CREATE OR DIE! Presenter: Paule Poulin, PhD Authors: P. Poulin, L. Austen et al., Health Technology & Innovation Department of Surgery Calgary Research Portfolio CADTH Symposium April 3-5, 2011 Vancouver, BC 1

2 Objectives: 1. Local HTA Program Overview 2. Multiple Criteria for Technology Evaluation 3. Lessons Learned 2

3 Objectives: 1. Local HTA Program Overview 2. Multiple Criteria for Technology Evaluation 3. Lessons Learned 3

4 Local decision-makers need to consider Local priority-setting Local operations Local population health needs Local alternatives Local presence of trained personnel Local infrastructure impact Local budget impact.. As well as HTAs by HTA agencies HTA into Practice Local HTA Decision-Support Program 4

5 Forms to gather information Technology Request Form A Support of Technology + + Request Form B Contract-Costing Check Form C Local HTA Check Form D Local HTA Committee evaluation SIGNIFICANT CHANGE Uncertainties about clinical safety, effectiveness and/or cost or resources impact MINOR CHANGE No significant clinical, contractual, cost or resource issues Clinical, Financial, or Economic Impact Forms E-G Referred for further assessment Approved for purchase/implementation 5

6 Tools for Decision Making Local HTA Committee evaluation Screening Guide SIGNIFICANT CHANGE Uncertainties about clinical safety, effectiveness and/or cost or resources impact Tool 1 MINOR CHANGE No significant clinical, contractual, cost or resource issues Evaluation Criteria Referred for further assessment Approved For purchase/implementation Tool 2 Evaluation Worksheet Tool 3 Recommendation Made by Local HTA Committee Decision Made by Executive Committee Recommendation and Decision Guide Tool 4 Prioritization Tool Tool 5 6

7 Objectives: 1. Local HTA Program Overview 2. Multiple Criteria for Technology Evaluation 3. Lessons Learned 7

8 Criteria Development (supported by CADTH grant) Modified Delphi Process Criteria list from literature Criteria revision by selected group Member survey (criteria ranking) Group retreat (criteria revision) Final List of Criteria Criteria cross-checked with information-gathering Forms Revised Forms Criteria used to formalize Technology Evaluation and Decision Tools Revised Tools 8

9 Tool 2: Evaluation Criteria Domain (5) Health Gain Service Delivery Strategic Fit Innovation Financial 9

10 Tool 2: Evaluation Criteria Domain (5) Health Gain Service Delivery Strategic Fit Innovation Financial 10

11 Tool 2: Evaluation Criteria Domain (5) Health Gain Service Delivery Strategic Fit Innovation Financial 11

12 Tool 2: Evaluation Criteria Domain (5) Health Gain Service Delivery Strategic Fit Innovation Financial CREATE OR DIE 12

13 Tool 2: Evaluation Criteria Domain (5) Health Gain Service Delivery Strategic Fit Innovation Financial 13

14 Tool 2: Evaluation Criteria Domain (5) Criteria (12) Example Clarifying Questions Health Gain Efficacy (patient) evidence for health gain? Population Health addresses burden of disease? Standard of Care new Standard of Care? Service Delivery Safety as safe as current practice? Training requires training or credentialing? Access improves access to the under-served? Service Coordination impact on other health services? Strategic Fit Strategic Fit aligned with strategic goals? Innovation Knowledge & Research knowledge generation or transfer? Financial Cost start-up and on-going costs? Sustainability can we afford it? Economic Analysis cost-effective? 14

15 Tools 2 & 3: Evaluation Criteria and Worksheet Local HTA Committee evaluation Screening Guide SIGNIFICANT CHANGE Uncertainties about clinical safety, effectiveness and/or cost or resources impact Tool 1 MINOR CHANGE No significant clinical, contractual, cost or resource issues Evaluation Criteria Tool 2 Evaluation Worksheet Tool 3 Referred for further assessment Recommendation Made by Local HTA Committee Decision Made by Executive Committee Approved For purchase/implementation 15

16 Tool 3: Evaluation Worksheet For each criterion: A. Score the QUALITY and COMPLETENESS of the information provided by the applicant: Adequate, Inadequate, NA B. Score the SIGNIFICANCE and IMPACT of the technology on a scale of 0-5 C. Make a RECOMMENDATION 16

17 Tool 3A: Device X Quality and Completeness of Information Domain Criteria Example Clarifying Questions SCORE Health Gain Efficacy (patient) Evidence for efficacy? Indications for use? Inadequate Population Health Adequate projections of prevalence? Adequate Standard of Care Potential for new standard of care? Inadequate Service Delivery Safety Complications and risks addressed? Inadequate Training Training implications described? Adequate Access Improves access? NA Service Coordination Impact on other health services described? Adequate Strategic Fit Strategic Fit Aligned with strategic goals? Adequate Innovation Knowledge & Research Innovative characteristics described? Outcome measures described? Inadequate Financial Cost Information on resources and impact complete? Adequate Sustainability Economic Analysis Evidence to support cost-benefits? Adequate 17

18 Tool 3: Evaluation Worksheet For each criterion: A. Score the QUALITY and COMPLETENESS of the information provided by the applicant: Adequate, Inadequate, NA B. Score the SIGNIFICANCE and IMPACT of the technology on a scale of 0-5 C. Make a RECOMMENDATION 18

19 Tool 2: Evaluation Criteria Domain (5) Criteria (12) Example Clarifying Questions Health Gain Efficacy (patient) evidence for health gain? Population Health addresses burden of disease? Standard of Care new Standard of Care? Service Delivery Safety as safe as current practice? Training requires training or credentialing? Access improves access to the under-served? Service Coordination impact on other health services? Strategic Fit Strategic Fit aligned with strategic goals? Innovation Knowledge & Research knowledge generation or transfer? Financial Cost start-up and on-going costs? Sustainability can we afford it? Economic Analysis cost-effective? 19

20 Tool 2: Criteria - Scoring Significance and Impact CRITERIA 0 POINTS 1 POINT 3 POINTS 5 POINTS Efficacy No improvement in patient health gain compared with current practices Minimal improvement Moderate improvement Vast improvement Population Health Addresses a condition with very low prevalence (rate/100,000 < 1) Low prevalence (rate/100,000 is 1-10) Moderate prevalence (rate/100,000 is ) High prevalence (rate/100,000 is 1,000-10,000) Standard of Care Does not represent Standard of Care in Alberta Represents Standard of Care in some health regions in Alberta Represents Standard of Care in most regions in Alberta Represents a new Standard of Care in our region or Alberta 20

21 Tool 3B: Device X Scoring Significance and Impact POTENTIAL IMPROVEMENT Domain Criteria None: 0 Minimal: 1 Moderate: 3 Vast: 5 Health Gain Efficacy (patient) X Population Health Standard of Care Service Delivery Safety X X X Training (vast to minimal) Access Service Coordination X X Strategic Fit Strategic Fit X X Innovation Knowledge & Research X Outcome Measures (quality) Financial Cost (viability) X Sustainability X Economic Analysis X 21

22 Tool 4: Recommendation and Decision Guide Local HTA Committee evaluation Screening Guide SIGNIFICANT CHANGE Uncertainties about clinical safety, effectiveness and/or cost or resources impact Tool 1 MINOR CHANGE No significant clinical, contractual, cost or resource issues Evaluation Criteria Referred for further assessment Approved For purchase/implementation Tool 2 Evaluation Worksheet Tool 3 Recommendation Made by Local HTA Committee Decision Made by Executive Committee Recommendation and Decision Guide Tool 4 22

23 Tool 4: Recommendation and Decision Guide 1. Rejected (No) Recommendation Made by Local HTA Committee Decision Made by Executive Committee Recommendation and Decision Guide Tool 4 2. Approved (Yes) 3. Conditional (Yes but a. Single case b. Clinical audit c. Other (e.g. training) 4. Research Use a. Clinical trial b. Support of project C r i t e r i a 5. Referred 23

24 Tool 4: Recommendation and Decision Guide RECOMMENDATION OR DECISION 1. Rejected CRITERIA & RATIONALE (e.g., may include) Negative, poor, or no data on efficacy Insufficient evidence of safety Decreases or worsens service delivery 2. Approved Efficacy and Safety well established Enhanced population health is likely Strategic fit is strong Will likely improve service delivery Financial impact is the same or better than current practice (Cost Neutral) Cost-effectiveness is likely the same or better than current practice 24

25 Tool 4: Recommendation and Decision Guide RECOMMENDATION OR DECISION 3a. Conditional Single Case 3b. Conditional Clinical Audit 3c. Conditional Other CRITERIA & RATIONALE Urgent/emergent approval of a single case where options are limited Evidence promising for efficacy, but limited for effectiveness Good evidence for safety Cost-effectiveness may be uncertain Advantage over current practice needs to be established Cost needs to be established May require new training protocols Patient population may need to be defined 25

26 Tool 4: Recommendation and Decision Guide RECOMMENDATION OR DECISION 4a. Research Use Clinical Trial 4b. Research Use Support of Project CRITERIA & RATIONALE Efficacy uncertain or controversial Safety uncertain Population health is uncertain Effect on service delivery is uncertain May be innovative Clinical trials protocols must be observed Innovative project proposed by applicant Funding and research protocols in place Acceptable use of resources 5. Referred Technology has potential impact beyond the scope of surgery Working group required 26

27 Tool 4: Device X - Decision RECOMMENDATION OR DECISION 4a. Research Use Clinical Trial CRITERIA & RATIONALE Efficacy uncertain or controversial Safety uncertain Population health is uncertain Effect on service delivery is uncertain May be innovative Clinical trials protocols must be observed 27

28 Tool 5: Prioritization Tool Proposal for Technology One Proposal for Technology Two Proposal for Technology Three Prioritize Technologies for funding or purchase Recommendations Prioritization Tool Prioritize Technologies for funding or purchase Decisions Tool 5 28

29 Tool 5: Prioritization Tool Step 1. Screening Guide Step 2. Criteria Review Step 3. Criteria Weighting Step 4. Criteria Rating Scales Step 5. Technology Scoring Step 6. Technology Ranking Step 7. Additional Checks Adapted from Drs. Craig Mitton and Cam Donaldson 29

30 Objectives: 1. Local HTA Program Overview 2. Multiple Criteria for Technology Evaluation 3. Lessons Learned 30

31 Lessons Learned: WEAKNESSES Criteria have not been weighted Prioritization Tool has not been tested Patient interests not well represented Administrators and practitioners sometimes resistant to change 31

32 Lessons Learned: STRENGTHS Bottom-Up approach User develop ensures buy-in Reviewers satisfied with criteria and technology evaluation tools Provides a systematic, consistent and transparent approach to technology evaluation and decision-making 32

33 Acknowledgements Supported by: Canadian Agency for Drugs and Technology in Health The Department of Surgery & Surgical Services Calgary The Calgary Health Research Portfolio Calgary Surgery Clinical Network HTAI - AHS Participating Departments from the former Calgary Health Region: Surgery & Surgical Services Anesthesia Cardiac Sciences Critical Care Medicine Internal Medicine Professional Practice & Development Medical Device Safety & Risk Management Pharmacy Physician Leadership Portfolio, and the Calgary Health Research Portfolio 33

34 Multiple Criteria HTA Into Practice Local HTA Program Innovation CREATE OR DIE! Contact information: Dr. Paule Poulin Alberta Health Services - Calgary (403) Paule.Poulin@AlbertaHealthServices.ca 34