Ethics In Evaluation: The ARECCI approach to enhancing how ethical risk is managed in your evaluation practice CES Conference May 11-14, 2008

Transcription

1 Ethics In Evaluation: The ARECCI approach to enhancing how ethical risk is managed in your evaluation practice 2008 CES Conference May 11-14, 2008

2 Presentation to 2008 Conference of the Canadian Evaluation Society Quebec, PQ Canada May 12, 2008 By Linda Barrett-Smith & Birgitta Larsson ARECCI Project Lead Principal Alberta Heritage Foundation BIM Larsson & Associates for Medical Research With Contributions by Laura LHeureux L Heureux

3 Today Situating Ethics in Evaluation Describe the Alberta Research Ethics Community Consensus Initiative (ARECCI) journey Background and Description Present some deliverables ARECCI Online Ethics Screening Tool ARECCI Ethics Guidelines (Review Considerations) for QI and Evaluation Forum for discussion

4 ARECCI Collaborating Partners AHFMR Alberta Cancer Board Alberta Mental Health Board Aspen Health Region Athabasca University, University Research Services Canadian Evaluation Society, Alberta Chapter Calgary Conjoint Health Research Ethics Board Calgary Health Region Capital Health Region Chinook Health Region College of Physicians and Surgeons Research Ethics Community Research Ethics Board of Alberta David Thompson Health Region East Central Health Region Health Quality Council of Alberta Michael Smith Foundation for Health Research (BC) Ministry of Health and Wellness, Government of Alberta Northern Lights Health Region Office of the Information and Privacy Commissioner of Alberta Palliser Health Region Provincial Health Ethics Network of Alberta SEARCH Canada Southern Alberta Child and Youth Health Network

5 Why is Ethics an Issue? Recent increase in volume of evaluation projects Increasing scope and sophistication of evaluation projects Lack of clear established guidelines and ethics oversight process Wide range of practitioners with various levels of skill leading the projects

6 Evaluation Realities in Alberta The Government Accountability Act Policies of funders of health and social services, such as Health Canada, Alberta Alcohol and Drug Commission, Regional Health Authorities and large foundations Expected that a portion of the overall project budget will be allocated to evaluation

7 Guidelines Health Information Act (HIA) identifies the designated REBs for the province of Alberta: Requires ethics review of health information used for research purposes Six REBs designated as able to perform such ethics review Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS) Interpretation that evaluation does not require formal ethics review

8 Research versus Evaluation Overlap between the processes of knowledge generating projects including research and evaluation which leads to grey areas and lack of clarity Research focuses on defining best practice, Program Evaluation serves to advance better practices

9 Evaluation Ethics Guidelines CES Guidelines AES Guidelines Australasian Evaluation Society EES Professional Guidelines FOIP HIA

10 Who is Responsible? Evaluators Project Leads Clients Organization Funders What to submit and to whom?

11 Consequences Differences in deciding if a project is research & therefore needs REB review Inconsistency / lack of transparency in ethics review decisions Avoidance of ethic review Inconsistent consideration of ethical implications in non- research projects

12 Consequences (cont d) Determining minimal risk Who and how decisions are made End result possibility of exposing participants to potential risks and disrespect

13 Issues that led to ARECCI work Lack of clarity related to what projects required ethical review Same projects sometimes treated differently by separate research ethics boards Some projects unnecessarily going g to designated REBs Some projects not receiving review when needed

14 Alberta Research Ethics Community Consens s Initiati e Consensus Initiative Collaborative undertaking Joint initiative Alberta Heritage Foundation for Medical Research (AHFMR) REBs in Alberta Health Authorities Alberta Health and Wellness Created in 2003 Completed 3 phases

15 ARECCI Objectives 1. To develop a common understanding and broad consensus on issues of ethics review 2. To increase the clarity, consistency, transparency, and efficiency of ethics review processes in Alberta 3. To provide an approach to answering three questions: what type of investigation? what processes should it go through? what level of review should occur?

16 ARECCI Objectives (cont d) 4. To develop guidelines and tools for Alberta s health researchers, managers, ethics boards and other stakeholders to implement the recommendations 5. To inform regional, provincial and 5. To inform regional, provincial and federal policy related to ethics review processes

17 Phase 1: Assumptions & Pi Principlesi Protecting people involved in evaluation projects is, in principle, just as important as protecting people participating in research investigations It is possible to pragmatically distinguish, for purposes of ethics review, between research and quality / evaluation If an investigation is determined to require ethicsre review, itsho should ldbere reviewed edthrough its own particular process of ethics review by a body having appropriate jurisdiction and capacity

18 Phase 1: Assumptions & Pi i l ( t d) Principles (cont d) Although REBs currently provide the highest level of ethics review across the province, there is no reason, in principle, that a similarly high level of ethics review should not be developed and provided by other processes for quality or evaluation projects Each institution responsible for the protection of people and increasing knowledge will comply with legislation, regulations, and policy application to its function

19 Phase 1: Results Five Recommendations for Ethics Screening and Review Pragmatic Screening Tool to determine project primary purpose Research or Quality Improvement / Evaluation Risk Filters to determine level of risk Research Evaluation and Quality Improvement

20 Phase 2: Field Test Results Finalized ARECCI document describing underlying principles and concepts Protecting People While Increasing Knowledge (Dec 05) Revised stand alone, paper-based tools One Screen - Determining Project Primary Purpose Two Risk Filters for Quality and Evaluation Projects for Research Projects Ongoing g connections with other stakeholders Inter-provincial, national, international Agreement to sustain momentum to enhance awareness & build ethics capacity across the system

21 Phase 3: Feb 2006 Mar 2008 Capacity Building - Decision Support Guides Revisions, i AUTOMATION, and further testing of ethics screening tools Development of ARECCI Ethics Guidelines for QI and Evaluation Draft Process / Protocol for review of higher risk QI, QA and Evaluation projects Development of Project ethics education May 4-6, 2008 Inaugural National Conference on the issues

22 ARECCI Innovations to Date Concept of knowledge generating o To describe a range of activities: Research and Non-research (i.e., evaluation, QI, QA) Concept of Project Ethics in relation to Striking a balance in protecting/respecting people while increasing knowledge o Ethics considerations across types of projects 5 Recommendations (Principles) - Published Ethics Screening Tool online, paper versions Ethics Guidelines for QI & Evaluation Projects Review Considerations for higher risk Evaluation/QI type projects May 4-6, 2008 Protecting People Conference o Fostered national dialogue and action

23 For more information: Linda Barrett-Smith ARECCI Project Lead Alberta Heritage Foundation for Medical Research Edmonton, Alberta Canada Birgitta Larsson Principal, BIM Larsson and Associates Edmonton, Alberta Canada Laura L Heureux Southern Alberta Child & Youth Health Network Alberta Children s Hospital Calgary, Alberta Canada laura@panoptik.ca Or visit the ARECCI website: