Improvement Science In Action. Improving Reliability

Transcription

1 Improvement Science In Action Improving Reliability Adapted from the IHI Whitepaper and material generously shared by Roger Resar Sandra Murray May 2nd, 2014 Objectives Define reliability Introduce reliability concepts into our improvement work Understand where reliability fits into our improvement efforts 1

2 Reliability Failure free operation over time The change we are testing is consistently achieving the desired performance Improvement Pathway Scale Up Spread Replicating in an independent site Implement Move from one setting to a larger setting and then an even larger setting, within an interrelated system Making this change a part of the routine day-to-day operation of the system in your pilot population Test Trying and adapting existing knowledge on small scale. Learning what works in your pilot population INSTITUTE FOR HEALTHCARE IMPROVEMENT SUMMARY REPORT: 90-DAY PROJECT-Sustainability of Improvement Interventions (August-October 2011) Kedar Mate, Zoë Sifrim, Lloyd Provost 2

3 So Where Does Reliability Fit In? INSTITUTE FOR HEALTHCARE IMPROVEMENT SUMMARY REPORT: 90-DAY PROJECT-Sustainability of Improvement Interventions (August-October 2011) Kedar Mate, Zoë Sifrim, Lloyd Provost 5 Model for Improvement Change Concepts for Reliability What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? Act Plan Study Do Source: Improvement Guide, p. 10 3

4 Change Concepts The Improvement Guide, Nolan, Nolan, Provost, Norman and Moen, 2009 Change Concepts for Reliability Focus on Variation 51. Standardization 52. Stop Tampering 53. Develop Operational Definitions 54. Improve Predictions 55. Develop Contingency Plans 56. Sort Product into Grades 57. Desensitize 58. Exploit Variation Mistake Proofing 59. Use Reminders 60. Use Differentiation 61. Use Constraints 62. Use Affordances The Improvement Guide, Nolan, Nolan, Provost, Norman and Moen,

5 Health System Perspective: Quantifying Reliability Reliability = Number of actions that achieve the intended result Total number of actions taken Failure rate easier to use: Unreliability = 1 minus Reliability It is convenient to use Unreliability as an index, expressed as an order of magnitude (e.g means that the action fails to achieve its intended result 1 time in 10 times, in 100 times) Related measure: Time or counts between failures transplant cases between organ rejections) (e.g.: Number of Reliability is failure free operation over time. Levels of Reliability Level Reliability Success Rate Failures in 10,000 actions %-95% %-99.5% % % % > <.1 Note: reliability less than level 1 is termed chaos White Paper, p. 4 5

6 Current State Reliability of providing known or required processes in health care (non- catastrophic) generally at 10-1 or worse This is due to: Heavy reliance on vigilance and hard work Initial focus on outcomes rather than process Failure to design standard work to facilitate training and testing Poor understanding of any methodology for designing reliable processes Exercise One 1. Review the goals on your improvement project. 2. What level of reliability are you targeting on your project? Other 1 Level Reliability Success Rate Failures in 10,000 actions %-95% %-99.5% % % % > <.1 6

7 Three-tiered Strategy for Reliable Systems Prevent Identify Mitigate White Paper, p. 6 Design the system to prevent failure Design procedures and relationships to make failures visible when they do occur so that they may be intercepted before causing harm Design procedures and build capabilities for fixing failures when they are identified or stop the harm caused by failures when they are not detected and intercepted Sources Earl Weiner, U of Miami Nolan. BMJ March 2000 Espinosa/Nolan, BMJ March 2000 Plus: Redesign based on failure analysis Design for Reliability Level 1: Intent, vigilance and hard work Level 2: Design informed by reliability science and research in human factors Level 3: Design of integrated systems and high reliability organizations 7

8 Prevent The focus of 10-1 performance is the creation and use of a standardized approach to care for eligible patients. Standard tools and techniques: Basic standardization, such as the use of common equipment brands or standard order sheets and guidelines Memory aids such as checklists Feedback mechanisms regarding compliance with standards Awareness-raising and training White Paper, p. 6 Prevent AND Identify and Mitigate Strategies in the second tier (10-2 ) reflect a focus on catching or identifying instances when the standardized approach is not used. Some useful concepts at this level are those that seek to: 1. reduce the opportunities for humans to make mistakes. (concepts often referred to as error-proofing, ) 2. seek to eliminate ambiguities in the way tasks are performed, reducing the need for workaround solutions. White Paper, p. 6 8

9 Four common methods for error-proofing Reminders: Examples include calling patients the day before their appointments to reduce no-shows and late arrivals, and using checklists or alarms to prompt specific actions. Differentiation: To reduce confusion when actions, parts, or numbers are similar, patterns are broken by color coding, sizing parts differently, numbering items in easily distinguishable ways, or separating similar items. Constraints: Constraints restrict or limit the performance of certain actions. For example, computers that signal an alarm when two medications prescribed for the same person should not be taken together serve as a constraint. Affordances: An affordance provides clear visual or other sensory clues that lead the user to use a product or tool correctly, or perform the correct action. An outward-swinging door with a push-plate but no handle is an example. See these four improvement Change Concepts in Improvement Guide White Paper, p. 6 Applying these 10-2 Concepts in Health Care Building decision aids and reminders into the system Making the desired action the default (based on evidence) Creating intentional redundancy Scheduling key tasks such as discharges Taking advantage of existing habits and patterns Agreement among doctors and nurses to follow and learn from standard processes Strategies such as these, effectively employed, can boost the reliability of a process to or toward White Paper, p. 7 9

10 Example of Level 2 Design: Taking Advantage of Habits and Patterns Noting that all patients are placed in a holding room before surgery, the improvement team gathered data on the length of time from when patients leave the holding room to the first incision. Since the data showed that in the vast majority of cases the elapsed time was 30 to 60 minutes, a protocol was created that calls for the antibiotic to be started as the patient is transferred to the OR. The compliance rate increased from 70 percent to 100 percent White Paper, p. 8 Types of Changes on your Projects Level 1 (10-1 ): Intent, Vigilance and Hard Work Design Concepts Awareness and training ( education) Common equipment (and other structural standardization) Standard orders sheets Personal check lists Feedback of information on compliance Level 2 (10-2 ): Design Concepts Standardization of processes Building decision aids and reminders into the system Taking advantage of existing habits and patterns Making the desired action the default (based on evidence) Use of constraints, affordances) Creating redundancy Scheduling using proper operations theory 10

11 Exercise Two 1. Review the changes you are planning to test and implement in your improvement project. 2. What level of reliability do they relate to? 3. Will the changes you are testing lead to the level of reliability implied by your goals?(see earlier exercise) 4. If not, what kind of changes do you need to think about? 10-3 Reliability: High Reliability Organizations Sophisticated design of human interactions and working relationships Weick s Attributes 1. Preoccupation with failure (Prevent) 2. Sensitivity to operations (Prevent) 3. Reluctance to simplify interpretations (Identify) 4. Deference to expertise (Identify/Mitigate) 5. Commitment to resilience (Mitigate) Weick, KE and Sutcliffe, Managing the Unexpected

12 Redesign: Performance at 10-3 and beyond Redesign from learning based on failures in 10-2 design: What weaknesses in the design of the standardized processes are leading to or might lead to failure? This requires a focus not only on processes, but also on the structure in which the processes operate (e.g. linkages between care locations) FMEA often used throughout reliability work White Paper, p. 8 Three Steps in Working Towards Reliability Step 1. Prevent initial failure by standardizing the process to achieve 80% 2. Identify failures in sustaining 80% and test human factor based changes to consistently achieve 80% 3. Identify further failures and test design changes to reach level 2 or 3 of reliability Design Techniques 1. -Identify process(es) to standardize -and segment population (small parts of population for testing process changes) 2. -Use robust way to make failures visible -Use appropriate change concepts to test changes 3. -Identify common failures -Measure and study failures -Use knowledge of failures to redesign and test changes for further improvement 12

13 Some Useful Reliability Questions 1. Is the connection between project goals and process clear in project work? 2. Have we avoided vigilance and hard work as our primary design strategy? 3. Are we using some type of segmentation of population to help us test changes more effectively? 4. Is standard work with testing to see how well we are following standard work part of our strategy? 5. Is a design methodology being used? 6. Are we using small tests of change in rapid succession? 7. Is our data collection and display rapid enough to foster great team learning? 13

14 Some Useful Reliability Questions 1. Is the connection between project goals and process clear in project work? 14

15 Some Useful Reliability Questions 1. Is the connection between project goals and process clear in project work? 2. Have we avoided vigilance and hard work as our primary design strategy? Hard Work and Vigilance Not Effect Improvement strategy Is important, even laudable, but not robust design strategy for reliable work Is necessary for, but not sufficient to achieve, 95% or greater reliability Feels comfortable because of our culture we expect it of ourselves and colleagues To achieve level 2 or 3 reliability we must go beyond and use Human Factors concepts in our improvement strategies 15

16 Some Useful Reliability Questions 1. Is the connection between project goals and process clear in project work? 2. Have we avoided vigilance and hard work as our primary design strategy? 3. Are we using some type of segmentation of population in our project to help us test changes more effectively? Example of Segmentation: Medication Reconciliation 16

17 Example of Segmentation: Medication Reconciliation (Cont.) Why Is It Important? Allows us to control some variables as we begin our work Defines boundaries around which sequential expectations of success can be expressed Helps us set up better timelines Allows us to test design without dealing with every barrier in our first round of tests Helps us uncover complexity of system in cost-effective way (smaller scale way) Fosters understanding of differences in design needed as we move to more and more segments of population 17

18 Some Useful Reliability Questions 1. Is the connection between project goals and process clear in project work? 2. Have we avoided vigilance and hard work as our primary design strategy? 3. Are we using some type of segmentation of population to help us test changes more effectively? 4. Is standard work with testing to see how well we are following standard work part of our strategy? Useful Question Is team really committed to establishing standard work for the processes they are designing or redesigning? Standard work permits better training of employees We can assess better and meet training needs Test: can 5 front line users of process state the steps in the process the same way? If so, we are headed towards level 2 reliability 18

19 New Standardization Concepts Standardize to provide the appropriate infrastructure (the what, how, where, who and when) The what we are standardizing is based on medical evidence The how does not need medical evidence but rather system knowledge Initial standardized protocols with small time investment by experts are tested at very small scale Changed then to the protocol in the initial stages should be expected, encouraged required Defects are studies and used to test redesign to the protocol or process Some Useful Reliability Questions 1. Is the connection between project goals and process clear in project work? 2. Have we avoided vigilance and hard work as our primary design strategy? 3. Are we using some type of segmentation of population to help us test changes more effectively? 4. Is standard work with testing to see how well we are following standard work part of our strategy? 5. Is a design methodology being used? 6. Are we using small tests of change in rapid succession? 19

20 Our Methodology MFI Small scale PDSAs that increase in size/complexity over time 3 Step approach to reliability Methodology-Key Points Perfection is the enemy of reliable design (we aim for 80% first then 90 or 95 we work our way up) People who will use the design are part of the design creation/testing We have constant testing based on defects observed We use segmentation to test our design Note: FMEA useful here Key question: are we testing rapidly enough?? 20

21 How Will We Know? Are testing cycles used routinely? Are we starting small? Is it natural for the team to go from observed defect or problem into the idea of designing a test related to that defect? Are we using huddles to share results of tests rather than waiting for team meetings? Do we document tests? Do all team members have test responsibilities? Some Useful Reliability Questions 1. Is the connection between project goals and process clear in project work? 2. Have we avoided vigilance and hard work as our primary design strategy? 3. Are we using some type of segmentation of population to help us test changes more effectively? 4. Is standard work with testing to see how well we are following standard work part of our strategy? 5. Is a design methodology being used? 6. Are we using small tests of change in rapid succession? 7. Is our data collection and display rapid enough to foster great team learning? 21

22 Measurement Are small samples being taken and placed on run charts often enough for team learning? Small samples over time should be plotted to assess process improvement Process data should be collected by the team with strict attention to operational definition Process measures should be the primary team measures Outcome measures likely lag behind process and not always collected by team (may be collected elsewhere in org.) Goal for process measures should not be 100% (more like 95%) Enhancing Reliability--Failure Modes and Effects Analysis FMEA is a useful tool for reliability work method for evaluating the structures of systems and predicting their performance. systematic way to evaluate a process in order to identify where and how it might fail and to assess the relative impact of different failures. useful in identifying the parts of the process that are most in need of change. White Paper, p. 8 22

23 FMEA calls for a careful review of: Identify Steps in the process Identify failure modes (What could go wrong?) Identify failure causes (Why would the failure happen?) Identify failure effects (What would be the consequences of each failure?) Failure modes that happen frequently can be addressed by some of the process design concepts. Failure modes that happen infrequently but have serious consequences can be addressed by the Prevent, Identifyand-Mitigate, Redesign approach. White Paper, p. 9 FMEA of Chemotherapy Administration Occ: Likelihood of Occurrence (1 10) Det: Likelihood of Detection (1 10) Note: 1=Very likely it WILL be detected, 10=Very likely it WILL NOT be detected Sev: Severity (1 10) RPN: Risk Priority Number (Occ x Det x Sev) White Paper, p

24 10-3 Reliability References Amalberti, R. (2001). The paradoxes of almost totally safe transportation systems. Safety Science, 37, Dekker, Sidney (2005). Ten Questions about Human Error. Lawrence Erlbaum, Mahwah, NJ. Leonard, M., Frankel, A., Simmonds, T., Vega, K. (2004) Achieving Safe and Reliable Healthcare: Strategies and Solutions. Health Administration Press,Chicago. Weick, Karl (1995). Sensemaking in Organizations. Sage, Thousand Oaks. Weick, K & Sutcliffe, K. (2001). Managing the Unexpected. Jossey-Bass. 24