We focus on things that are easy to see

Transcription

1 Beyond Compliance

2 We focus on things that are easy to see

3 To reduce risk (and improve safety), we need to focus on things that are not easy to see

4 Personal experience

5 We didn t know what we didn t know!

6 JFK, Jr. Example

7 JFK, Jr. didn t know what he didn t know!

8 Montrose Example

9 They didn t know what they didn t know!

10 SWA Example

11 They didn t know what they didn t know!

12 % Human Error Source: Shappell and Wiegmann

13 ICEBERG We focus on things that are easy to see, but ignore the hidden dangers lurking beneath the waterline

14 LESSONS LEARNED

15 PUBLICATIONS The amount of information is overwhelming FAA (in the US) has over 5,500 publications!

16 ICAO Rules of the Air Pre-flight Action Before beginning a flight, the pilot-in-command of an aircraft shall become familiar with all available information appropriate to the intended operation.

17 Forgetting Curve 100 % Retention Day 30 Days Time

18 Risk = Probability X Severity

19 Uncertainty Natural Variability Cannot be reduced by more or better information

20 Knowledge Uncertainty Can be reduced by more or better information

21 Knowledge is the Key to mitigating risk

22 Data Information Knowledge

23 Data Information Knowledge Discrete objective facts about events Raw material for information No structure or context

24 Data Information Knowledge Data becomes information when we provide the context Weather Temperature Dew Point 4 2 Precipitation

25 Data Information Knowledge Knowledge gives us perspective 1 (Connections) Chance of Icing 2 (Consequences) Adversely affect aircraft performance 3 (Comparison) This caused the Montrose crash 4 (ACTION) I need to deice my aircraft in accordance with the AFM

26 ACTION Action is the key to mitigating risk (awareness of a hazard is insufficient)

27 Quick Summary Knowledge reduces uncertainty and facilitates action Action is the key to mitigating risk Knowledge and action are the benchmarks we need to use to determine the effectiveness of any risk assessment system

28 Types of Risk Assessments Risk Matrix Hazard Scoring

29 Risk Matrix Graphically depicts risk (intersection of probability and severity) Subjective (can create own definitions) Does not increase knowledge or facilitate action

30 Hazard Scoring Assign numerical values to various hazards We add up the score The higher the score, the higher the risk Appears quantitative Many hidden dangers

31 Hidden FRAT Dangers 1. Based on Preflight Perception 2. Knowledge Assumption 3. Awareness Paradox 4. Trending Trap 5. Empirical Illusion

32 A B Which square is darker?

33 Bottom Line Things are not always what they seem!

34 Hidden FRAT Dangers 1. Based on Preflight Perception 2. Knowledge Assumption 3. Awareness Paradox 4. Trending Trap 5. Empirical Illusion

35 Knowledge Assumption PIC had over 12,000 hours and 7 type ratings We assumed he had the knowledge Our assumptions were incorrect!

36 Bottom Line How do you know what you don t know?

37 Hidden FRAT Dangers 1. Based on Preflight Perception 2. Knowledge Assumption 3. Awareness Paradox 4. Trending Trap 5. Empirical Illusion

38 Awareness Paradox When using hazard scoring systems, less experienced pilots appear to face lower risk than more experienced pilots given the same hazards Since a more experienced pilot is more aware of his/her surroundings, he/she will always be aware of more hazards If a hazard is identified, you must give the hazard a numerical value Therefore, a more experienced pilot will tend to score risk higher than less experience pilots Example: Thunderstorm Less experienced pilot sees only the thunderstorm and scores it a 5 More experienced pilot sees all of the hazards embedded within a thunderstorm (e.g., lightening, hail, turbulence) and scores it a 45

39 Bottom Line Ignorance is not bliss!

40 Hidden FRAT Dangers 1. Based on Preflight Perception 2. Knowledge Assumption 3. Awareness Paradox 4. Trending Trap 5. Empirical Illusion

41 Trending Trap The Trending Trap is the belief that by trending our hazard scores over time, we are gaining insight into the risks our organization is facing

42 Trending Trap Example Severe Icing Night Contaminated Runway Pop-up Trip Low Ceilings High Crosswinds Mountainous Terrain Thunderstorm No Published Approaches High Pressure Altitude Twilight Pop-up trip and Twilight add up to a low score (so we assume it is low risk)

43 Severe Icing Night Contaminated Runway Pop-up Trip Low Ceilings High Crosswinds Mountainous Terrain Thunderstorm No Published Approaches High Pressure Altitude Twilight Severe icing (5) has the same score as Pop-up Trip + Twilight (5) Would we consider Severe Icing low risk? To get insight, we must look at the hazards instead of the number Therefore, trending the numbers does not provide the insight we need

44 Overconfidence Bias IF WE THINK WE HAVE THE ANSWERS, THEN WE ARE GOING TO STOP ASKING QUESTIONS!

45 Bottom Line Avoid the Trap!

46 Hidden FRAT Dangers 1. Based on Preflight Perception 2. Knowledge Assumption 3. Awareness Paradox 4. Trending Trap 5. Empirical Illusion

47 Empirical Illusion The Empirical Illusion is the mistaken belief that 2+2 = 4

48 QUALITATIVE = C = 3 which means =

49 NUMERICAL QUANTITATIVE

50 NUMERICAL Relating to, or expressed as, a number QUANTITATIVE Relating to, or measured by, a quantity

51 Most Information Ratio Interval Ordinal Least Information Nominal

52 Nominal Ordinal Interval Ratio Used as labels All we know is two things are not alike Example: football players, country dialing codes

53 Nominal Ordinal Interval Ratio Used to rank order things All we know is that something is more than something else (but we don t know by how much) We cannot add things together because there are no fundamental units Examples: - food rating systems - movie rating systems

54 Nominal Ordinal Interval Ratio IF we cannot add numbers, THEN we cannot multiply or divide! since 5 x 3 =

55 Nominal Ordinal Interval Ratio which means, we cannot use averages

56 Nominal Ordinal Interval Ratio Interval measurements systems have defined units, but no true zero scale Example: thermometer

57 Nominal Ordinal Interval Ratio The traditional math we learned in school It has a zero scale and defined units This allows us to add, subtract, multiply, and divide = 4

58 Bottom Line

59 Hazard scoring systems are ORDINAL systems, not RATIO systems Danger when you confuse them.

60 By sharing our knowledge and experience with other pilots, we can help other pilots make better decisions (and avoid accidents)

61 The Troop Who Rides One In We should all bear one thing in mind when we talk about a troop who rode one in. He called upon the sum of all his knowledge and judgement. He believed in it so strongly that he knowingly bet his life on it. That he was mistaken in his judgement is a tragedy not stupidity. Every supervisor and contemporary who ever spoke to him had an opportunity to influence his judgement. so, a little bit of all of us goes in with every troop we lose. Author Unknown

62