Evidence for Expertise in Fingerprint Identification

Transcription

1 Evidence for Expertise in Fingerprint Identification (2011) Psychological Science, 22, Matthew Thompson Jason Tangen Duncan McCarthy The University of Queensland National ICT Australia Queensland Police Service

2

3

4

5

6 THE FORENSIC REASONING PROJECT ForensicReasoning.com

7

8 US National Academy of Sciences Report (2009)...testimony based on faulty forensic science analyses may have contributed to wrongful convictions of innocent people. frequent absence of solid scientific research demonstrating the validity... reliability and accuracy of forensic analyses... The simple reality is that the interpretation of forensic evidence is not always based on scientific studies to determine its validity. This is a serious problem.

9 NEWS FEATURE SCIENCE IN COURT NATURE Vol March 2010 THE FINE PRINT A single incriminating fingerprint can land someone in jail. But, Laura Spinney finds, there is little empirical basis for such decisions. T

10

11

12

13 Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach The Report of the Expert Working Group on Human Factors in Latent Print Analysis The Fingerprint Inquiry Report February 2012 National Institute of Standards and Technology Volume Forensic identification can no longer be regarded as a methodology that is detached from human judgement and subjective decision making.

14 Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach The Report of the Expert Working Group on Human Factors in Latent Print Analysis The Fingerprint Inquiry Report February 2012 National Institute of Standards and Technology Volume

15 Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach The Report of the Expert Working Group on Human Factors in Latent Print Analysis The Fingerprint Inquiry Report February 2012 National Institute of Standards and Technology Volume The subjective nature of fingerprint evidence Recommendation 1 (Para ) Fingerprint evidence should be recognised as opinion evidence, not fact, and those involved in the criminal justice system need to assess it as such on its merits.

16 Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach The Report of the Expert Working Group on Human Factors in Latent Print Analysis The Fingerprint Inquiry Report February 2012 National Institute of Standards and Technology Volume Recommendation 2 (Para ) that are examined, their ability to observe detail in mark and print reliably, the subjective interpretation of observed characteristics, the cogency of explanations for any differences

17 Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach The Report of the Expert Working Group on Human Factors in Latent Print Analysis The Fingerprint Inquiry Report February 2012 National Institute of Standards and Technology Volume Recommendation 3 (Para 38.77)

18 Forensic DNA Evidence: The Myth of Infallibility William C. Thompson Department of Criminology, Law & Society and School of Law University of California, Irvine September 2011

19 Strengthening Forensic Science in the United States: A Path Forward "'$)-%("$.)!"/"$!0 /)1/"')*+#2/#., )38)(97865:;58<)6=7)%779>)3:)6=7)*3?78>5@)!@578@7),344A856;, )38)!@578@7B)"7@=83C3<;B)D89)EDF 13C5@;)D89)&C3GDC)/::D5?>), )38)/HHC579)D89)"=73?765@DC)!6D65>65@>.5I5>538)38)$8<5877?58<)D89)1=;>5@DC)!@578@7> Recommendation 3: Research is needed to address issues of accuracy, reliability, and validity in the forensic science disciplines. The National Institute of Forensic Science (NIFS) should competitively fund peer-reviewed research in the following areas: (a) (b) Studies establishing the scientific bases demonstrating the validity of forensic methods. "'$)%/"(+%/E)/,/.$J($!)1#$!! 2D>=58<638B).K,K The development and establishment FFFK8DHK79A of quantifiable measures of the reliability and accuracy of forensic analyses. Studies of the reliability and accuracy of forensic techniques should reflect actual practice on realistic case scenarios, averaged across a representative sample of forensic scientists and laboratories. Studies also should establish the limits of Copyright National Academy of Sciences. All rights reserved. reliability and accuracy that analytic methods can be expected to achieve as the conditions of forensic evidence vary. The research by which

20

21

22 How should human fingerprint matching expertise be tested?

23 Stimuli Representative Tightly Controlled Difficult

24 Stimuli Representative Tightly Controlled Difficult Participants Baseline Performance Qualified Experts

25 Stimuli Representative Tightly Controlled Difficult Participants Baseline Performance Court-qualified Experts Design Representative Signal Detection

26 Stimuli Representative Tightly Controlled Difficult Participants Baseline Performance Court-qualified Experts Design Representative Signal Detection Results Understandable across Disciplines Inform Expert Testimony

27 Stimuli Representative Tightly Controlled Difficult Participants Baseline Performance Court-qualified Experts Design Representative Signal Detection Results Understandable across Disciplines Inform Expert Testimony

28 Stimuli Participants Design Results

29 Stimuli

30 Queensland Police Training Materials

31 Let s test experts You mean the experts you want to test? Queensland Police Training Materials They were identified by experts But how do you know the matching prints actually come from the same source?

32 Stimuli Participants Ground truth Design Results

33

34

35

36

37

38

39

40 FIBR0013-time2-face-portrait-neutral.jpg

41 Stimuli Participants Ground truth Simulated crime-scene Design Results

42 Stimuli Participants Ground truth Targets Simulated crime-scene Design Results

43 Distractors

44

45

46

47

48 Stimuli Participants Ground truth Simulated crime-scene Targets Distractors Design Results

49 Stimuli Participants Ground truth Simulated crime-scene Targets Distractors Design Results

50 Participants

51

52

53 Stimuli Participants Ground truth Targets Experts QLD, NSW, SA, VIC, AFP Simulated crime-scene Distractors Novices UQ Undergrads Design Results

54 Stimuli Participants Ground truth Targets Experts QLD, NSW, SA, VIC, AFP Simulated crime-scene Distractors Novices UQ Undergrads Design Results

55 Design

56 36 Pairs 12 matches (targets) 12 non-matches (not similar distractors) 12 non-matches (highly similar distractors)

57

58 Stimuli Participants Ground truth Targets Experts QLD, NSW, SA, VIC, AFP Simulated crime-scene Distractors Novices UQ Undergrads Design Results Comparison 2AFC + Signal Detection Untimed

59 Stimuli Participants Ground truth Targets Experts QLD, NSW, SA, VIC, AFP Simulated crime-scene Distractors Novices UQ Undergrads Design Results Comparison Task 2AFC Untimed

60 Results

61 Match 100% 92.12% 74.55% 0% Experts Novices Tangen, J. M., Thompson, M. B., McCarthy D. J. (2011) Identifying Fingerprint Expertise. Psychological Science

62 Match 100% 92.12% 74.55% 0% Experts Novices Tangen, J. M., Thompson, M. B., McCarthy D. J. (2011) Identifying Fingerprint Expertise. Psychological Science

63 Match 100% 92.12% 74.55% 0% Experts Novices Tangen, J. M., Thompson, M. B., McCarthy D. J. (2011) Identifying Fingerprint Expertise. Psychological Science

64 Match 100% 92.12% 74.55% Not Similar 0% 100% 100% 77.03% 0% Experts Novices Tangen, J. M., Thompson, M. B., McCarthy D. J. (2011) Identifying Fingerprint Expertise. Psychological Science

65 Match 100% 92.12% 74.55% Not Similar 0% 100% 100% 77.03% 0% Experts Novices Tangen, J. M., Thompson, M. B., McCarthy D. J. (2011) Identifying Fingerprint Expertise. Psychological Science

66 Match 100% 92.12% 74.55% Not Similar 0% 100% 100% 77.03% 0% Experts Novices Tangen, J. M., Thompson, M. B., McCarthy D. J. (2011) Identifying Fingerprint Expertise. Psychological Science

67 Match 100% 92.12% 74.55% Not Similar 0% 100% 100% 77.03% Similar 0% 100% 99.32% 44.82% 0% Experts Novices

68 Match 100% 92.12% 74.55% Not Similar 0% 100% 100% 77.03% Similar 0% 100% 99.32% 44.82% 0% Experts Novices

69 Match 100% 92.12% 74.55% Not Similar 0% 100% 100% 77.03% Similar 0% 100% 99.32% 44.82% 0% Experts Novices All pairwise <.05

70 Hits 92% False Alarms Misses Correct Rejections 99.32%

71 Hits 92% False Alarms 0.68% Misses 8% Correct Rejections 99.32%

72 Roughly 5% of autopsies reveal lethal diagnostic errors for which a correct diagnosis coupled with treatment could have averted death, and an estimated 40,000 to 80,000 US hospital deaths result from misdiagnosis annually (Institute of Medicine, 1999; Newman-Toker & Pronovost, 2009). Thompson, Tangen, & McCarthy (accepted pending minor revisions)

73 These figures suggest that more Americans are killed in US hospitals every 6 months than died in the entire Vietnam War, and is equivalent to three fully loaded jumbo jets crashing every other day. Thompson, Tangen, & McCarthy (accepted pending minor revisions)

74 There is a benefit of expertise Experts are conservative Experts make mistakes

75 Our experiment was not designed to measure the pervasiveness of errors in practice or even assess the accuracy of individual examiners or forensic departments. We were measuring expertise rather than absolute performance. That is, how well fingerprint examiners perform relative to novices at comparing matching and nonmatching prints. Thompson, Tangen, & McCarthy (accepted pending minor revisions)

76 Match 100% 92.12% 74.55% Qualified fingerprint Not Similar 0% 100% 100% examiners now have evidence to 77.03% legitimately claim specialised knowledge, 0% which may satisfy Similar 100% 99.32% legal admissibility criteria % 0% Experts Novices

77 We need to conduct similar validation experiments for other forensic disciplines. Shoeprints Toolmarks Firearms DNA Hair and fibre Bite marks Bloodstain patterns CCTV comparison etc.

78 THE FORENSIC REASONING PROJECT ForensicReasoning.com

79 ForensicReasoning.com THE FORENSIC REASONING PROJECT Expertise What sets an expert apart from a novice? How does forensic expertise develop over time? Does the speed of expert decision making influence performance? How does memory for forensic information relate to matching accuracy?

80 ForensicReasoning.com THE FORENSIC REASONING PROJECT Training Can training time be reduced without compromising performance? What s the best way to provide feedback? Do examiners know when to ask for help? Are textbook instructions more important than experience?

81 ForensicReasoning.com THE FORENSIC REASONING PROJECT Testimony What can examiners reasonably claim when testifying in court? What s the most effective way to present forensic evidence to juries? On what basis should judges admit forensic testimony? Should examiners report opinions or statistics?

82 THE FORENSIC REASONING PROJECT ForensicReasoning.com

83 Forensic Informatics Biometric Repository Fingerprints FIB-R.com Shoeprints Voices & Handwriting Irises Faces & CCTV Footage