Scale of conclusions for the value of evidence

Transcription

1 Law, Probability and Risk (2012) 11, 1 24 Advance Access publication on October 24, 2011 doi: /lpr/mgr020 Scale of conclusions for the value of evidence ANDERS NORDGAARD The Swedish National Laboratory of Forensic Science (SKL), SE Linköping, Sweden and Department of Computer and Information Science, Linköping University, SE Linköping, Sweden RICKY ANSELL The Swedish National Laboratory of Forensic Science (SKL), SE Linköping, Sweden and Department of Physics, Chemistry and Biology, Linköping University, SE Linköping, Sweden AND WEINE DROTZ AND LARS JAEGER The Swedish National Laboratory of Forensic Science (SKL), SE Linköping, Sweden [Received on 8 April 2011; revised on 12 September 2011; accepted on 13 September 2011] Scales of conclusion in forensic interpretation play an important role in the interface between scientific work at a forensic laboratory and different bodies of the jurisdictional system of a country. Of particular importance is the use of a unified scale that allows interpretation of different kinds of evidence in one common framework. The logical approach to forensic interpretation comprises the use of the likelihood ratio as a measure of evidentiary strength. While fully understood by forensic scientists, the likelihood ratio may be hard to interpret for a person not trained in natural sciences or mathematics. Translation of likelihood ratios to an ordinal scale including verbal counterparts of the levels is therefore a necessary procedure for communicating evidence values to the police and in the courtroom. In this paper, we present a method to develop an ordinal scale for the value of evidence that can be applied to any type of forensic findings. The method is built on probabilistic reasoning about the interpretation of findings and the number of scale levels chosen is a compromise between a pragmatic limit and mathematically well-defined distances between levels. The application of the unified scale is illustrated by a number of case studies. Keywords: evidence value; ordinal scales; likelihood ratio; logical approach. 1. Introduction Forensic science is science used for the purpose of law (Caddy and Cobb, 2004). The demand for forensic science expertise has grown steadily throughout the last decades. To some extent this is due to the increased demand for the prosecution to present physical evidence, such as forensic evidence in court, as well as developments of novel techniques and introduction of forensic databases. Some disciplines such as DNA and IT forensics has experienced an exponential growth, whereas other disciplines such as handwriting at the same time experienced a diminishing demand. anders.nordgaard@liu.se c The Author Published by Oxford University Press. All rights reserved.

2 2 A. NORDGAARD ET AL. In order to be used accurately, the outcoming results of the forensic scientific analyses of traces and evidence material has first to be interpreted for evidential value which then has to be converted into phrasings standardized and understandable in the court process. One facilitator in bridging different scientific findings and make their weight transparent and comparable to the legal process is to report their weight by using a verbal scale. 2. Forensic casework Forensic casework in Sweden today is to some extent served by county police laboratories, though the bulk is served by one centralized national forensic science laboratory, SKL (Swedish National Laboratory of Forensic Science) positioned as an independent laboratory within the police. In Sweden, the disciplines of forensic medicine, forensic psychology, forensic toxicology and paternity testing are positioned outside the police forming a separate authority serving those specific demands. The police laboratories perform a limited array of forensic investigations, whereas the duties of the national laboratory cover most forensic disciplines and also include research and development of old and novel forensic skills and techniques. Thus, the vast majority of the forensic investigations requested for at SKL originate from the police and they cover investigation of any kind of evidence seized at a crime scene investigation or from individuals involved. The investigations requested for vary considerably in complexity, depending on the specific case in question. A large part of the investigations performed at the laboratory are limited to one specific or a few forensic disciplines, whereas other cases such as murder and armed robbery cases cover a broad span of expertise needed to fulfil the investigations requested. The laboratory digital case management system (LCMS) in use is modern and there is a high degree of electronic information passing to and from the police and within the different laboratory functions. More or less digitally connected to the LCMS are a magnitude of different expert systems and databases that support different expertise in use. The laboratory information management system (LIMS) created for the analyses and information flow for biological samples and DNA, in particular the high throughput chain handling reference DNA samples from sampling to DNA databasing and hit reporting, is highly developed (Hedman et al., 2008). As the laboratory delivers expert witness statements from multiple forensic disciplines to serve as part of the crime investigation report, sometimes reported in one multidisciplinary statement, a unified way to report the findings are needed. In addition, the combination of evidence from results originating from different disciplines must be achievable in a balanced way. The endpoint of any forensic crime investigation is the presentation of the forensic evidence in court. One or several forensic reports covering different areas of expertise will be part of the written evidence presented by the prosecution in the court proceedings. The forensic expert, or reporting officer, may also be required to attend court as an expert witness giving oral evidence, although this is generally not the case in Sweden. A vital part of the testimony is to present the robustness and evidential weight of the findings. 3. A unified scale of conclusions at the Swedish National Laboratory of Forensic Sciences The forensic interpretation at SKL has almost always been made according to a scale of conclusions. The wording in the scale has somewhat shown a span between the inconclusive and some type of support for an assumption there are words, intensifiers, to pinpoint a rising strength. During the 1990s,

3 SCALES OF CONCLUSIONS IN FORENSIC INTERPRETATION 3 a number of predominant intensifiers used became the skeleton of some type of a common scale of conclusions, although scales with other wordings or other levels still existed in some of the disciplines. Moreover, there was no congruence in what way the questions from the commissioner were answered or when and how an interpretation was done. The differences between identifying something, interpreting what could be source of something and interpreting some type of activity/event were also not distinct. The propositions forwarded often concerned a mix of those three issues, which led to diffuse statements, and it was frequently unclear how the outcomes of different methods interfered in the interpretation according to propositions at different levels. Should the intrinsic features of a trace and the location, number and distribution of findings have an input on the interpretation of a source or of an activity? Then, which evidence could be combined and in what way? A project was initiated during the late 1990s with the aim to decide what principle of evidence interpretation to be used, standardize a common scale of conclusions and increase the overall knowledge about the interpretation of evidence. The existing scale of conclusions today at SKL (first edition in 2005) is designed in such a way that the interpretation of the findings is made with respect to a pair of propositions, and at the same time the probabilities of the propositions themselves are not interpreted. It has also been designed to fit a logical approach using likelihood ratios (Buckleton, 2005). In our training (in interpretation) of forensic experts, it has been pinpointed as important to have a clear and transparent addressed proposition, and also to state what is included in the alternative proposition. It has further been thoroughly discussed what shall and can be used when interpreting at a source level of propositions compared to an activity level. Today we often reformulate the issue of the commissioner to distinct propositions, and try to pre-assess the value of the investigation, even before the investigation is commenced. In Fig. 1 is shown the current edition of the scale of conclusions at SKL translated into English. The scale has nine levels expressed as consecutive integers ranging from 4 to +4. The positive numbers are used when the findings are such that they are more consistent with the proposition forwarded by the commissioner (or more specifically the proposition that is the reformulation of the commissioner s question) than with the alternative proposition. The negative numbers are used for the opposite situation. Level 0 is used when the findings are (in principal) equally consistent with both propositions. To exemplify: If the commissioner asks whether a trace G originates from a potential source S, this question is typically reformulated as the proposition S is the source of G and an alternative proposition is typically formulated as Some other source is the source of G. If the findings are more consistent with the former proposition, a positive level is used, and if they are more consistent with the latter proposition then a negative level is used. The question from the commissioner does not however lead to a formulated proposition that is incriminating. As an example, the commissioner may ask: Is this passport authentic? The reformulated proposition would be The passport is authentic, but the suspicion behind the question is of course that the passport is a forgery which also becomes the alternative proposition. Thus, if the findings are consistent with a forgery, they will be reported with a negative level in the scale, which in turn shall be interpreted as support for a criminal activity. Each level of the scale comes with its number, a verbal equivalent to this number and an explanatory text (in italics in Fig. 1). The explanatory text is for most levels formulated in such a way that it should be clear that the logical approach (Buckleton, 2005) is used for evaluation. Exceptions are the levels 1, 0 and 1 where a more simple explanation is used, still not jeopardizing the meaning of the level. Note that in the scale the word hypothesis is used instead of proposition. Generally we consider these two to be synonymous expressions in the framework of forensic interpretation,

4 4 A. NORDGAARD ET AL. FIG. 1. Scale of conclusions used at The Swedish National Laboratory of Forensic Science (Statens Kriminaltekniska Laboratorium). but in the scale we have avoided to change the wording as the word hypothesis is basically the same in Swedish. In the following, we shall merely use the word proposition with some exceptions (e.g. when relating to general statistical theory). 4. Ordinal scales in forensic interpretation 4.1 Scales in general Many features in our daily environment are valued against scales of different kinds. One of the most common scales is the temperature scale. No person without knowledge about physics would be able give an objective and transparent definition of what is meant by stating that the temperature is 10 C

5 SCALES OF CONCLUSIONS IN FORENSIC INTERPRETATION 5 or 60 F, but most people would have a personal opinion about what such a statement means to her or him. If we could follow the interpretation within a person s mind we would probably find a less accurate scale with levels like extremely cold, very cold, cold, chilly, moderately chilly, moderately warm, warm, hot, very hot, extremely hot. There are several levels used with this scale, but still far from the accuracy that characterizes e.g. a thermometer. While the thermometer has what is referred to as an interval scale, the verbal levels described above constitute an ordinal scale. The difference between these two is about the distances between levels. For the interval scale, the distance is the same between two consecutive levels no matter where in the scale the measuring is done. For the ordinal scale, the corresponding distances (may) vary. Ordinal scales may be easier to use since the levels are limited in number, but the interpretation often suffers from subjectivity. This is particularly true for different types of grading scales, such as course grades, DNA quality grades, wine tasting grades etc. Hence, to make an ordinary scale useful, there is need for careful instructions about how to select levels in a particular situation. Ordinal scales are frequent in the area of jurisdiction and forensic interpretation of evidence. For instance, the (Swedish) Police use a scale to express in what degree of suspicion an individual is arrested or remanded in custody for being the perpetrator of a crime, with the levels identification, reasonable suspicion (lower degrees) and probable cause, objectively grounded suspicion (higher degrees). At the other end of the legal process, the court statements would be dichotomous in the sense that they would either convict beyond reasonable doubt or acquit with respect to insufficient evidence. One would expect that in a particular case, the level of such a scale is selected with almost no element of subjectivity, but there is yet no guarantee that two cases that are in principle identical would result in identical levels. The science of law is not exact and the use of scales therein cannot be compared with the selection of scale levels within e.g. physics or chemistry. The ordinal and interval scales are the most frequent ones for evaluation purposed to judge upon whether a particular value should be considered low or high (or anything in-between). For classification purposes, there is also the nominal scale, which completely lacks numerical relationships between the levels (e.g. nuances of paint). Scientific measurements are most often given on a ratio scale in which there is a well-defined zero value allowing us to compare values on a relative basis. These two scales, however, are not in focus for the evaluation of evidence and will therefore not be considered in the current paper. 4.2 Scales of conclusions Forensic science takes an intermediate position in that it comprises several parts from various disciplines, more or less exact. Ordinal scales of conclusions are practised within forensic interpretation objecting to present the findings in a form that is free from statements and terminology requiring knowledge within such different scientific areas. In particular, statements regarding forensic evidence will by natural reasons contain probabilistic reasoning, but probabilities themselves may be very difficult to communicate outside the field of expertise. Therefore, the selection of wording in an ordinal scale of conclusions for forensic evidence evaluation becomes a challenge. We shall illustrate this through an example. At a forensic laboratory, one might have come to the conclusion that a particular piece of evidence is consistent with a statement forwarded by the police, for instance that a glove was worn by a particular suspect. Let us assume that the police bring the glove and some biological samples (hairs, blood etc.) from the suspect to the forensic laboratory along with the question Was this glove worn

6 6 A. NORDGAARD ET AL. by the suspect?. Without careful thinking, it might be close to express the findings like There is a high probability that the glove was worn by the suspect since this in fact is a direct answer to the question. However, such a statement is not correct as value of evidence but as a statement of the case once the forensic findings have been applied to the prior beliefs about whether the suspect wore the glove or not (cf. Aitken and Taroni, 2004). It is not up to the forensic examiner to give statements about the case since he or she should not possess any prior beliefs. Now let us alter the statement so it reads, Our findings give that the probability that the glove was worn by the suspect is high. A person inexperienced with probability reasoning might find the meaning of the latter statement different from the meaning of the former, but they of course mean the same, and it is only the wording that is different. To change the meaning the word probability (or a synonymous term or expression) must go for the findings and not for the case. Consider the following two expressions: (i) Our findings are highly probable if the glove was worn by the suspect. (ii) There is much higher probability to obtain our findings if the glove was worn by the suspects than if it was not. What is the main difference between these two statements? Both statements are consistent with the way evidence evaluation is pursued as they both express probabilities for the findings and not for the statements forwarded by the police (or by a prosecutor). However, (i) is not complete since it only concerns the probability of the findings if the suspect actually wore the glove. There is no value in this statement useful to the court because the probability of the findings might be equally high if the suspect did not wear the glove. Statement (ii) concerns two probabilities for the findings, one given the glove was worn by the suspect and one given it was not. Thus, the findings have a value that can be related to prior beliefs about the case. The example shows that care must be taken what the word probability goes for but in addition the statement cannot be built on a single probability. This in turn implies that the probability scale cannot be used directly for evidence values, which might sound like a drawback keeping in mind that the inclusion of technical evidence in a case requires probabilistic reasoning. However, it is not until the evidence has been applied to the prior beliefs that statements may be forwarded with a degree of (un)certainty; in other words, that the final statement is true up to a probability. As we shall discuss below, statement (ii) is built up from the ratio of two probabilities and such a ratio has another scale than has the probability scale. It is seldom easy to interpret a statement telling how much higher is the probability of the findings given that the glove was worn by the suspect compared with the case were it was not. For instance, what would we mean by stating that the findings are times more probable if the glove was worn by the suspect than if it wasn t? To make the statement more interpretable, we must translate it from wording with probabilities to wording with more ordinary expressions. The statement (ii) may for example be alternatively expressed as (ii*) Our findings strongly support that the glove was worn by the suspect. Note that this wording might be interpreted as if we were referring to the case and not the evidence. The expression is then confused with something like Our findings give with high probability that the glove was worn by the suspect. Nevertheless, there is a distinction between the expressions strongly supports and gives with high probability. The former means that looking at the evidence from different perspectives (i.e. wore or wore not the glove), we consider the findings to be

7 SCALES OF CONCLUSIONS IN FORENSIC INTERPRETATION 7 (much) more expected if the glove was worn by the suspect and thus makes that proposition the best explanation for the findings. The latter means that we have come to a conclusion about the state of the proposition with a high probability. Once we have selected a system for rephrasing statements originally involving probabilities, as is the case when translating (ii) to (ii*), we can construct an ordinal scale within this system. A statement representing a lower value than does (ii*) can be Our findings support that the glove was worn by the suspect and with an even lower value it can be Our findings support to some extent that.... Corresponding expressions amplifying the degree of support will give higher levels on the scale. The resulting scale, i.e. the collection of levels, becomes ordinal, since there is no possibility to select expressions that have numerically meaningful distances (that would be normalized) unless the expressions are just verbal counterparts to real numbers. The number of levels that should be defined on such a scale depends to a great extent on how the scale is adopted among jurors, prosecutors, defence attorneys and the police. Usually there must be a multistage procedure of developing a scale, including several propositions, feedback and revision before the final scale is settled. 4.3 Issues of interpretation Interpretation of verbal expressions for the value of evidence has previously been studied. Sjerps and Biesheuvel (1999) made an experiment in which they studied jurists opinions about two different scales. The first scale was the one currently used at that time within The Netherlands Forensic Institute. This scale used expressions that from a probabilistic point of view related more to the case and less to the evidence. The second scale was a suggested alternative in which the formulations were in line with the expressions (ii) and (ii*) above, i.e. the scale levels related to how probable the findings were under the assumption that one proposition was true compared with how probable they were under other assumptions. The results nevertheless showed that jurists in general had problems to understand the necessity of the second scale and preferred the first one. Broeders (1999) addresses the same type of problem, but by studying in which way forensic scientists report and understand their reporting of their findings. Recently, SKL has conducted a survey among different actors in the Swedish judicial process (Nordgaard et al., 2010) about their perceived strength of different phrases (in Swedish) used as amplifiers of the support. The results from this survey show that there is substantial variation among the actors in perceived strength but also in their final opinion about a particular case when a certain phrase have been used in the evaluation statement for the technical evidence. For instance, wording that from a linguistic perspective should mean that the support in practise implies certainty about the case was by many respondents interpreted as just moderately strong. So far we have not included probabilistic formalism into the discussion. This is partly due to the fact that mathematics is problematic to use in the interface between the forensic laboratory, which to a large extent uses scientific methods originating in natural and mathematical sciences, and the judicial community (police, prosecutors, lawyers and judges), where such scientific methods may be very hard to understand. Later, we will shortly review and use results from probability theory to explain how findings can be translated to an ordinal scale, but for the moment we may say that the development of ordinal scales for forensic interpretation is thoroughly built on correct probabilistic reasoning and in addition on how relations between probabilities can be interpreted. This is a work that needs to be done by forensic scientists, with sufficient knowledge about probabilistic reasoning and its applicability in forensic biology, chemistry, informatics etc., but at the same time integration with the judicial community is by all means necessary.

8 8 A. NORDGAARD ET AL. 5. Reporting forensic findings on the scale of conclusions 5.1 The likelihood ratio The state-of-art in forensic interpretation is to evaluate forensic evidence with the use of a likelihood ratio (cf. Aitken and Taroni, 2004). The likelihood ratio expresses the relative strength of the evidence in the comparison of one proposition, often referred to as the prosecutor s hypothesis, against another referred to as the defence s hypothesis. The former will hereafter be denoted H P and the latter H D. It should be mentioned that the terms prosecutor and defence should not always be literally interpreted, even if this is the case when a particular piece of evidence is handled within the courtroom. One objective of the current paper is to show how a scale of conclusions can be developed within a forensic laboratory primarily working with evidence material from crime (scene) investigations. However, the body of commission is usually the police and most of the evidence evaluation addresses source level propositions. The prosecutor s hypothesis may therefore very well be a reformulation of the question (task) forwarded by the police into a proposition (cf. Section 3) and the defence s hypothesis is the (natural) alternative to that proposition. In other areas of evidence evaluation, these two hypotheses are just two disjoint alternatives where the task is to decide upon which of these is the one most probable. The likelihood ratio used in evidence evaluation is a particular component in the theory of Bayesian hypothesis testing, in which the hypotheses are evaluated by the so-called Bayes factor (cf. Berger, 1985). For a pair of mutually exclusive and exhaustive hypotheses (H P, H D ) the prior odds of H P to H D is the ratio Pr(H P )/Pr(H D ) and the posterior odds of H P to H D given the observed data E is the ratio Pr(H P E )/Pr(H D E ). The Bayes factor, B, is defined as the ratio of the posterior odds to the prior odds, i.e. B = Pr(H D E)/Pr(H D E). (5.1) Pr(H P )/Pr(H D ) If both hypotheses are simple, i.e. each hypothesis depicts a fixed scenario for which a prior probability can be assumed, the Bayes factor simplifies to a likelihood ratio: L(H P ; E) = V, (5.2) L(H D ; E) where the likelihoods L(H P ; E) and L(H D ; E) measure how likely the observed data are (relative to other data) under the assumptions that H P and H D, respectively, are true. For evidence evaluation, E stands for the evidence itself, or the findings from a case. Depending on the scale of the measured data of these findings, the likelihood is either a distinct probability or a value from a probability density function. The former case is the one mostly appearing in forensic literature and the likelihood ratio can then be written V = Pr(E H P) Pr(E H D ). (5.3) As a simple example of the latter, consider a case where the piece of evidence is a digital image of a disguised person and the question is about whether it is a man or a woman. The forensic method used is estimating the length of the person from the image. Let us say the length estimate is 177 cm. This scale of measurements is not of the kind that we may address distinct

9 SCALES OF CONCLUSIONS IN FORENSIC INTERPRETATION 9 probabilities to each value. Instead utilization of the probability densities of lengths for males and females must be used. Depicting the former f M (x) and the latter f F (x) the applicable likelihood ratio is V = L(H P ; E)/L(H D ; E) = f M (177)/ f F (177). Equating the Bayes factor with the likelihood ratio, (5.1) can be rewritten as Pr(H P E) Pr(H D E) = V Pr(H P) Pr(H D ) (5.4) (also known as Bayes theorem on odds form). A likelihood ratio greater than one will thus make the posterior odds higher than the prior odds, while the opposite holds for a likelihood ratio less than one. In a forensic case where V is of the kind (5.3) and greater than one the test result can be expressed as the probability of the findings when H P is true is V times higher than the probability of the findings when H D is true, but this expression is less proper when V is a ratio of evaluated densities. A more general expression would be the findings increase the prior odds of H P to H D with a factor V and such an expression also moves the focus from discussing probabilities in court to the discussion of supporting evidentiary strength. Moreover, the last expression is also consistent with the general case of Bayesian hypothesis testing, i.e. without substituting the likelihood ratio for the Bayes factor. One example where the Bayes factor cannot be separated from the prior odds (as is the case when it is replaced by the likelihood ratio), is where H P states that the donor of a blood stain is X and H D states that the donor is either the brother of X or another person not related to X. The evidence is a match between the DNA profile obtained from the stain and the DNA profile of X. Here H D is the union of two mutually exclusive propositions with different likelihoods and possibly different prior probabilities, and therefore the likelihood ratio (as expressed in (5.2)) is not the equivalent of the Bayes factor (cf. Buckleton et al., 2006). The more general view of interpreting the likelihood ratio (or the Bayes factor) as how much the prior odds is increased (or decreased) is supportive of the way evidence evaluation should be reported to the commissioner (e.g. in court) when the likelihood ratio cannot be computed explicitly, but estimated to a level of magnitude. Instead of interpreting the likelihood ratio as how more (or less) probable are the findings given one proposition is true than given the other is true, we can say that the findings support one of the propositions and thus imply an amplification (or attenuation) of the prior odds to get the posterior odds. In the absence of an explicit numerical likelihood ratio, the degree of support can be given on an ordinal scale. 5.2 From a likelihood ratio to a scale level Once the likelihood ratio (or the Bayes factor when they are not equivalents) has been obtained, it should be interpreted on the scale of conclusions used. Evett et al. (2000) suggested a scale where a likelihood ratio between 1 and 10 would be reported verbally as Limited evidence to support, while a likelihood ratio above would be reported as Very strong evidence to support. DNA evidence is probably still the area of application where a likelihood ratio can be reported on almost a continuous scale, and it is far known that calculated likelihood ratios to a vast majority would exceed the value of The Netherlands Forensic Institute uses a scale (NFI, 2008), in which the findings (analogously to Evett et al., 2000) are reported to support with different degrees the statement in hand. The scale currently used at SKL (Fig. 1) is analogous to the ones in Evett et al.

10 10 A. NORDGAARD ET AL. (2000) and NFI (2008), with the exception that the statement for which the level is reported is always the proposition advanced in the forensic examination. This means that likelihood ratios above and below one are differently interpreted on the scale. For the other scales mentioned above only likelihood ratios greater than one are interpreted and the proposition supported may vary from case to case. Consider the scale of Fig. 1. To simplify, we will hereafter refer to this scale as the SKL scale. There are nine levels used but the levels 1, 2, 3 and 4 can be seen as mirrors of the Levels +1, +2, +3 and +4 since shifting the propositions would change a level Lto level +L and vice versa (L = 1, 2, 3, 4). Rules for translation of obtained likelihood ratios to the scale can therefore be developed for the Levels 0, +1, +2, +3 and +4 and be applied either directly to the likelihood ratio or to its inverse. One might ask why the scale is not constructed with only five levels, but the current reporting system at the laboratory is such that findings consistent with what could be the prosecutor s hypothesis should be reported with positive levels, and findings not consistent with that proposition should be reported with negative levels. Translation now means dividing the range from one to infinity of possible likelihood ratio values into five intervals corresponding with the Levels 0, +1, +2, +3 and +4. It might be argued that Level 0 should be used solely for the case where the likelihood ratio is exactly one, but here and in the forthcoming we must remember that a likelihood ratio obtained in a particular case is most often a point estimate with the amount of uncertainty it may contain (i.e. the likelihood ratio used is an approximation). Thus, an interval for Level 0 is also motivated and the lower limit of this interval is naturally one (for likelihood ratios greater than or equal to one). Each individual may have their own opinions about how intervals should be chosen to be consistent with the verbal expressions used in the SKL scale, and there is no unique mathematical rule for doing it. In a forensic laboratory, the choice of a unified translation system is therefore a question of compromising between the different opinions within and outside that laboratory. We will however suggest a statistically assisted choice based on Bayes theorem and some common opinion about uncertainty. Assume that one piece of evidence is the component that should decide whether a suspect should be convicted or acquitted. The evidence value is then combined with the prior odds of guilt to obtain the posterior odds of guilt. What is the lowest posterior odds that can lead to conviction? This question has naturally no distinct answer, but reasoning about it is important for the construction of intervals for the likelihood ratio. Ceci and Friedman (2000) discuss this topic in terms of the social cost of conviction. One conclusion is that the celebrated statement by the British judge Sir William Blackstone ( ) that it is better that ten guilty persons escape, than that one innocent suffer should be updated, and if numbers are to be set out it would be more consistent with today s practice in conviction beyond reasonable doubt to substitute 99 for 10 in Blackstone s statement. Adopting this, a posterior odds of 99:1 or equivalently a posterior probability of 0.99 is considered to be high enough to say that there is support for adopting the corresponding proposition. The same argument was used by Thompson et al. (2003) when discussing how the probability of false positives affects the posterior probabilities in DNA cases. Following these arguments, we suggest that Level +2 of the SKL scale ( support without attributes) should be reached when the likelihood ratio is greater than or equal to a value that on the average would give a posterior probability of at least 0.99 for the proposition forwarded. To explain this more thoroughly, consider again Bayes theorem on odds form as presented in (5.4). If we assume H P and H D to be complementary propositions, the probability ratios of that

11 SCALES OF CONCLUSIONS IN FORENSIC INTERPRETATION 11 expression are true odds and we may rewrite the relationship as Pr(H P E ) 1 Pr(H P E ) = V Pr(H P ) 1 Pr(H P ) = V Odds(H P), (5.5) where Odds(H P ) stands for the prior odds of H P. From (5.5) we deduce the posterior probability of H P as Pr(H P E) = V Odds(H P) V Odds(H P ) + 1. (5.6) On the average may now be represented by a prior odds equal to one, which changes (5.6) into Pr(H P E ) = V V + 1. (5.7) Another argument for temporarily setting the prior odds to 1 is that it is the case where the posterior odds are solely determined by the value of evidence. It therefore seems natural to normalize the scale at this point. For the posterior probability to be at least 0.99, the likelihood ratio V must be at least 99. This is however an odd number and the choice of 100 is easier to communicate. Hence, the lower limit of the interval interpreted as Level +2 on the SKL scale is settled. The interval corresponding with Level +4 is by natural reasons open-ended towards infinity. The current practice up till now for reporting DNA evidence values on the scale has been to require the random match probability to be less than one in a million to reach Level +4. This choice depends on the sizes of the populations of potential perpetrators involved in Swedish crimes. We have no reason today to question this choice and therefore the lower limit for the interval interpreted as Level +4 of the SKL scale is set to one million. The corresponding lowest posterior probability with a prior odds of one is , which is far beyond any debate about the uncertainty. This anchoring of the lower interval limits for the scale Levels +2 and +4 will now be used to construct the complete interval division of the range one to infinity of likelihood ratio values. The first two interval limits, i.e. 1 and 100 are exactly the scale levels exponentiated with base 10. It is then close to think of the scale as a logarithmic one even if the third interval limit (i.e. one million) does not fit into this. In another perspective, we would strive for coming above Level 0 for evidence that clearly puts one proposition in favour of another even if we consider the evidence value not to be strong. This is particularly important for evidence primarily analysed for intelligence work. As a consequence, the interval lengths must increase with increased level. They certainly do with a full Briggsian 1 logarithmic scale, i.e. logarithms with base 10, but using all consecutive numbers in such a scale would require seven levels before the interval limit one million is reached. To avoid that many levels, we suggest the following: The increase in Briggsian logarithm of two consecutive lower interval limits should be (approximately) proportional to the level corresponding with the higher of 1 The logarithm with base 10 is often referred to as the Briggsian logarithm after the British 17th century mathematician Henry Briggs ( ).

12 12 A. NORDGAARD ET AL. the two limits. We can write the intervals and corresponding scale levels as 1 V < R 1 : 0 R 1 V < R 2 : +1 R 2 V < R 3 : +2 R 3 V < R 4 : +3 R 4 V < : +4, (5.8) where we have already fixed R 2 to 100 and R 4 to The lower limit 1 for the interval corresponding with Level 0 is as natural as is the upper undefined limit ( ) for the interval corresponding with Level +4. Thus, the first and the last interval are left out of this construction, and we require log 10 R i log 10 R i 1 k i; i = 2, 3, 4. (5.9) With R 2 = 100 and R 4 = 10 6, a solution to (5.9) is found in which k 0.5, R and R Now what do these values imply for the posterior probabilities? With prior odds equal to one (as before), we obtain the lowest posterior probabilities for Levels 1 and 3 as and , respectively. The former should be consistent with that the findings support H P to some extent according to the SKL scale (Fig. 1). Any probability above 0.5 would be consistent with that expression depending on how some extent is interpreted, but of course would not be a debatable value in that sense. The latter probability, i.e , is quite high but still difficult to interpret from general probability reasoning. Although it is not appropriate to compare posterior probabilities with P values from traditional frequentistic approaches to hypothesis testing, we may reflect on common discussions about such quantities. In general a P value below 0.05 is interpreted as significant evidence and a P value below 0.01 as strong significant evidence. In medical studies on effects of drug therapies, P-values below are considered to give very strong statistical evidence of effects. In the view of this, a posterior probability of at least would be consistent with the expression strong support as given in the SKL scale (Fig. 1). It may be argued that the derived lower limits (1), 5.625, 100, 5625 and 10 6 are impractical from a numerical point of view. Rounding the numbers upwards, i.e. substituting 6 for and 6000 for 5625 gives a more convenient representation. The resulting minimum posterior probabilities with prior odds equal to one will then be and (the latter is slightly changed although not visible until the fifth decimal). The increase in logarithms of lower limits between two consecutive levels will no longer be proportional to the higher of the two limits but fairly close. The resulting translation table between values of V and scale levels (both sides of 0) is given in Table 1. In Figure 2 are plotted lowest posterior probabilities against prior odds for the five scale levels 0 to +4 (equating each level as the lowest possible likelihood ratio). 5.3 Reporting on the scale without reference data Forensic scientists may be sceptical to the logical approach of evidence evaluation. This is so because a majority of forensic cases are still such that no explicit reference data exist with which estimations of likelihoods can be done. It is however important to realize that the lack of explicit reference data do not disqualify the use of the logical approach, not even the use of a likelihood ratio. The latter

13 SCALES OF CONCLUSIONS IN FORENSIC INTERPRETATION 13 TABLE 1 Intervals of V (likelihood ratios) and corresponding scale levels Interval Scale level V < V 1/ /6000 < V 1/ /100 < V 1/6 1 1/6 < V < V < V < V < V +4 FIG. 2. Posterior probabilities plotted versus prior odds for each of the Levels 0, +1, +2, +3 and +4 of the SKL scale of Fig. 1. Calculation of posterior probabilities were done with a likelihood ratio equal to the lower limit for the likelihood ratios for each level. always exists for simple propositions and the problem lies in its estimation. By explicit reference data, we mean an objectively compiled database of observations of the same kind as our findings for the current piece of evidence, and with enough background data to make it possible to classify these observations with respect to the propositions in question. However, reference data may still exist although not formally stored in a database. An eyewitness may state that the person he saw was a man, implicitly saying that it was not a woman. How is it possible to give such a statement? There are several possibilities, but one may be the case of exclusion: If that person was a woman I wouldn t have expected her to have the skull shaved, that s why I think it was a man. Note that this way of thinking is the same as what is promoted with the logical approach. His findings consist of the shaved skull and these findings are in his opinion more probable was it a man that was it a woman, although he merely uses the conditional probability of his findings given it was a man (i.e. the numerator of the likelihood ratio). To be fair, however, his thinking could have been the opposite: The person s skull was shaved, thus I think it was a man. The latter is a statement corresponding with a conditional probability that the person was a man given his findings and would be the numerator of the posterior odds.

14 14 A. NORDGAARD ET AL. The important issue of the previous example is that reasoning with probabilities or likelihoods can be done without any explicit databases. The forensic scientist has, similar to the eyewitness, an experience that allows her to make likelihood statements with a precision depending on this experience. The statements will to some extent be subjective, yes, but so are the statements of the eyewitness. The difference is that while the eyewitness bases his conclusions upon what may be referred to as common experience, the expert witness bases her conclusions on specific experience. The logical approach still works no matter if the conclusions are based on an objective database or on specific experience as long as the evidence value is in the form of a likelihood ratio. The forensic scientist should consider the two competing propositions, and based on her experience both from historical cases and from the field of expertise itself she should find out the degrees of expectation of the findings under each of the propositions. The proposition under which the degree of expectation is maximized is the proposition that is supported by the findings. The degree of support is based on how much more expected are the findings under that proposition than they are under the other proposition. This judgement is the evidence value and can be reported on the scale and is at the same time the best estimation at hand of the underlying likelihood ratio. It is possible that neither of the propositions is particularly supported by the findings, or that both propositions are approximately equally supported. The evidence value should then be reported with 0 on the scale. This case is probably the simplest to handle at the forensic laboratory since referencing to general uncertainty may serve as an argument for the stated degree of support. What about all cases where support has been concluded? As soon as the findings give more support to one of the propositions, and this cannot be treated as just having occurred by chance, the forensic scientist should report the evidence value at least at Level +1 or Level 1 of the scale depending on which of the two propositions gets the more support. For sake of simplicity, we will from now on only discuss reporting on the positive side of the scale keeping in mind that the negative side can just be thought of as a mirror of the positive side. Equating the Level +1 to a likelihood ratio of at least 6 according to Table 1 means that we state that the findings support the forwarded proposition at least six times more than the alternative proposition. It might be argued that 6 times more is too strong to be consistent with the argumentation above stating that +1 should be used whenever the indicated support is not just a matter of chance. However, consider the following simple example: A 12-sided dice is suspected to be false in that sense that it gives the result twelve every second roll, while each of the other 11 results are equally likely (i.e. comes each with probability 1/22). The alternative proposition says the dice is balanced. We roll the dice once and obtain twelve. The likelihood for the proposition of a false dice is then 1/2 and the likelihood for the proposition of a balanced dice is 1/12. Thus, the likelihood ratio becomes exactly 6 and our findings support the proposition of a false dice six times more than they support the alternative proposition. However, if our result had been e.g. five, the likelihood ratio would have been 0.54, which supports the proposition of a balanced dice about 1.8 times more than the proposition of a false dice. This example is of course not comparable with a typical forensic case but still illustrates that a likelihood ratio of 6 is not at all too large to harmonize with the scale Level +1. No jury or court would come to the verdict that the dice is false just on the basis of one roll, but we cannot ignore that the only findings in the case, i.e. the observed twelve, give support to some extent for that proposition. The findings may have occurred by chance but that is not the only explanation. The highest level of the scale, i.e. +4, might from a first point of view be impossible to choose without explicit reference data. This is however not true, and this level may on the contrary be quite

15 SCALES OF CONCLUSIONS IN FORENSIC INTERPRETATION 15 natural under the right conditions. The forensic scientist still has two propositions (H P and H D ) to consider. In a case where she believes that the findings cannot by any means be obtained given H D is true, while they very well can if H D is true, the likelihood of the former proposition is very close to zero, and the resulting likelihood ratio must be so large that scale level +4 is reached (in support of H P ). From a philosophical point of view, this situation resembles the one where it can be stated as a fact that one of the propositions is false, i.e. the so-called leap-of-faith has disappeared. The question whether a scale of conclusions should be used in such cases is of course interesting but is out of the scope of the current paper. Depending on how certain the forensic scientist is about the impossibility of obtaining the findings under one of the propositions, the leap-of-faith is more or less pronounced. The two remaining levels, i.e. +2 and +3 are the ones most difficult to decide upon in a particular case (without explicit reference data). Besides long experience within her own field of expertise, the forensic scientist needs to calibrate her opinions against casework from other areas. There are a lot of similar situations in the various fields of forensic interpretation and within a laboratory you should benefit on this. At SKL, we are practising a system of calibration meetings, where forensic scientists from different fields discuss each other s cases. A case is prepared in such a way that you clarify what are your propositions, what are your findings, what are your conclusions and why. This should be done in such a way that e.g. a person trained in forensic biology could understand a case from forensic informatics and vice versa. By presenting your own case, assimilating questions from your colleague and then doing the opposite procedure the general likelihood reasoning about your own cases is enhanced. 6. Examples from casework at SKL We will here present three typical cases from daily work at the laboratory to illustrate the use of the scale of conclusions. The first case is about forensic interpretation of glass fragments and shows how the scale is used when the likelihood ratio can be estimated on the basis of reference data. The second case is about comparison of paints and illustrates how evidence evaluation is made and how the scale is used without the access to reference data. The third case comes from DNA analysis and shows a combination of evidence evaluation with and without reference data. We would like to make clear that these examples illustrate the current practise at the laboratory, and our objective is not to communicate recommendations about how particular cases should be handled with a scale of conclusions, merely to show how this handling may work. 6.1 A glass case Background. A side window of a car has been broken in a smash-and-grab incident. The police later seized two people as suspects of the incident. Control glass from the broken window was collected and the suspects jackets were sent for investigation Investigation and results. From the jacket of Suspect 1 about 20 glass fragments were recovered. The glass fragments were green-tinted tempered float glass with a thickness of 3.16 mm. Four of the fragments were examined by means of refractive index (RI) with a GRIM instrument (GRIM) before (RI before ) and after (RI after ) annealing. The following data were obtained: (RI before ) , (RI after ) The elemental composition of one of the recovered glass fragments was also examined by means of scanning electron microscope equipped with an energy dispersive X-ray