Food Quality and Preference

Transcription

1 Food Quality and Preference 19 (8) Contents lists available at ScienceDirect Food Quality and Preference journal homepage: Replicated triangle and duo trio tests: Discrimination capacity of assessors evaluated by Bayes rule S. Bayarri, I. Carbonell, L. Izquierdo *, A. Tárrega Instituto de Agroquímica y Tecnología de Alimentos, CSIC, P.O. Box 73, 461 Burjassot (Valencia), Spain article info abstract Article history: Received 5 September 7 Received in revised form 1 February 8 Accepted 3 February 8 Available online 4 March 8 Keywords: Replicated difference tests Bayesian inference Discrimination rates of panellists performing replicated difference tests are estimated in the present paper according to Bayes rule by considering the successive replications as different steps and using the posterior distribution obtained in each step as prior distribution of the following step. Data are also successively obtained in real situations and, thus, this approach imitates what happens in practice and differs from other published approaches which consider all replications as if they were simultaneously observed. Before performing the first step, a non informative prior distribution is used as density function of the discrimination rate but after the first step has been completed each prior distribution (posterior distribution obtained in the previous step) is informative. The density functions of the discrimination rates are not proper beta distributions but the sections of beta distributions corresponding to the interval of the independent variable from 1/3 to 1 in triangle tests and from 1/ to 1 in duo trio tests. Ó 8 Published by Elsevier Ltd. 1. Introduction Difference tests are widely used in food analysis and the statistical approaches used to interpret their results have been object of many papers. We recommend the book from Bi (6) as an excellent review in this area. The purpose of this paper is to study a very specific aspect related with discrimination rates of panellists and with independency among trials performed by a panellist. Thus, only literature related with these points will be commented. We refer to the above mentioned book for details that do not directly affect our objectives. Under the null hypothesis of no differences between the two tested products each particular answer to a difference test follows a Bernoulli distribution with parameter p (probability of success p = 1/3 for triangle tests, p = 1/ for duo trio tests). Thus, the results from n trials follow a binomial distribution with parameter p. This holds whatever the number of trials a panellist can perform. But if differences between the tested products exist the interpretation is not so simple. A way (see comments from Bi, 3) isto consider that the probability of success in each answer is a value higher than p but constant for all answers. This implies the assumption that all panellists and all replications from a panellist are interchangeable. Nevertheless, when differences exist between the products it is not logical to accept that the capability of noticing them is exactly the same for all panellists. It seems more reasonable to accept discrimination rates, h, different among * Corresponding author. Tel.: ; fax: address: luisiz@iata.csic.es (L. Izquierdo). panellists, ranging in the interval from to 1. Thus, the probability conditioned to h of success in a single response of a panellist is h +(1 h)p. A simple approach (see Meilgaard, Civille, & Carr, 1999) is to assume that panellists are divided in two groups, one formed by individuals who distinguish and always succeed in their answers and another one by non-discriminators that only succeed by chance. In terms of discrimination rates, h = 1 for all panellists of the first group and h = for all panellists of the second group. This assumption may be reasonable when the panellists perform a unique test and the only available result from each panellist is either a single success or a single failure. Carbonell, Carbonell, and Izquierdo (7) applied this criterion to estimate the number of true discriminators according to Bayes rule. But when replications from a panellist are available and successes and failures are observed, a panellist cannot be classified either as a complete discriminator or a non-discriminator. It is more rational to accept that the discrimination rate is an unknown value between and 1 to be estimated according with the observed number of successes and failures. Kunert (1) considered h a constant parameter for each panellist whereas Brockhoff (3) considered h a random variable with a beta distribution. Bi (7) also assumed a beta distribution for h and used a Bayesian approach to estimate the probability of success. Duineveld and Meyners (8) applied a hierarchical Bayesian approach for the same purpose. When analysing replicated data from a panellist it is common to treat the observed results, s successes out of n trials, as if they were simultaneously obtained, but it is obvious that the results were obtained step by step and in a given order, for instance a success /$ - see front matter Ó 8 Published by Elsevier Ltd. doi:1.116/j.foodqual.8..8

2 5 S. Bayarri et al. / Food Quality and Preference 19 (8) after a failure after a success. Bayesian inference performs in a similar way, since a prior probability is combined with data obtained in each step to infer the posterior probability, which, in its turn, will be used as prior probability for the next step. The purpose of this paper is to apply this criterion to estimate the discrimination rate of individual panellists according to the successive observed successes and failures in a series of difference tests. The overall discrimination power of the whole panel is then determined as a function of the individual discrimination rates. pðsþ ¼ pðs; hþdh ¼ pðs=hþpðhþdh The probability of failure can also be obtained by integration pðf Þ¼ or just as pðf Þ¼1 pðsþ pðf ; hþdh ¼ pðhþdh ¼ ð1 pðs=hþþpðhþdh ð5þ ð6þ ð7þ. Assumptions.3. Tested samples.1. Conditional probabilities of success and of failure Conditioned to a given discrimination capacity (h) of the panellist the probability of success in a trial is, as mentioned above pðs=hþ ¼h þð1 hþp ¼ p þð1 pþh ð1þ where h is the discrimination capacity, considered continuously distributed in the range from to 1, and p is the probability of success just by chance, 1/3 for triangle tests and 1/ for duo trio tests. The probability of failure, also conditioned to a given discrimination capacity of the panellist, is ¼1 pðs=hþ ¼ð1 pþð1 hþ The probability of success conditioned to h is always given by Expression (1)and, thus, pðs 1 =hþ ¼pððs =s 1 Þ=hÞ ¼pððs =f 1 Þ=hÞ ¼¼pðs=hÞ ¼p þð1 pþh ð3þ where s 1 means success in the first trial, s /s 1 success in the second trial given success in the first one and so on. In the same way pðf 1 =hþ ¼pððf =s 1 Þ=hÞ ¼pððf =f 1 Þ=hÞ ¼¼ ¼ 1 pðs=hþ ¼ð1 pþð1 hþ.. Marginal probabilities of success and of failure The total probability of success of a panellist in a trial is obtained by integration over all possible values of h from to 1 ðþ ð4þ The discrimination rate of a panellist depends on her/his capability but also on the real differences between the samples tested. If both are identical h is obviously and if they are completely different h is 1 or very close to 1. In the theory discussed in this paper the real differences between the two tested samples is not taken into account but it is assumed that all panellists test the same samples in all repeated trials. 3. First trial 3.1. Marginal probabilities of success and of failure Before the panellist performs the first trial, and if no previous knowledge about the discrimination rate exists, it is reasonable to accept a uniform prior distribution of h from to 1 or, in other words, a constant value for the density function p(h) of the distribution of h. In these conditions Expression (5) applied to the first trial becomes ¼ pðs 1 =hþpðhþdh ¼ ct ¼ ct 1 þ p pðs=hþdh ¼ ct ðp þð1 pþhþdh since p(h) is a constant value and p(s 1 /h)=p(s/h)=p + (1-p)h (Expression (3)). In the same way, Expression (6) becomes pðf 1 Þ¼ct ð1 pþð1 hþdh ¼ ct 1 p Table 1 Probabilities of success and failure in three successive triangle tests performed by a panellist First test a Second test Third test Success /3 Given success in the first test Success 39/54 Given two previous successes Success.769 Failure.31 Failure 15/54 Given a previous success and a previous failure Success.6 Failure 1/3 Given failure in the first test Success 3/54 Failure.4 Failure 4/54 Given two previous failures Success.5 Failure.5 a Assuming a uniform prior distribution of the discrimination capacity of the panellist. Table Probabilities of success and failure in three successive duo trio tests performed by a panellist First test a Second test Third test Success 3/4 Given success in the first test Success 7/9 Given two previous successes Success.84 Failure.196 Failure /9 Given a previous success and a previous failure Success.687 Failure 1/4 Given failure in the first test Success 6/9 Failure.313 Failure 3/9 Given two previous failures Success.65 Failure.375 a Assuming a uniform prior distribution of the discrimination capacity of the panellist.

3 S. Bayarri et al. / Food Quality and Preference 19 (8) The value of the constant must be 1 to satisfy p(s 1 )+p(f 1 )=1. Thus, p(s 1 )=(1 + p)/ and p(f 1 )=(1 p)/. Table 1 lists these probabilities of success and failure for triangle tests and Table for duo trio tests. 3.. Posterior distributions of the discrimination rate According to Bayes rule the posterior density function of h given a success in the first trial is pðh=s 1 Þ¼ pðs 1=hÞpðhÞ ¼ pðs=hþ ¼ p þð1 pþh 1þp This is a linear function of h that in triangle tests (p = 1/3) simplifies to 1/ + h and in duo trio tests to /3(1 + h). In the same way, given a failure in the first test the posterior density function of h is pðh=f 1 Þ¼ pðf 1=hÞpðhÞ ¼ pðf 1 Þ 4. Second trial pðf 1 Þ ð1 pþð1 hþ ¼ ¼ ð1 hþ 1 p 4.1. Marginal probabilities of success and of failure In the second test the posterior distribution of h obtained from the first test, p(h/s 1 ) in case of success, p(h/f 1 ) in case of failure, is used as prior distribution. In both cases, success and failure can be observed. The possibilities are, thus, success after success (s / s 1 ), success after failure (s /f 1 ), failure after success(f /s 1 ) and failure after failure (f /f 1 ). Adapting Expression (5) to these cases pðs =s 1 Þ¼ pðs =f 1 Þ¼ pðf =s 1 Þ¼ pðf =f 1 Þ¼ pððs =s 1 Þ=hÞpðh=s 1 Þdh pððs =f 1 Þ=hÞpðh=f 1 Þdh pððf =s 1 Þ=hÞpðh=s 1 Þdh pððf =f 1 Þ=hÞpðh=f 1 Þdh Taking into account that all probabilities of success or failure conditioned to h are, respectively, p(s/h) and 1 p(s/h) and substituting the posterior density functions of h according to Expressions (8) and (9) pðs =s 1 Þ¼ pðs =f 1 Þ¼ pðf =s 1 Þ¼ pðf =f 1 Þ¼ pðs=hþ pðs=hþ dh ¼ 1 pðs=hþ pðf 1 Þ dh ¼ 1 1 pðs=hþ dh ¼ 1 pðf 1 Þ dh ¼ 1 1 ðpðs=hþþ dh ð8þ ð9þ ð1þ pðs=hþð1 pðs=hþþdh ð1 pðs=hþþpðs=hþdh ð1 pðs=hþþ dh Substituting p(s 1 )by(1+ p)/, p(f 1 )by(1 p)/ and p(s/h) orp(f/h) according to Expressions (1) and () the probabilities obtained are those shown in Tables 1 and for triangle and duo trio tests respectively. For instance, the probability of success after success is obtained by pðs =s 1 Þ¼ 1 þ p ðp þð1 pþhþ dh ¼ 1 þ p 1 þ p þ p 3 As shown in Tables 1 and a success is more probable after a success than after a failure (39/54 and 3/54 for triangle, Table 1; 7/9 and 6/9 for duo trio, Table ). In the same way a failure is more probable after a failure than after a success. 4.. Posterior distributions of the discrimination rate Applying Expression (8), the posterior density function of h given a success preceded by a success is pðh=ðs =s 1 ÞÞ ¼ pððs =s 1 Þ=hÞpðh=s 1 Þ ¼ pðs=hþpðh=s 1Þ pðs =s 1 Þ pðs =s 1 Þ and substituting p(h/s 1 ) according to Expression (8) and p(s /s 1 ) according to (1) pðh=ðs =s 1 ÞÞ ¼ pðs=hþ 1 pðs=hþ ðpðs=hþþ dh ¼ R ðpðs=hþþ 1 ðpðs=hþþ dh 1 In the same way ð1 pðs=hþþ pðh=ðf =f 1 ÞÞ ¼ ð1 pðs=hþþ dh and pðh=ðs =f 1 ÞÞ ¼ pðh=ðf =s 1 Þ¼ 5. Further trials ð1 pðs=hþþpðh=sþ ð1 pðs=hþþpðh=sþdh For each combination of successes and failures in previous trials performed by a panellist, success or failure can be observed in the next trial. In general, n different series of events (i.e., success, failure, failure, success...) are possible being n the total number of trials and considering different series those that, even when constituted by the same number of successes (y) and failures (n y), are present in different orders. Nevertheless, different series can have the same probability if the same prior distribution of h is used to compute them Posterior distribution of the discrimination rate Generalising what was exposed above for the second test, the posterior distributions of h after n trials are used as prior distributions to compute probabilities for the (n+1)th trial. The number of different posterior distributions of h after n trials is n + 1. Each one of these distributions corresponds to one of the n+1 possible values of y from to n and will be used to compute the probability of success in n series, all of them having the same probability but y differing due to the order in which previous successes and failures were observed. The same distribution of h is also used to compute n the probabilities of other series, all of them failures corresponding one-to-one to successes. Alternatively, the probabilities y of failures can be obtained by p(f) = 1 p(s). In general, the density functions of the posterior distribution of h after n tests given that y successes and n y failures have been observed is ðpðs=hþþ y ð1 pðs=hþþ n y ðpðs=hþþy ð1 pðs=hþþ n y dh ð11þ with y ranging from to n. Fig. 1 shows the posterior density functions of h for panellists who performed 4 (n) triangle trials and succeeded in, 1,, 3, or 4 (y) of them. Considering beta a,b distributions (a = y+1, b = n y+1) the density functions in Fig. 1 are the sections of these beta distributions corresponding to the interval of the independent variable, p(s/h), from 1/3 (h =)to1(h = 1). The proportion

4 5 S. Bayarri et al. / Food Quality and Preference 19 (8) new test. As the number of selected and rejected panellists increases, the number of necessary further tests decreases. In these methods only the number of correct answers is considered, but not the discrimination rate (h) of the panellist, object of most recent papers referred above Marginal probabilities of success and of failure Expression (11) gives the posterior density function of the discrimination rate of a panellist after n trials. Applying this expression, the probability of success of this panellist in the (n+1)th trial is pðs nþ1 Þ¼ ðpðs=hþþy ð1 pðs=hþþ n y pðs=hþdh ðpðs=hþþy ð1 pðs=hþþ n y dh ð1þ and the probability of failure pðf nþ1 Þ¼ ðpðs=hþþy ð1 pðs=hþþ n y ð1 pðs=hþþdh ðpðs=hþþy ð1 pðs=hþþ n y dh Fig. 1. Posterior density functions of discriminations rates of panellists performing four triangle trials according to the number of observed successes. of the area under the curve inside this interval (in relation to the total area of the complete beta distribution) increases as y, the number of successes, does. It is respectively 13%, 47%, 79%, 96% and 99.6% for the values of y from to 4 in the density functions represented in Fig. 1. The area outside the interval is not negligible even for relatively large values of n if the proportion of successes is not too high. For instance, if the number of tests is 3 and the number of successes is 1, 13, or 15, the area outside the interval is 45%, 11% and 3%, respectively. These hypothetical results can easily arise in practical applications of replicated triangle tests such as in the estimation of the detection threshold of a substance. 5.. Comparison with other approaches As mentioned above, published studies usually consider all replications as simultaneously obtained. In these conditions, and when a Bayesian approach is applied, a prior (informative or non informative) distribution for all replications must be selected. Bi (7) showed that different prior distributions produced clearly different results and Duineveld and Meyners (8) indicated that the hyper prior distribution (these authors applied a hierarchical Bayesian approach) affected the individual discrimination rates. With our approach it is out of question to consider this aspect. The posterior distributions are obtained step (replication) by step, considering the posterior distribution from each step as prior distribution for the next one. According to this, only the prior distribution of the first step is not also a posterior distribution, since no previous steps exist. We selected as prior distribution for this first step a non informative distribution (beta 1,1, the uniform distribution, since it is assumed that previous information does not exist and, consequently, y = n = ). Observe that the prior distribution for the first step of our approach must necessarily be non informative, since, if previous information exists, this information would imply knowledge about successes or failures of the panellist in previous tests, what simply means that this panellist is not in the first step but in a posterior one. In the aspect of considering a sequence of steps, the procedure proposed in this paper remembers Walds s and Rao s sequential analyses used for panel selection (Bradley, 1953). Panellists successive perform difference tests and in each step the ratio between total (from the first test to the last one) correct answers and total number of tests is compared with critical values previously established. The decision concerning a panellist may be, according to the result of the comparison, selection, rejection or performance of a Tables 1 and show these probabilities for the third test in triangle and duo trio tests. 6. Performance of the whole panel From a panellist, i, the basic aspect of interest is to estimate her/ his marginal probability of success in the next test (n i + 1) that is estimated according to the number of successes, y i, observed in n i previous tests. Obviously, a unique observation is possible, a single success or a single failure. From a panel formed by h panellists, what interests is to estimate the probabilities of the possible number of successes, x, ranging from to h. If the null hypothesis of no differences between samples is true these probabilities obey the binomial law with parameter p. But in case of real differences between the two samples the discrimination rates, h i, vary among panellists and the probability of each possible value of x cannot be computed according to the binomial formula. For instance, let us suppose that two panellists perform a triangle test and that in a previous test one panellist succeeded and the other one failed. The probabilities of success and failure in the second test of each one of these panellists are given in Table 1. Three possible number of successes (x) exist, (when both panellists failed), 1 (when only one panellists succeeded), and (when both succeeded). The corresponding probabilities are Pðx ¼ Þ ¼ ¼ :41 Pðx ¼ 1Þ ¼ þ ¼ :476 Pðx ¼ Þ ¼ ¼ :13 The number of addends in each case is, as in the binomial formula, but with different probabilities in each addend which h x does not allow to combine them. Nevertheless, these probabilities (such as all discussed in this paper) can be easily calculated using standard computer facilities. As a slightly more complicated example let us suppose that four panellists are going to perform a triangle test and that their backgrounds are 1, 4, 7, and 8 successes in, 5, 1, and 14 previous trials. Applying Expression (1) with the corresponding values of y and n, the respective probabilities of succeeding in the new trial are.6,.7,.67, and.57 respectively for each one of these four panellists. Thus, applying the same procedure discussed for two panellists the probabilities of, 1,, 3, and 4 successes in the hypothetical test that will be performed by these four panellists are.,.1,.3,.38, and.16, respectively.

5 S. Bayarri et al. / Food Quality and Preference 19 (8) Fig.. Probability of success in 1 successive triangle trials as affected by success or failure in previous trials. sponding to five hypothetical panellists, a to e, have been selected and shown in Fig.. This figure represents the successive probabilities of success of these five panellists in the 1st, nd,..., 11th trial according to the order in which successes and failures were observed (for the first three trials these probabilities of success have been already shown in Table 1). Although the final probability of success in the 11th trial is the same for all panellists, the paths are completely different. Panellist a had six successes in the first six trials what suggests a good discrimination capacity but failed in the last trials what can be interpreted as lack of motivation or as fatigue. Panellist e behaved in the opposite way what could suggest low initial capacity of discrimination but improved as experience increased. When the objective of discrimination tests is to detect subtle differences between samples, the ideal panellist is somebody with total experience and completely motivated. Since either experience or lack of motivation can increase as the number of trials do, it is logical to give more positive importance to successes and more negative importance to failures at the end of the series of trials than at the beginning. Thus, a rough index to measure how a panellist is affected by the combination of experience and fatigue could be obtained by assigning rank order numbers to the performed trials, changing the signs of these numbers corresponding to failures and adding up the obtained results. This index is 13( ) for panellist a and 35 for panellist e. Panellists b, c, and d, with successes and failures in the beginning and at the end, have a common index of 11. The indices so obtained can be of some usefulness to compare the behaviours of panellists who performed the same number of trials and had the same number of successes. It would be also useful to develop indices to compare, according to the order in which successes and failures were observed, panellists who performed different number of tests. 7. The order of successes and failures. Experience, fatigue or lack of motivation As exposed above, the discrimination rate of a panellist depend on the total number of previous successes but not on the order they occurred. Nevertheless, it seems that this order must be related in some extent with the behaviour of a panellist. Let us suppose that in n successive tests failures accumulate in the first trials and successes in the last ones. Intuitively, this suggests that the panellist is getting experience. Although not the same, this situation resembles what in sensory profile occurs when panellists are trained to reduce the residual variance. The opposite situation, successes cumulated in the first trials and failures in the last ones, can suggest fatigue or lack of motivation. It would be interesting to develop an index to evaluate experience or fatigue according to the history of the series of successive successes and failures observed from a panellist. In this section some aspects related with this idea are discussed with exploratory purposes. Let us suppose that 1 triangle trials have been performed by some panellists, all of them succeeding in six and failing in four trials. The probability of success in the 11th trial should be.595 according to Expression (1) for all panellists independently of the order in which successes and failures happened. The number of possible different orders is 1 6 = 1, from which 5, corre- 8. Conclusion By considering replicated difference tests as a succession of steps (one per replication) the discrimination rate of a panellist can be estimated by Bayes rule using as prior distribution in each step the posterior distribution obtained in the previous step. The overall rate of a test panel is evaluated as a function of the individual discrimination rates of each panellist. Standard computer facilities are suitable to perform all these calculations. Acknowledgements This research was supported by the Spanish Government (Ministerio de Educación y Ciencia, project AGL6-589ALI) and FEDER founds and by AGROALIMED (Conselleria d Agricultura, Pesca i Alimentació, Generalitat Valenciana, Spain). Authors also thank Fondo Social Europeo for author Bayarri s contract in the program I3P from CSIC. References Bi, J. (3). Difficulties and a way out: a Bayesian approach for sensory difference and preference tests. Journal of Sensory Studies, 18, Bi, J. (6). Sensory discrimination tests and measurements: Statistical principles, procedures and tables. Oxford: Blackwell Publishing. Bi, J. (7). Bayesian analysis for proportions with an independent background effect. British Journal of Mathematical and Statistical Psychology, 6, Bradley, R. (1953). Some statistical methods in taste testing and quality evaluation. Biometrics, 9, 38. Brockhoff, P. B. (3). The statistical power of replications in difference tests. Food Quality and Preference, 14, Carbonell, L., Carbonell, I., & Izquierdo, L. (7). Triangle tests. Number of discriminators estimated by Bayes rule. Food Quality and Preference, 18, Duineveld, K., & Meyners, M. (8). Hierarchical Bayesian analysis of true discrimination rates in replicated triangle tests. Food Quality and Preference, 19, Kunert, J. (1). On repeated difference testing. Food Quality and Preference, 1, Meilgaard, M., Civille, G. V., & Carr, B. T. (1999). Sensory evaluation techniques (3rd ed.). Boca Raton: CRC Press.