Sensory Discrimination Tests and Measurements. Statistical Principles, Procedures and Tables

Transcription

1 Sensory Discrimination Tests and Measurements Statistical Principles, Procedures and Tables

2 Sensory Discrimination Tests and Measurements Statistical Principles, Procedures and Tables Jian Bi Sensometrics Research and Service Richmond, Virginia, USA

3 Jian Bi is a Senior Statistician and the President of Sensometrics Research and Service, Richmond, Virginia. C 2006 Jian Bi All rights reserved Blackwell Publishing Professional 2121 State Avenue, Ames, Iowa 50014, USA Orders: Office: Fax: Web site: Blackwell Publishing Ltd 9600 Garsington Road, Oxford OX4 2DQ, UK Tel.: +44 (0) Blackwell Publishing Asia 550 Swanston Street, Carlton, Victoria 3053, Australia Tel.: +61 (0) Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by Jian Bi, provided that the base fee of $.10 per copy is paid directly to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA For those organizations that have been granted a photocopy license by CCC, a separate system of payments has been arranged. The fee codes for users of the Transactional Reporting Service are ISBN-13: ; ISBN-10: /2005 $.10. First edition, 2006 Library of Congress Cataloging-in-Publication Data Bi, Jian, 1949 Sensory discrimination tests and measurements : statistical principles, procedures, and tables / Jian Bi. 1st ed. p. cm. Includes bibliographical references. ISBN-13: (alk. paper) ISBN-10: Agriculture Statistical methods. 2. Sensory discrimination Statistical methods. I. Title. S566.55B dc The last digit is the print number:

4 To Yulin

5 Contents Preface ix 1 Introduction A Brief Review of Sensory Analysis Methodologies Method, Test, and Measurement Standard Discrimination Methods Classification of Sensory Discrimination Methods 3 References 5 2 Standard Discrimination Tests Binomial Model for Discrimination Testing Discrimination Tests Using Forced-Choice Methods Discrimination Tests Using the Methods with Response Bias 12 References 20 3 Statistical Power Analysis for Standard Discrimination Tests Introduction Power and Sample Size for Forced-Choice Methods Power and Sample Size for the Methods with Response Bias Efficiency Comparisons of Discrimination Tests 38 References 44 4 Modified Discrimination Tests The Modified Triangle Test The Degree of Difference Test The Double Discrimination Test The Preference Test with No Preference Option 72 References 76 5 Multiple-Sample Discrimination Tests Multiple-Sample Comparison Based on Proportions Multiple-Sample Comparison Based on Ranks Multiple-Sample Comparison Based on Categorical Scales 98 References Replicated Discrimination Tests: Beta-Binomial Model Introduction The Beta-Binomial Distribution Estimation of Parameters of Beta-Binomial Model 109 vii

6 viii contents 6.4 Applications of Beta-Binomial Model in Replicated Tests Testing Power and Sample Size for Beta-Binomial Tests 122 References 127 Appendix 6A Replicated Discrimination Tests: Corrected Beta-Binomial Model Introduction The Corrected Beta-Binomial Distribution Estimation of Parameters of Corrected Beta-Binomial Model Statistical Testing for Parameters in Corrected Beta-Binomial Model Testing Power and Sample Size 148 References 150 Appendix 7A Replicated Discrimination Tests: Dirichlet-Multinomial Model The Dirichlet-Multinomial Distribution Estimation of Parameters of Dirichlet-Multinomial Model Applications of DM model in Replicated Tests Testing Power for Dirichlet-Multinomial Model 179 References Measurements of Sensory Difference: Thurstonian Model Introduction Thurstonian Variance of d Tables for d and Variance of d 237 References Statistical Analysis for d Data Estimates of Population or Group d Statistical Inference for d Data 248 References Similarity Testing Introduction Similarity Testing for Preference Similarity Testing Using Forced-Choice Methods Similarity Testing Using the A Not A and the Same Different Methods 261 References 268 Appendix 11A 269 Appendix A List of S-Plus Codes 287 Author Index 289 Subject Index 293

7 Preface Discriminative analysis, including discrimination tests and measurements, is the most fundamental type of methodology in sensory science. The validation of the methodology depends to some extent on sound statistical models. The objective of this book is to deal with statistical aspects of the methodology and to provide the reader with statistical principles, procedures and tables for some methods. The book attempts to give a unified picture of the state of the subject and to reflect some features of advanced sensory discriminative analysis. This book consists of eleven chapters. It is organized as follows: Chapter 1 briefly reviews sensory methodologies with emphasis on six standard, widely used discrimination methods: the 2-AFC, 3-AFC, Duo Trio, Triangle, A Not A, and the Same Different methods. Chapters 2 to 5 discuss discrimination testing including standard discrimination tests (Chapters 2 3), modified discrimination tests (Chapter 4), and multiple-sample discrimination tests (Chapter 5) under the conventional assumption that the consumer population is composed of discriminator and non-discriminator and panelists of a laboratory panel have the same discrimination ability. Chapters 6 to 8 present a unified approach to replicated discrimination tests using a beta-binomial framework under the assumption that discrimination ability or preference for each individual consumer and panelist is not a constant but a random variable. The assumptions under discrimination testing discussed in Chapters 2 to 5 and Chapters 6 to 8 are philosophically different. Chapters 9 to 10 are devoted to a discussion on sensory measurement using Thurstonian discriminal distance (or d ). Chapter 11, the last chapter, discusses similarity testing, which is practically and theoretically important but often confusing. The book is intended for researchers and practitioners in the sensory and consumer field and has been written keeping both the statistical and non-statistical readers in mind. It is not difficult to apply most of the methods by following the numerical examples using the corresponding formulas and tables provided in the book. For some of the methods involving complicated calculations, computer programs are needed. Thanks to modern computer technology, calculations are much easier than before. The extent of computational complication involved in a method should not be regarded as a major concern in the selection of methods. For some statistical considerations behind the methodology and some mathematical derivations in the book, readers with a more statistical background will understand them without major difficulty. Some S-PLUS codes, which appear in the book and are listed in Appendix A, are available from the author on request. The author may be contacted via at BBDJCY@aol.com. ix

8 x preface Acknowledgments I am greatly indebted to the Series Editor, Dr. Max Gacula, who encouraged me to write this book, reviewed the manuscript, and provided insightful comments. I wish to express my gratitude to Professor Edgar Chambers, Dr. Morten Meilgaard, Professor Michael O Mahony, and Dr. Daniel Ennis for their valuable support and help for the past years. I would like to thank the publisher and my editors Mark Barrett, Dede Pedersen, Susan Borts, and Judi Brown at Blackwell Publishing and Suditi Srivastava at TechBooks for publishing my book and bringing the project to completion. Finally, I wish to thank deeply my wife, Yulin, for her patience, understanding, and encouragement during the preparation of this book. Jian Bi

9 Sensory Discrimination Tests and Measurements: Statistical Principles, Procedures and Tables Jian Bi Copyright 2006 by Jian Bi 1 Introduction 1.1 A brief review of sensory analysis methodologies To conduct valid tests and to provide reliable sensory measurements are the main functions of sensory analysis. Statistical inference is the theoretical basis of sensory tests. Psychometrics, which provides invariable indexes, which is independent of methods, is the theoretical basis of sensory measurements. Sensory analysis can be divided into two parts: laboratory sensory analysis and consumer sensory analysis. In the laboratory sensory analysis, a trained panel is used as an analytical instrument to measure sensory properties of products. In the consumer sensory analysis, a sample of specified consumer population is used to test and predict consumer responses for products. The two types of sensory analysis have different goals and functions, but they share some of the same methodologies. Discriminative analysis and descriptive analysis are the main classes of methodology for both the laboratory and consumer sensory analyses. Discriminative analysis includes discrimination tests and measurements. Discrimination tests are used to determine, usually using a 2-point scale or a ranking scale, whether a difference exists between treatments for confusable sensory properties of products. Discrimination measurements are used to measure, using an index, the extent of the difference. There are two sources of sensory differences: intensity and preference. Discriminative analysis is referred to difference test when testing difference of intensity. Discriminative analysis is referred to preference test when testing difference of preference. Descriptive analysis is to determine, using a rating scale, how much a specific characteristic difference exists among products, which is quantitative descriptive analysis, and to characterize a product s sensory attributes, which is qualitative descriptive analysis. Quantitative descriptive analysis for preference is also called acceptance testing. Acceptance or preference testing for a laboratory panel is of very limited value (Amerine et al. 1965). However, the consumer discriminative and descriptive analyses for both intensity and references are valuable. The laboratory difference testing, using a trained panel under controlled conditions, has been called the Sensory Evaluation I, whereas the consumer difference testing, using a sample of untrained consumers under ordinary using (eating) conditions, has been called the Sensory Evaluation II (O Mahony 1995). They are different types of difference testing. Misusing the two types of difference testing will lead to misleading conclusions. The controversy over whether the consumer can be used for difference testing may ignore the fact that the laboratory and consumer difference tests have different goals and functions. The distinction between the discriminative analysis and the quantitative descriptive analysis is not absolute from the viewpoint of modern sensory analysis. The Thurstonian model that will be discussed in Chapters 9 10 of this book can be used for both discriminative 1

10 2 sensory discrimination tests and measurements analysis and quantitative descriptive analysis. The Thurstonian (or d ), which is a measure of sensory difference, can be obtained from any kind of scales used in discriminative and descriptive analyses. In addition, rating scale, which is typically used in descriptive analysis, is also used in some modified discrimination tests. Besides discriminative analysis and descriptive analysis, there are other classes of sensory methodologies, i.e., sensitivity analysis, time-intensity (TI) analysis, and similarity testing. Sensitivity analysis is to determine sensory thresholds, including individual and population thresholds. Threshold is a statistical concept. It is an intensity that produces a response with a 0.5 probability. There are many specific statistical methods for estimating and testing thresholds (for review, see, e.g., Bi and Ennis 1997). Time-intensity analysis or shelf-life analysis is used to determine the relationship between sensory intensity and time. Survival analysis, which is a well-developed field, provides sound statistical methodology for TI analysis. Time-intensity analysis is conventionally included in the descriptive analysis. Considering the specifications of the methodology, it seems that TI analysis should be separated from the conventional descriptive analysis. Similarity testing is relatively new and is not well developed in the sensory field. Unlike discrimination testing, the objective of similarity testing is to demonstrate similarity rather than difference. Similarity testing uses the same sensory analysis methods for discrimination tests, but different statistical models. This book is primarily concerned with methodology, mainly in statistical aspects, of sensory discriminative analysis including laboratory and consumer discriminative analyses. Similarity testing is briefly discussed in Chapter Method, test, and measurement In this book, a distinction is made among the three terms: sensory discrimination method, sensory discrimination test, and sensory discrimination measurement. In sensory discriminative analysis, some procedures are used for experiments. The procedures are called discrimination methods, e.g., the Duo Trio method, the Triangular method. When the discrimination procedures are used for statistical hypothesis testing, or in other words, when statistical testing is conducted for the data from a discrimination procedure, the procedure is called discrimination testing, e.g., the Duo Trio test, the Triangular test. When the discrimination procedures are used for measurement, or in other words, when an index, e.g., Thurstonian (or d ), is produced using the data from a discrimination procedure, the procedure is called discrimination measurement, e.g., the Duo Trio measurement, the Triangular measurement. 1.3 Standard discrimination methods Six standard and basic discrimination methods are the focus of this book. They are: (a) The 2-Alternative Forced-Choice method (2-AFC) (Green and Swets 1966): This method is also called the paired comparison method (Dawson and Harris 1951, Peryam 1958). In this method, the panelist receives a pair of coded samples, A and B, for comparison on the basis of some specified sensory characteristic. The possible pairs are AB and BA. The panelist is asked to select the sample with the

11 introduction 3 strongest (or the weakest) sensory characteristic. The panelist has to select one even if he or she cannot detect the difference. (b) The 3-Alternative Forced-Choice method (3-AFC) (Green and Swets 1966): Three samples of two products A and B are presented to each panelist. Two of them are the same. The possible sets of samples are AAB, ABA, BAA; or ABB, BAB, BBA. The panelist is asked to select the sample with the strongest or the weakest characteristic. The panelist has to select one sample even if he or she cannot identify the one with the strongest or the weakest sensory characteristic. (c) The Duo Trio method (Dawson and Harris 1951, Peryam 1958): Three samples of two products A and B are presented to each panelist. Two of them are the same. The possible sets of samples are AAB, ABA, ABB, BAA, BAB, and BBA. The first one is labeled as the control. The panelist is asked which one in the two test samples is the same as the control sample. The panelist has to select one sample to match the control sample even if he or she cannot identify which one is the same as the control sample. (d) The Triangular (Triangle) method (Dawson and Harris 1951, Peryam 1958): Three samples of two products A and B are presented to each panelist. Two of them are the same. The possible sets of samples are AAB, ABA, BAA, ABB, BAB, and BBA. The panelist is asked to select the odd sample. The panelist has to select one sample even if he or she cannot identify the odd one. (e) The A Not A method (Peryam 1958): Familiarize the panelists with the samples A and Not A. One sample which is either A or Not A is presented to each panelist. The panelist is asked if the sample is A or Not A. (f) The Same Different method (see, e.g., Pfaffmann 1954, Amerine et al. 1965, Macmillan and Kaplan 1977, Meilgaard et al. 1991, among others, for the method in different names): A pair of samples is presented to each panelist. The pair is one of the four possible sample pairs: AA, BB, AB, and BA, where A and B are the two products for comparison. The panelist is asked if the sample pair that he or she received is the same or different. 1.4 Classification of sensory discrimination methods Sensory discrimination methods are typically classified according to the number of samples presented for evaluation, i.e., the single sample (stimulus), the two samples, the three samples, and the multiple samples. This classification is natural, but it does not reflect the inherent characteristic in the methods. In this book, the discrimination methods are classified according to the decision rules and cognitive strategies involved in the methods. This kind of classification may be more reasonable and profound. In the following chapters, we will see how the methods in the same class correspond to the same type of statistical models and decision rules Methods requiring and not requiring the nature of difference There are two different types of instructions in the discrimination method. One type of instruction is to ask the panelists to indicate the nature of difference in the products for

12 4 sensory discrimination tests and measurements evaluation, e.g., Which sample is sweeter? (the 2-AFC and 3-AFC methods); Is the sample A or Not A? (the A Not A method). The other type of instruction is related to the comparison of distance of difference, e.g., Which of the two test samples is same as the control sample? (the Duo Trio method); Which sample is the odd one in the three samples? (the Triangular method); Are the two samples the same or different? (the Same Different method). The two types of instructions involve different cognitive strategies and result in different proportions of correct responses. Hence the discrimination methods can be divided into these two types: the methods using the skimming strategy and the methods using the comparison of distance strategy (O Mahony et al. 1994) Methods with and without response bias Response bias is a basic problem with sensory discrimination methods. Many authors, e.g., Torgerson (1958), Green and Swets (1966), Macmillan and Creelman (1991), O Mahony (1989, 1992, 1995), addressed this problem. Sensory discrimination methods are designed for detection and measurement of confusable sensory differences. There is no response bias if the difference is large enough. However, response bias may occur when the difference between two products is so small that a panelist makes an unsure judgment. In this situation, the decision criterion of how large a difference can be judged as a difference may take a role in the decision process. Criterion variation, i.e., strictness or laxness of criterion causes response bias. A response bias is a psychological tendency to favor one side of a criterion. Response bias is independent of sensitivity. This is why the methods with response bias (e.g., the A Not A and the Same Different methods) can also be used for difference testing. However, response bias affects test power. The influence of response bias on difference testing will be discussed in Chapter 3. Forced-choice procedures can be used to stabilize decision criterion. Hence most sensory discrimination methods are designed in a forced-choice procedure. A forced-choice procedure must have at least three characteristics: (1) Two sides of a criterion must be presented in a forced-choice procedure. The two sides of a criterion may be strong and weak, if the criterion is about the nature of the difference of products. The two sides of a criterion may be same and different, if the criterion is about the distance of the difference of products. Because a single sample or a same type of sample cannot contain two sides of a criterion, evaluating a single sample or same type of samples is not a forced-choice procedure. Because a single pair of samples or a same type of sample pairs cannot contain two sides of a criterion about the distance of a difference, evaluating a single sample pair or a same type of sample pairs is not a forced-choice procedure, either. (2) A panelist should be instructed that the samples presented for evaluation contain the two sides of a criterion. (3) A response must be given in terms of one clearly defined category. The don t know response is not allowed. In the six standard and basic sensory discrimination methods, the 2-AFC, 3-AFC, Triangular, and Duo Trio methods are the forced-choice methods. In the 2-AFC and 3-AFC methods, the criterion is about the nature of the difference for products. Two and three samples that contain two products are presented and instructed to a panelist in the methods. A panelist is asked to select the sample with the strong or the weak sensory property, even if the panelist cannot detect the difference. In the Duo Trio and Triangular methods, the criterion is about comparison of distance of difference. A same sample pair and an

13 introduction 5 Table 1.1 A two-way classification of six standard and basic sensory discrimination methods Requiring the nature of difference Comparing distance of difference Without response bias 2-AFC Duo Trio (Forced-choice procedure) 3-AFC Triangular With response bias A Not A Same Different odd sample are composed of the samples presented in the methods. A panelist is asked to select the odd sample, even if he or she cannot find the odd sample. In the six standard and basic sensory discrimination methods, the A Not A method and the Same Different method are the methods with response bias, because only one sample, either sample A or Not A, is presented to a panelist in the A Not A method; and only one sample pair, either a concordant sample pair or a discordant sample pair, is presented to a panelist in the Same Different method. The six standard and basic sensory discrimination methods are classified based on response bias and strategies for determination of difference. Table 1.1 gives a two-way classification for the methods. References Amerine, M. A., Pangborn, R. M. and Roessler, E. B Principles of Sensory Evaluation of Food. Academic Press, New York, NY. Bi, J. and Ennis, D. M Sensory threshold: Concepts and methods. Journal of Sensory Studies 13, Dawson, E. H. and Harris, B. L Sensory methods for measuring differences in food quality. Agriculture Information Bulletin 34, US Department of Agriculture, Washington, DC. Green, D. M. and Swets, J. A Signal Detection Theory and Psychophysics. John Wiley, New York. Macmillan, N. A. and Kaplan, H. L The psychophysics of categorical perception. Psychological Review 84, Macmillan, N. A. and Creelman, C. D Detection Theory: A User s Guide. Cambridge University Press, New York. Meilgaard, M., Civille, G. V. and Carr, B. T Sensory Evaluation Techniques (2nd ed.), CRC Press, Boca Raton, FL. O Mahony, M Cognitive aspects of difference testing and descriptive analysis: Criterion variation and concept formation. In Psychological Basis of Sensory Evaluation, eds R. L. McBride and H. J. H. MacFie. Elsevier Applied Science, New York, pp O Mahony, M Understanding discrimination tests: A user-friendly treatment of response bias, rating and ranking R-index tests and their relationship to signal detection. Journal of Sensory Studies 7, O Mahony, M Sensory measurement in food science: Fitting methods to goals. Food Technology 49, O Mahony, M., Susumu, M. and Ishii, R A theoretical note on difference tests: Methods, paradoxes and cognitive strategies. Journal of Sensory Studies 9, Peryam, D. R Sensory difference tests. Food Technology 12, Pfaffmann, C Variables affecting difference tests. In Food Acceptance Testing Methodology, A Symposium. National Academy of Science and National Research Council, Washington, DC, pp Torgerson, W. S Theory and Methods of Scaling. John Wiley, New York.

14 Sensory Discrimination Tests and Measurements: Statistical Principles, Procedures and Tables Jian Bi Copyright 2006 by Jian Bi 2 Standard discrimination tests Discrimination testing is one of the main functions of discriminative analysis. It includes difference testing and preference testing. In this chapter, the standard discrimination tests, i.e., the discrimination testing using six standard discrimination methods under conventional conditions will be discussed. All the six methods can be used for difference testing. Of these six, only the paired comparison method (2-Alternative Forced-Choice method) can be used for both difference testing and preference testing. 2.1 Binomial model for discrimination testing Discrimination testing is assumed to be involved in a binomial experiment. The number of correct responses in a discrimination testing is assumed to be a binomial variable following a binomial distribution. In this section, the validity of using the binomial model for a discrimination testing will be discussed. Binomial experiment A binomial experiment possesses the following properties: (a) (b) (c) (d) The experiment consists of n trials. Each response is a binary variable that may be classified as a success or a failure. The trials are independent. The probability of success, denoted by p, remains constant from trial to trial. Binomial variable The number of successes in n trials of a binomial experiment is called a binomial variable, X, which follows a binomial distribution. Binomial distribution The probability that there are exactly x successes in n independent trials in a binomial experiment is given by the probability function ( ) n P(x; p, n) = p x (1 p) n x, x = 0, 1, 2,..., n. (2.1.1) x The cumulative distribution function is given by x ( ) n F(x) = p k (1 p) n k. (2.1.2) i k=0 The parameters of the binomial distribution are n and p. The mean is E(X) = np and the variance is Var(X) = np(1 p). In a standard discrimination testing, n responses (trials) are obtained from n panelists. Each panelist gives only one response so that the n responses can be regarded as independent of each other. The response of each panelist is a binary variable because each response results in one of two possible outcomes and the no difference response is not allowed 6

15 standard discrimination tests 7 in the tests. Obviously, the first three properties of a binomial experiment are satisfied in a standard discrimination testing. The question that often arises is how to understand the fourth property of a binomial experiment in a standard discrimination testing. The question, how to understand each panel, has the same probability of correct responses. The conventional assumption for a consumer discrimination testing is that a consumer panel is a representative sample of a specific consumer population. Consumers in a specific population are divided into discriminator and nondiscriminator for the products compared. Because each panelist has the same probability of becoming a discriminator, it is equivalent to that each panelist has the same probability of correct responses. For a laboratory panel, which is regarded as an instrument and is not a sample of consumer population any more, the underling assumption is that the panelists have the same discrimination ability. Hence each panelist can be assumed to have the same probability of correct responses. The conventional sensory difference and preference tests are based on statistical hypothesis testing for proportions. For the forced-choice methods, the testing involves comparison of a proportion with a specified value. For the methods with response bias, the testing mainly involves comparison of two proportions. 2.2 Discrimination tests using forced-choice methods Guessing model Guessing model for difference tests There is a guessing model for a difference test using a forced-choice method. The guessing model indicates the relationship among three quantities probability of correct responses or preference, p c, probability of correct guess, p 0, and proportion of discriminators (for consumer discrimination testing) or probability of discrimination (for laboratory discrimination testing), p d : p c = p d + p 0 (1 p d ). (2.2.1) If the two products are the same, the probability of a correct response for each panelist should be a chance probability (p 0 ) in a forced-choice method. Otherwise, if the two products are different, a discriminator gives a correct response with a probability of 1, whereas a nondiscriminator gives a correct response with a chance probability p 0. There is a p d probability to get a consumer panelist who is just a discriminator and there is a 1 p d probability to get a consumer panelist who is just a non-discriminator. According to the theorem on total probabilities, 1 the probability of a correct response or preference for each consumer panelist should be as given in (2.2.1). The similar situation is for a laboratory panelist. For each trained panelist, the probabilities of discrimination and non-discrimination are p d and 1 p d, respectively. If the panelist can discriminate the products, the probability of a correct response is 1, whereas if the panelist cannot discriminate the products, the probability of a correct response is the guessing probability. Hence the probability of a correct response for each trained panelist should also be as given in (2.2.1) according to the theorem on total probabilities. 1 Theorem on total probabilities: If an arbitrary event E intersects the mutually exclusive and collectively exhaustive event A i, then the probability of event E is P(E) = i P(A i )P(E/A i ), where P(E/A i ) is the conditional probability of E at the condition A i (see, e.g., Sachs 1982).

16 8 sensory discrimination tests and measurements Guessing model for preference testing The guessing model for the consumer preference testing is different from that for the difference testing. There are two independent proportions, p a and p b, which denote the proportions of consumers preferring product A and B, respectively, in a consumer population. It is assumed that p a + p b 1 and p n = 1 p a p b is the proportion of consumers with no preference. A consumer panelist should give response A with probability 1, if he or she prefers A; should give response A with probability 0, if he or she prefers B; should give response A with probability 0.5, if he (or she) has genuinely no preference, but No preference option is not allowed in a test. Hence the total probability of preferring A in a preference test should be P A = p a + p n /2 = (1 + p a p b )/2. (2.2.2) The total probability of preferring B in a preference test should be P B = 1 P A = (1 p a + p b )/2. (2.2.3) It should be noted that (2.2.2) and (2.2.3) are not independent of each other Hypothesis test for discrimination Null and alternative hypotheses Testing whether there is a difference between two products is the same as testing if p d = 0orp c = p 0. Hence discrimination tests using a forced-choice method involve comparison of one proportion with a fixed value, i.e., p 0 = 0.5 for the 2-AFC and the Duo Trio methods and p 0 = 1/3 for the 3-AFC and the triangular methods. The null hypothesis is H 0 : p c = p 0 and the alternative hypothesis is H 1 : p c > p 0 for a one-sided test or H 1 : p c p 0 for a two-sided test. Testing whether there are different preferences for two products is same as testing if p a = p b or P A = 0.5 (or P B = 0.5 ). In discrimination testing, the objective is to reject the null hypothesis. If the null hypothesis is not rejected, it is inappropriate to conclude that the null hypothesis is proved or established regardless of the sample size One-sided and two-sided tests There is only one-sided test situation for the 3-AFC, the Duo Trio, and the triangular tests because only p c > p 0 is possible and concerned when the null hypothesis is rejected. However, there are both one-sided and two-sided testing situations for the preference and nondirectional 2-AFC tests. The choice depends on the purpose of the experiment. For example, in a test for sweetness of two products (current product and a new product), we know in advance that the new product contains more sugar than the current product. In this situation, the one-sided test should be selected because only one direction of possible difference is of interest. Or, for example, in a preference test for two products, wherein we do not know in advance which one is more popular, the two-sided test should be selected. The decision to use a one-sided or a two-sided test should be made before the experiment Type I and type II errors In hypothesis testing two types of errors may be involved. A type I error has been committed if we reject the null hypothesis when it is true. This error is denoted as and is also called significance level. = 0.1, 0.05, 0.01

17 standard discrimination tests 9 are conventionally selected. A type II error has been committed if we accept the null hypothesis when it is false. This error is denoted as and = 0.2, 0.1 are conventionally selected Test statistic and critical value The test statistic based on the binomial distribution in (2.1.2) is the number of correct responses, X. The critical values for one-sided and twosided tests are given in Table 2.1 according to c ( n ) p k 0 (1 p 0 ) n k 1 (2.2.4) i k=0 and c ( n ) p k 0 (1 p 0 ) n k 1 /2, (2.2.5) i k=0 where is the significance level and c is the critical value. Table 2.1 gives critical values (c) for sample size n from 10 to 100, = 0.05 and 0.1 for the preference and nondirectional 2-AFC, directional 2-AFC and Duo Trio, and one-sided 3-AFC and triangular tests, respectively. If the observed number of correct responses or preference is larger than the corresponding critical value, a conclusion of significant difference between the products for comparison can be made. If the sample size is outside the range of values given in Table 2.1, an appropriation of the binomial distribution by the normal distribution can be used. The test statistic is as given in (2.2.6), which follows approximately a standard normal distribution: Z = X np np0 (1 p 0 ). (2.2.6) The critical values at = 0.01, 0.05, and 0.1 are 2.33, 1.65, and 1.28, respectively, for the one-sided test and are 2.58, 1.96, and 1.65, respectively, for the two-sided test. Example For illustration of the procedures in this section, a numerical example is given below. In order to determine if there is detectable difference between a current product and an improved product for preference, 100 consumer panelists were drawn from a consumer population of heavy users of the product and a significance level = 0.05 is selected. The test is two-sided because any one of the two products can be preferred. The null hypothesis is H 0 : p c = 0.5 and the alternative hypothesis is H 1 : p c 0.5. The observed numbers of preference for the new product and the current product are 62 and 38, respectively. Because the larger number (62) of the two numbers (62 and 38) is larger than corresponding critical value (61) in Table 2.1, a conclusion is drawn that there is a significant difference between the two products for preference in the specific consumer population at a 0.05 significance level. The consumer has a preference for the new product. If the normal approximation is used, from (2.2.6) Z = (1 0.5) = 2.3 > Hence the same conclusion can be drawn.

18 10 sensory discrimination tests and measurements Table 2.1 methods Minimum number of correct responses for difference and preference tests using forced-choice 2-AFC and Duo Trio 3-AFC and Triangular 2-AFC (Two-sided) (One-sided) (One-sided) N = 0.01 = 0.05 = 0.1 = 0.01 = 0.05 = 0.1 = 0.01 = 0.05 =

19 standard discrimination tests 11 Table 2.1 Contd 2-AFC and Duo Trio 3-AFC and Triangular 2-AFC (Two-sided) (One-sided) (One-sided) N = 0.01 = 0.05 = 0.1 = 0.01 = 0.05 = 0.1 = 0.01 = 0.05 =

20 12 sensory discrimination tests and measurements Parameter estimate Estimate of proportion of discriminator or probability of discrimination Once we have concluded that the two products for comparison are significantly different, we can estimate the proportion of discriminators for the products in a specific consumer population or the probability of discrimination for the products in a trained panel. We can get the estimate of p d from ˆp d = ˆp c p 0, (2.2.7) 1 p 0 where ˆp c is the observed proportion of correct responses or preference, ˆp c = x/n. An approximate 95% confidence interval for p d is given by ˆp d ± 1.96 V ( ˆp d ), (2.2.8) where V ( ˆp d ) is the estimate of variance of ˆp d. According to the Taylor series, ˆp d = f ( ˆp c ) f ( ˆp c0 ) + f ( ˆp c0 )( ˆp c ˆp c0 ), where ˆp c0 denotes an observation of ˆp c and f ( ˆp c0 ) denotes the first derivative with respect to ˆp c evaluated at ˆp c0. Hence Var( ˆp d ) = f 2 ( ˆp c0 )Var( ˆp c ), i.e., 1 ˆp c (1 ˆp c ) V ( ˆp d ) =. (2.2.9) (1 p 0 ) 2 N Example For Example 2.2.1, ˆp d = = 0.24, V ( ˆp d ) = 0.62 (1 0.62)/100 = , and ˆp d ± 1.96 V ( ˆp d ) = 0.24 ± = (0.05, 0.43). This means that the estimated proportion of discriminators for the two products is 0.24 and the 95% confidence interval for the proportion is (0.05, 0.43). We should interpret and use the estimate of p d with caution. The only difference between p c and p d is that the guessing effect is included in p c and excluded in p d. The quantity p d is the proportion of correct responses above chance. However, p d is still dependent on the method used. It is not a pure index of difference or discrimination. We will discuss further this problem in Chapter Estimate of proportions of preference It is often required to estimate proportions of preference, p a and p b, from a preference test. However, it is clearly impossible to do this with equation (2.2.2) or (2.2.3) for a conventional preference testing. There are two independent parameters, but only one independent equation. In order to estimate p a and p b, a replicated test is needed. See Section 4.3 or 4.4 of Chapter 4 for estimates of p a and p b, using the data from a double preference testing without No preference option or two-visit method with No preference option. 2.3 Discrimination tests using the methods with response bias In the methods with response bias, there is no guessing probability as p 0 in a forcedchoice method. This is the main distinction between the two types of methods. The data for discrimination tests using the A Not A or the same different method can be set out