Advances in Tetrad Testing

Size: px

Start display at page:

Download "Advances in Tetrad Testing"

Lesley McDonald
5 years ago
Views:

1 Sensometrics 2012 Rennes, France Advances in Tetrad Testing John M. Ennis The Institute for Perception Rune H.B. Christensen Technical University of Denmark

2 Discrimination Testing Discrimination testing as important as ever: Compliance with health initiatives Cost reductions Changes to ingredients, processes, packaging, handling, etc. Quality control Three challenges: 1. Identify sensitive methods for unspecified testing 2. Measurement: a) Quantify sensory differences b) Understand precision in measurement 3. Determine size of meaningful difference 2

3 The Tetrad Test - Methodology Four samples presented: Group the stimuli into two groups of two samples based on similarity Six possible presentation orders: Guessing probability = 1/3 AABB, ABAB, ABBA BBAA, BABA, BAAB 3

4 The Tetrad Test - History Mentioned by Lockhart (1951) and Gridgeman (1954) Revisited by O Mahony, Masuoka, & Ishii (1994) First experiments: Masuoka, Hatjopolous, & O'Mahony (1995) Delwiche & O'Mahony (1996) Psychometric function derived by Ennis et al. (1998) Support for Tetrad testing in IFPrograms (2009) Sample size tables published by Ennis & Jesionka (2011) Operational power-based comparison with Triangle test by Ennis (2012) Large-scale comparison with Triangle test by Garcia, Ennis, & Prinyawiwatkul (2012) Support for Tetrad testing in sensr (2012) 4

Experimental Results (1/3) Masuoka, Hatjopoulos &

5 Experimental Results (1/3) Masuoka, Hatjopoulos & O Mahony (1995) Beer samples varying in bitterness 9 judges with 12 replications: N=108 per condition Proportion Correct d' Triangle Tetrad 3-AFC Triangle Tetrad 3-AFC d' values not significantly different 5

6 Experimental Results (2/3) Delwiche & O Mahony (1996) Chocolate pudding varying in sweetness 13 judges with 12 replications: N = 156 per condition Proportion Correct d' Triangle Tetrad 3-AFC Triangle Tetrad 3-AFC d' values not significantly different 6

7 Experimental Results (3/3) Garcia, Prinyawiwatkul, Ennis (2012) Apple juices varying in sweetness 404 children: 1 Tetrad, 2 Triangle evaluations Proportion Correct d' Triangle Tetrad Triangle Tetrad d' values not significantly different 7

8 Percentage correct Thurstonian Theory Psychometric function (Ennis et al.,1998) 100% 3-AFC Tetrad Triangle 90% 80% 70% 60% 50% 40% 30% δ 8

9 Δ (a) (b) (c) (d) (e) Triangle/Tetrad Possible Cases (d = 1.5) W W W W S W W S S W W S W W Correct S Wrong Wrong Wrong Correct 48.0% 17.9% 28.5% 3.2% 2.3% 63.0% 20.2% 6.7% 6.9% 2.1% 1.0% P c = 50.3% P c = 64.0% W W W W S W W S S S S W W S S S S W W W W S S S Correct Wrong Wrong Wrong Wrong Correct (d) 9 (a) (b) (c) (e) (f)

10 Triangle/Tetrad Sample Sizes Suppose α = 0.05 and want 80% power If δ = 1.5 Tetrad N = 20 Triangle N = 57 If δ = 1.0 Tetrad N = 65 Triangle N = 220 Tetrad sample sizes are roughly 1/3 Triangle sample sizes See Ennis & Jesionka (2011) for more information Tetrad Triangle δ 10

11 B Value Precision of Measurement (1/4) Variance in estimate of δ (Bi, Ennis, & O Mahony, 1997) Variance is B value divided by sample size 35 Tetrad Triangle δ 11

12 Precision of Measurement (2/4) Tetrad test can be analyzed using GLM framework (Brockhoff and Christensen, 2010): P C f 1 δ Convenient access to statistical analysis 12

13 Relative likelihood Precision of Measurement (3/4) Relative likelihood (Christensen & Brockhoff, 2009) 1.00 Function shape gives improved estimate of precision Example: N = 60, δ ~ Tetrad Triangle δ 13

14 Expected Width Precision of Measurement (4/4) Expected widths of likelihood confidence intervals 2.0 N = 60, 95% confidence Tetrad Triangle δ 14

Proportion correct Tetrad Triangle 63% 53% 43% 33% Mild Savory Pesto Alfredo Neopolitan

15 Comparative Examples (1/2) Six pasta sauces for food service applications Research to compare Triangle and Tetrad tests Test sample sizes vary between 96 and % Proportion correct Tetrad Triangle 63% 53% 43% 33% Mild Savory Pesto Alfredo Neopolitan Meat 2.00 d' values Tetrad Triangle Mild Savory Pesto Alfredo Neopolitan Meat 15

16 Comparative Examples (2/2) Likelihood confidence intervals: δ Triangle Tetrad } Mild } Savory } Pesto } Alfredo } Neopolitan } Meat Tetrad test gives more precise estimate of sensory difference in each case 16

17 Final Points Future topics: Equivalence Unequal variance Multivariate Tetrad model Comparison to 2-AFCR Decision rule investigation Thanks to: Daniel Ennis & Benoit Rousseau, The Institute for Perception Pieter Punter, OP&P Product Research Per Brockhoff, Technical University of Denmark 17

18 Sensometrics 2012 Rennes, France Advances in Tetrad Testing John M. Ennis The Institute for Perception Rune H.B. Christensen Technical University of Denmark

Image credit: wikipedia.org

Equipped with our five senses, we explore the universe... and call the adventure science - Edwin Hubble Image credit: wikipedia.org www.ifpress.com 1 Society of Sensory Professionals Jersey City, New Jersey