Direct estimation of multidimensional perceptual distributions: Assessing hue and form

Transcription

1 Perception & Psychophysics 2003, 65 (7), Direct estimation of multidimensional perceptual distributions: Assessing hue and form DALE J. COHEN University of North Carolina, Wilmington, North Carolina The procedures developed to assess the perceptual and decisional processes associated with detection in multidimensional space all require specialized statistical skills and analysis programs. The present article describes a regression model, designed to assess dimensional interactions, that is both computationally simpler and more accessible than those procedures. The paper validates the regression model by comparing the perceptual space associated with the detection of hue and form mapped by the regression model with that mapped by Kadlec and Townsend s (1992a, 1992b) macro- and microanalyses. The results of both analyses showed that hue strongly influences the perception of form but that form only weakly influences the perception of hue. The parallel results of the two analyses suggest that the regression model is a valid alternative to multidimensional signal detection theory analysis. All stimuli are multidimensional. Ashby and Townsend (1986) have made the point that a fundamentally important problem is to determine how these dimensions are combined in perceptual processing (p. 154). Toward that end, Ashby and Townsend proposed a general recognition theory (GRT), which extends signal detection theory (SDT) to multidimensional space. Unfortunately, assessing the perceptual and decisional processes associated with detection in multidimensional space is more complicated than equivalent assessments in unidimensional space (e.g., Ashby & Townsend, 1986). The few procedures developed to assess these processes in multidimensional space all require specialized statistical skills and analysis programs (e.g., Ashby & Lee, 1991; Kadlec & Townsend, 1992a, 1992b; Maddox, 2001; Maddox & Ashby, 1996; Wickens, 1992). It is perhaps because of this requirement that multidimensional stimulus detection has not received the attention afforded to unidimensional stimulus detection. In an effort to make multidimensional stimulus detection more accessible, the present article proposes and validates a relatively simple regression procedure that provides information about dimensional interaction similar to that provided by an established analytic procedure based on SDT (Kadlec & Townsend s, 1992a, 1992b, macro- and microanalyses). GENERAL RECOGNITION THEORY Perhaps the most widely recognized theory of dimensional interaction is GRT (Ashby & Townsend, 1986). GRT assumes a multidimensional perceptual space, with the perceptual effect (C) of each dimension lying on separate I thank S. Cohen for her helpful comments and support at various stages of this project. I also thank Helena Kadlec and two anonymous reviewers for their very helpful critiques. Correspondence concerning this article should be sent to D. J. Cohen, Department of Psychology, University of North Carolina, Wilmington, NC ( cohend@uncw.edu). axes. For example, one may create a two-dimensional space with the perceptual effect of Dimension A on the x-axis and the perceptual effect of Dimension B on the y-axis (see Figure 1). A specific stimulus presentation is represented by a point in space that corresponds to the perceptual effect of each dimension on that presentation. GRT, like SDT, assumes that there is noise in the system. Therefore, a specific stimulus will give rise to different perceptual effects at each viewing. Multiple viewings of a stimulus will, therefore, produce a cloud of points, with each point landing in a region of space with a particular probability (called the perceptual distribution of the stimulus). The bivariate mean m 5 [m A, m B ] and the variance/ covariance matrix S 5 [sa, 2 cov AB, s 2 B] of the perceptual distribution summarize the perception of a particular stimulus along the dimensions assessed. 1 To assess dimensional interaction, one must evaluate at least three of the four stimuli (i.e., A 0 B 0, A 1 B 0, A 0 B 1, A 1 B 1 ) displayed in Figure 1 (Tanner, 1956). These four stimuli are composed of a factorial combination of two dimensions (A and B), with each dimension composed of two values, i 5 {0,1} and j 5 {0,1}. If two dimensions are independent, then (1) the feature value of one dimension will not affect the perception of the other dimension (i.e., m Ai B 0 5 m Ai B 1 and s A 2 i B 0 5 s A 2 i B 1, for all values of i), and (2) the perceptual noise on the two dimensions will be uncorrelated (Ashby & Townsend, 1986). The former is a characteristic of the relation between stimuli and is termed perceptual separability. The latter is a characteristic of an individual stimulus and is termed perceptual independence. Figure 1 illustrates a variety of perceptual spaces. In Figure 1, perceptual separability holds for a dimension if (1) the dotted lines connecting the bivariate means of the perceptual distributions are perpendicular to that dimension s axis and (2) the equal likelihood contours of the perceptual distributions along that dimension are constant across the values of the other dimension. Perceptual inde Copyright 2003 Psychonomic Society, Inc.

2 1146 COHEN strategy decisional separability. Decisional separability is represented in the perceptual space by decision criteria that are parallel to the perceptual axes. Figure 1. Three two-dimensional perceptual spaces. The dotted polygon connects the bivariate means of the perceptual distributions. The solid ellipses represent equal likelihood contours of the perceptual distributions. The top graph shows a perceptual space where both perceptual separability and perceptual independence hold. In the middle graph, perceptual separability fails for both dimensions, and perceptual independence holds. In the bottom graph, perceptual separability fails for both dimensions, and perceptual independence fails for two stimuli. pendence holds if the major and the minor axes of the ellipse representing equal likelihood contours of the perceptual distribution are parallel to the perceptual axes (i.e., the cloud of points will be circular, or if elliptical, the orientation of the cloud of points should be vertical or horizontal). In addition to modeling the perceptual space, GRT also models the response strategy of the participant. When making a response, the participant can judge the perceptual effect of each dimension separately, or his or her assessment of one dimension may affect his or her assessment of the other dimension. GRT terms an independent response Processing Model Ashby (1989) instantiates GRT in a processing model that incorporates a separate processing stream for each dimension assessed. Each processing stream is divided into three consecutive stages in which dimensional interaction may occur. Figure 2 presents a version of that model. Stage 1 is characterized by the sensitivity of each processing stream to various feature dimensions. A Stage 1 interaction occurs when the receptors of the stream processing Dimension A are also sensitive to variations in Dimension B. For example, if the receptors in the stream that is processing form are also somewhat sensitive to hue, variation in the hue dimension may affect the perception of the form dimension. Thus, Stage 1 interaction is an interaction between a single processing stream and the dimensions that make up the physical stimulus. Stage 2 is characterized by communication between processing streams. Stage 2 interaction occurs when the activation of Stream A influences the activation of Stream B. For example, the stream that is processing form may also be sensitive to the activation of the stream that is processing hue. In this instance, the stream that is processing hue mediates the influence of hue on the stream that is processing form. Thus, Stage 2 interaction is an interaction between (at least) two processing streams. Stage 3 is characterized by the participant s response strategy. This stage addresses whether the participant s expressed judgments of the stimulus dimensions influence one and other. The participant may judge each dimension without consideration of the other dimension, or he or she may conflate the two judgments. Whereas interactions of Stages 1 and 2 are perceptual, Stage 3 interaction is cognitive. In a version of the processing model above, Ashby (1989) noted associations between (1) the failure of perceptual separability (termed perceptual integrality) and Stage 1 interaction, (2) the failure of perceptual independence (termed perceptual dependence) and Stage 2 interaction, and (3) the failure of decisional separability (termed decisional integrality) and Stage 3 interaction. Although Ashby and Maddox (1994) stated that stimulus components X and Y are perceptually separable if and only if they are processed by separate channels with nonoverlapping tuning curves (i.e., Stage 1 interaction) ( p. 434), Ashby (1989) noted that interactions in Stage 1 or Stage 2 can produce failures of perceptual separability. 2 Analysis Procedures A few specialized analysis procedures have been developed to assess the perceptual and decisional processes in GRT space. 3 These procedures include Kadlec and Townsend s (1992a, 1992b) macro- and microanalyses, Wickens s (1992) maximum-likelihood model, and Maddox and Ashby s (1996) parameter estimation procedures. Perhaps the most widely implemented of these procedures

3 DIRECT ESTIMATION OF MULTIDIMENSIONAL PERCEPTUAL DISTRIBUTIONS 1147 Figure 2. A model of three broad stages of processing during which dimensional interaction may occur. are Kadlec and Townsend s macro- and microanalyses, which are based on SDT analysis. Kadlec and Townsend (1992a, 1992b) proposed an experimental and statistical technique based on SDT to assess perceptual separability, perceptual independence, and decisional separability. In its simplest form, they proposed that experimenters conduct a complete identification experiment, using stimuli composed of a factorial combination of two dimensions (A and B), with each dimension composed of two values, i 5 {0,1} and j 5 {0,1}. In a complete identification experiment, participants are briefly presented one of the four stimuli (i.e., A 0 B 0, A 1 B 0, A 0 B 1, or A 1 B 1 ) and are asked to identify which stimulus was presented. Kadlec and Townsend described (1) a macroanalysis, based on univariate marginal distributions, that is used to place at least one axis of each perceptual distribution in GRT perceptual space and (2) a microanalysis, based on conditional d9s (i.e., assessing d9 along Dimension A conditional on a hit or a miss on Dimension B), that is used to provide some limited information about the variance/covariance matrix of the perceptual distributions. Although Kadlec and Townsend s procedure does not distinguish between perceptual and decisional processes, it can test properties of both perceptual and decisional processes. (For a complete description of macro- and microanalyses, see Kadlec & Townsend, 1992a, 1992b.) Several other procedures, based on parameter estimation, have also been proposed. Two such parameter estimation procedures are those of Wickens (1992) and Maddox and Ashby (1996). Wickens s maximum-likelihood model uses a maximum-likelihood procedure to estimate the parameters of each multivariate Gaussian perceptual distribution. The estimation procedure is based on polychoric correlation analysis and assumes decisional separability. Wickens s model does not address the distinction between decisional and perceptual effects. Maddox and Ashby proposed a more extensive parameter estimation approach that addresses the distinction between decisional and perceptual processes by developing and comparing models with different assumptions about these processes (e.g., Maddox, 2001; Maddox & Ashby, 1996; Maddox & Bogdanov, 2000). Whereas Maddox and Ashby s procedure provides more precise information about perceptual and decisional processes in multidimensional space than do both Kadlec and Townsend s (1992a, 1992b) macroand microanalyses and Wickens s maximum-likelihood model, it requires large amounts of data to implement (i.e., the degrees of freedom must exceed the number of parameters to be estimated; see Maddox & Ashby, 1996). In sum, GRT extends SDT to multidimensional space. Unfortunately, assessment of perceptual and decisional processes in multidimensional space is difficult, and the available procedures to recover information about these processes require specialized statistical programs and skills. Below, I will describe a relatively simple regression model with which to assess the perceptual and decisional processes associated with detecting multidimensional stimuli in GRT space. The regression model can be run on any available statistical analysis program and provides estimates of m and S for each perceptual distribution assessed. REGRESSION MODEL DeCarlo (1998) has illustrated the advantages of using a generalized linear model to estimate the parameters of unidimensional SDT analysis. Similar to standard SDT analysis, DeCarlo used participants ratings to identify regions between criteria that intersect underlying logistic or normal distributions. The procedure proposed here uses participants ratings as direct estimates of the perceptual effects of the stimuli (e.g., Hirsch, Hylton, & Graham, 1982). If one has a direct estimate of the perceptual effect of each dimension on every trial of a complete identification experiment, one can estimate the m and S of each perceptual distribution, using a regression model that is theoretically linked to (1) the stages of the processing model

4 1148 COHEN described above and (2) the perceptual space described in GRT. Ashby (1989) has outlined such a model and the equations that predict its behavior on the basis of both observable and unobserverable variables. I will present a version of Ashby s (1989) equations based solely on observable variables. In the present example, the participant s perceptual effects for each dimension are estimated using a fine ordinal scale of the participant s confidence in the presence or absence of an assigned target value in each dimension (discussion of the assumed qualities of the response scale will be presented below in the Assumptions of the Response Scale section). The regression model predicts a participant s estimate of the perceptual effect of each stimulus dimension (a i and b j ) from (1) the feature value of Dimension A (A i ), (2) the feature value of Dimension B (B j ), (3) an interaction between the feature values of Dimensions A and B (A i B j ), and (4) the response on the secondary dimension (a i or b j when a i is predicted, Dimension A is primary and Dimension B is secondary; when b j is predicted, Dimension B is primary and Dimension A is secondary). Thus, for Dimension A, a = + ( ) + ( ) b e a b A b B + b ( A B ) + b ( ) +, i a a1 i a2 j a3 i j a4 j a and similarly for Dimension B, b = a + b ( B ) + b ( A ) + b ( A B ) + b ( a ) + e (1a) j b b1 j b2 i b3 i j b4 i b. (1b) The response pair (a i, b j ) is assumed to be a direct estimate of the mean of the perceptual distribution associated with each stimulus (i.e., A 0 B 0, A 1 B 0, A 0 B 1, and A 1 B 1 ). b. 1 carries the influence of the value of the primary dimension on the participant s response to the primary dimension. b. 2 and b. 3 carry the influence of the secondary dimension on the participant s response to the primary dimension. Specifically, b. 2 carries the additive effect and b. 3 carries the nonadditive effect of the secondary dimension on the participant s response to the primary dimension. For example, if Dimension A s influence on Dimension B is non-zero and constant across all the stimuli, then b b2 Þ 0 and b b If Dimension A s influence on Dimension B is dependent on properties of the stimulus, then b b3 Þ 0. b. 4 carries the influence of the participant s response to the secondary dimension on his or her response to the primary dimension. The term representing the response on the secondary dimension (i.e., b. 4 ) creates a dependency between the two equations. However, because each equation includes identical values of regressors (e.g., A i, B j, and A i B j ), a separate estimation of each equation without the secondary dimension response term (i.e., b. 4 ), together with a correlation of residuals, provides the same information as a simultaneous estimation of the equations with the secondary dimension response terms (Dwivedi & Srivastava, 1978; Kmenta, 1986). Therefore, to estimate a i and b j, one simply regresses the stimulus variables on each response variable, and a b = i a + a b a ( i ) + b a2 ( j ) + b a3 ( i j ) + a 1 A B A B = j a + b b b1( B j ) + b b2 ( A i ) + b b3 ( A i B j ) + b, (2a) (2b) and assesses the correlation between the residuals of the two equations ( r eaeb ). The interpretation of how the parameters of the regression model relate to the processing model described above and to GRT are summarized in Table 1. The correlation between the residuals of the Equations 2a and 2b (r eaeb ) indicates whether a dependency exists between the participant s responses to each dimension (a i and b j ). A dependency between a participant s responses to each dimension is assumed to result from either a Stage 2 interaction or a Stage 3 interaction. For example, if the activation of Stream A influences Stream B (i.e., a Stage 2 interaction), the noise in Stream A will correlate with the noise in Stream B. If a i and b j are valid estimates of the perceptual effects (an assumption of the regression model), evidence of this correlated noise will be apparent in the participant s responses (i.e., r eaeb Þ 0). Similarly, if a participant s response on Dimension A influences his or her response on Dimension B (i.e., a Stage 3 interaction), a participant s responses to each dimension will be correlated (i.e., r eaeb Þ 0). Thus, although r eaeb indicates whether interstream influence has been exerted, one cannot mathematically determine whether the influence was exerted during processing (Stage 2) or after processing (Stage 3). 4 In short, if a Stage 2 and/or a Stage 3 interaction occurs, then r eaeb Þ 0. The converse is also true; if r eaeb 5 0, then no interaction occurs in Stage 2 or 3. Because perceptual independence and decisional separability correspond to Stage 2 and Stage 3, respectively, failure of perceptual independence or decisional separability will produce r eaeb Þ 0. Likewise, if r eaeb 5 0, one can conclude that perceptual independence and decisional separability hold. Significant bs for the third and/or the fourth terms (b. 2 and/or b. 3 ) of Equations 2a and 2b indicate mean shift integrality as described by GRT. Mean shift integrality is a form of perceptual integrality in which the perception of the secondary dimension influences the mean perceptual effect of the primary dimension. Because the stimulus variables of the secondary dimension (e.g., B j and A i B j ) contribute to perceptual integrality (i.e., b. 2 Þ 0 and/or b. 3 Þ 0) and these same stimulus variables influence the dependent term (e.g., b. 4 ), both the original stimulus variables and the dependent term can produce perceptual integrality (recall that when implementing Equations 2a and 2b, the influence of the dependent term is expressed in r eaeb ). Therefore, Stage 1, 2, and 3 interactions may all be e e

5 DIRECT ESTIMATION OF MULTIDIMENSIONAL PERCEPTUAL DISTRIBUTIONS 1149 Table 1 A Summary of the Relation Between the Parameters of the Regression Model, the Processing Model, and General Recognition Theory (GRT) GRT Processing Model Error Correlation Error Correlation r eaeb 5 0 r eaeb Þ 0 r eaeb 5 0 r eaeb Þ 0 Dimension Regression Coefficients Stage Interaction A B A B b a2 5 0 and b a3 5 0 b a2 Þ 0 and/or b a3 Þ 0 ì í î ì í î b b A Å B A Å B PS Hold Hold Hold Hold and 2 A Å B A «B PI Hold Fail b b A Å B A «B DS Hold Fail b b2 Þ 0 1 A B A B PS Hold Fail Hold Fail and/or 2 A Å B A «B PI Hold Fail b b3 Þ 0 3 A Å B A «B DS Hold Fail b b A B A B PS Fail Hold Fail Hold and 2 A Å B A «B PI Hold Fail b b A Å B A «B DS Hold Fail b b2 Þ 0 1 A B A «B PS Fail Fail Fail Fail and/or 2 A Å B A «B PI Hold Fail b b3 Þ 0 3 A Å B A «B DS Hold Fail Note PS denotes perceptual separability; PI denotes perceptual independence; DS denotes decisional separability; Å denotes no interaction; and denote that an interaction is present and indicate the direction of influence (e.g., A B indicates that Dimension A influences the perception (Stage 1 or 2) or the decision (Stage 3) processes of Dimension B); «denotes that an interaction is present but the direction of the influence is indeterminate; brackets ( ) indicate that at least one of the interactions connected with the brackets is present. a source of perceptual integrality. To intuitively understand how the dependent term can produce perceptual integrality, suppose that one s perception of form (c form ) is a function of the two following stimulus variables: the form dimension (F i ) and an interaction between the form and the hue dimensions (F i H j ). Further suppose that one s perception of hue (c hue ) is influenced by the hue dimension, H j, and c form (i.e., a Stage 2 interaction). In this instance, the participant s hue response will be influenced by H j directly and by F i and F i H j indirectly (through c form ). Thus, the hue response will appear to be influenced by all three stimulus variables (e.g., F i, H j, and F i H j ) and have correlated residuals with the form description. A similar interaction at Stage 3 produces the same result. Therefore, perceptual integrality (as defined by GRT) uniquely specifies a Stage 1 interaction only when r eaeb 5 0. If a perceptual integrality exists and r eaeb Þ 0, one cannot mathematically determine which of the three stages produced the perceptual integrality. Perceptual Versus Decisional Effects The relation between the parameters of the regression model and GRT illustrates the importance of establishing decisional separability. That is, because decisional integrality (Stage 3 interaction) can produce patterns of data that mimic both perceptual integrality and perceptual dependence, if decisional separability fails one cannot determine whether any or all effects present are the result of perceptual or cognitive processes. Fortunately, the regression model provides some limited information about decisional separability. That is, failure of decisional separability will, by definition, result in dependence between the participant s responses to each dimension. Therefore, if r eaeb 5 0, decisional separability must hold. Nevertheless, if r eaeb Þ 0, the regression model cannot mathematically distinguish decisional integrality from perceptual dependence. The regression model is not alone in its difficulty in differentiating a participant s decision strategy from his or her perception in multidimensional SDT space. When one generalizes SDT to a multidimensional space, it becomes exceedingly difficult to determine the multidimensional response bias (i.e., decisional separability). Kadlec and Townsend (1992b) proposed that micro- and macroanalyses can provide some clues concerning the participant s twodimensional decision strategy if one (1) assumes linear decision bounds and (2) has some knowledge of the placement and shape of the perceptual distributions. Because one must have some information concerning the perceptual space to determine the decision space (Kadlec & Townsend, 1992b), the information about two-dimensional response bias that one can glean from micro- and macroanalyses is limited (see Kadlec & Townsend, 1992a). In the absence of convenient statistical techniques for differentiating a participant s decision strategy from his or her perception in multidimensional SDT space, researchers have turned to logical arguments and/or a priori assumptions. For example, often researchers assume decisional separability and, therefore, attribute interactions to Stages 1 and/or 2 (e.g., Olzak, 1986; Wickens, 1992; Wickens & Olzak, 1992). Others have assumed that participants use a single response strategy during experimental sessions.

6 1150 COHEN Therefore, if perceptual and decisional separability holds for one dimension, researchers assume that any interactions present in other dimensions are not due to a Stage 3 interaction (Ashby, 1989). Finally, researchers implement experimental techniques to encourage decisional separability. For example, Maddox and Bogdanov (2000) used a response-terminated matching task to assess a participant s perceptual space, because they assumed that such a task would minimize decisional integrality. The authors later confirmed their model with a separate categorization task. In light of the fact that the regression model is limited in its ability to distinguish perceptual from decisional processes, it is recommended that researchers implement experimental controls to encourage decisional separability and, if r eaeb Þ 0, assess whether the pattern of interactions can be more parsimoniously explained by decisional or perceptual processes (e.g., Ashby, 1989). The following experimental controls may encourage decisional separability: (1) requiring participants to rate each dimension separately, (2) randomizing trial type and order of responses, and (3) not providing corrective feedback. Specifically, Ashby and Maddox (1994) suggested that a naive participant may identify dimensions separately if instructed to do so, even if doing so does not result in optimal performance. Furthermore, by randomizing trial type and order of responses, participants are prevented from devising stimulus-specific attentional and response strategies, because they cannot anticipate which stimulus will be presented and which dimension will be requested first. Finally, because corrective feedback may prompt even a naive participant to adopt an optimal decision strategy (even if the optimal response strategy is not decisionally separable), corrective feedback should be avoided. Assumptions of the Response Scale Perhaps the most formidable obstacle to implementing the regression model is the choice of the response scale used to estimate the perceptual effects of each dimension. Most response scales require participants to assign numbers to the perceived magnitudes of stimuli (e.g., magnitude estimation, category rating, etc.). Although the regression model does not constrain the form of this scale, it does assume that a participant s estimates are valid. The issues associated with the validity of numerical estimates of perceptual experiences have been discussed extensively elsewhere (e.g., Gescheider, 1988; Marks & Algom, 1998; Stevens, 1986). Generally speaking, the validity of subjective reports of internal experience cannot be unequivocally established and, therefore, must be assumed. Numerical estimates of perceptual effects have been used successfully to map unidimensional space (Braida & Durlach, 1972; Cohen & Lecci, 2001; Durlach & Braida, 1969). Specifically, Durlach and Braida presented a formal decision model that links a participant s numerical descriptions of the perceptual effect of a stimulus to the hypothesized underlying distributions of unidimensional SDT. Braida and Durlach (1972) provided empirical evidence that numerical descriptions (i.e., magnitude estimates, absolute estimates, and category ratings) and unidimensional SDT analysis may afford equivalent information. Recently, Cohen and Lecci generalized this model to assess the perceptual effect of specific dimensions of a stimulus. If the practical application of these findings can be generalized to a multidimensional perceptual space, then one may use these measures (i.e., magnitude estimates, absolute estimates, and category ratings) to map GRT perceptual space, using the regression model. Researchers assessing dimensional interaction in multidimensional space are interested in the relative effects of one stimulus dimension on the perception of the other stimulus dimension. Such an assessment requires that the participant apply the response scale to the perceptual effects according to a rule (other than randomness; Stevens, 1986). Such a mapping ensures that the participant s estimates are ordinally related to the actual perceptual effects but does not ensure equal intervals between successive numbers. Finally, the mapping rule need not be the same for each dimension assessed. Therefore, to assess dimensional interaction, using the regression model, the participant s estimates must be at least ordinally related to his or her perceptual experiences, and decisional separability must hold. For example, one may assess the effects of Dimension A on the perception of Dimension B (i.e., perceptual separability) by comparing, along Dimension B, (1) the means and variances of A 0 B 0 and A 1 B 0 and (2) the means and variances of A 0 B 1 and A 1 B 1. If perceptual separability holds for Dimension B, A 0 B 0 should be identical to A 1 B 0, and A 0 B 1 should be identical to A 1 B 1, along Dimension B. Furthermore, if decisional separability holds for Dimension B, participants criteria will be perpendicular to the axis representing Dimension B. If both perceptual and decisional separability holds, participants ratings will intersect A 0 B 0 in exactly the same places that they intersect A 1 B 0, and participants ratings will intersect A 0 B 1 in exactly the same places that they intersect A 1 B 1, regardless of the ordinal scale that the participant adopts. Therefore, if the participant s perceptual and decisional processes associated with Dimension B are unaffected by Dimension A (i.e., perceptual and decisional separability), the means and variances of the participant s ordinal estimates of Dimension B will not differ for A 0 B 0 and A 1 B 0 or A 0 B 1 and A 1 B 1. Similarly, the assessment of perceptual separability is not affected by unidimensional bias. That is, if decisional and perceptual separability holds and a participant has a liberal bias when estimating the perceptual effects of Dimension B, this bias will have equal effects on the participant s estimates of Dimension B for all stimuli, regardless of the level of Dimension A. If either decisional or perceptual separability fails, the estimates of the mean and variance of the relevant perceptual distributions will be unequal, again, regardless of the ordinal scale that the participant adopts and any unidimensional response bias. Ordinal estimates also provide information about perceptual independence. Ashby (1988) has shown that the interrating correlation is related to the correlation between the underlying perceptual distributions. However, Ashby (1988) also has shown that unidimensional response bias and the participant s choice of an ordinal scale may result in an un-

7 DIRECT ESTIMATION OF MULTIDIMENSIONAL PERCEPTUAL DISTRIBUTIONS 1151 derestimation of that relation. Nevertheless, the presence or absence of a relation can be assessed with sufficient power. Although ordinal measures of perceptual effects can be used to assess the presence or absence of dimensional interaction, they are limited in two ways. First, the participant s implementation of the ordinal scale may result in reduced statistical power. Specifically, the greater the inequality of intervals between successive numbers, the greater the reduction in statistical power. Second, the ordinal measure does not provide information about the absolute magnitudes of effects. To draw conclusions about absolute magnitudes, one must assume that the participant s estimates carry more than ordinal information. The appropriateness of the regression model is also complicated by the statistical assumption that the dependent variable contains at least interval information. Conover and Iman (1981) have shown, however, that applying a standard parametric procedure (such as a multiple regression) to ranked scores (which are equivalent to ratings data) results in a powerful test that makes no distributional or interval assumptions. Therefore, these tests are appropriate for use with an ordinal measure of perceptual effects if one draws conclusions only about relative magnitudes. In sum, the regression model makes several assumptions about the information contained in a participant s numerical responses. If these assumptions hold (or are common across analytic procedures), the regression model should produce the same conclusions as more established analytic procedures. Thus, to validate the regression model, I (1) conducted a concurrent ratings experiment in which the perceptual space associated with hue and form was assessed, using isoluminant stimuli, (2) mapped the perceptual space, using both Kadlec and Townsend s (1992a, 1992b) macro- and microanalyses and the regression model, and (3) compared the results of the two analyses. Kadlec and Townsend s analytic procedure was chosen as a comparison because it is founded on SDT analysis and the validity of SDT analysis has been well studied (e.g., Green & Swets, 1966; Macmillan & Creelman, 1991). EXPERIMENT The present experiment is an extension of Cohen (1997). Cohen reported a visual detection experiment that provided evidence that color and form are not detected independently in early vision. The present experiment extended Cohen by using isoluminant stimuli and obtaining separate estimates of the perceptual effect of each perceptual dimension. Method Participants Sixteen participants from the general psychology subject pool volunteered to participate in three sessions over 3 days. Apparatus and Stimuli All the stimuli were presented on a 15-in. color monitor with a 60- Hz refresh rate, controlled by a Pentium microcomputer using a DOS operating system. The resolution of the monitor was A set of elements was constructed that was a factorial combination of two hues and two forms. Each element subtended 1º of visual angle. The two hues were isoluminant shades of purple and blue. 5 Isoluminance was achieved for each participant by using a flicker fusion task prior to the experiment. In the flicker fusion task, each participant was presented a square patch of color, subtending 1º of visual angle, that alternated between the two colors at the speed of the refresh rate (16.7 msec; using palette animation). Initially, there was a perceptual flicker when the shades alternated. The participant was instructed to adjust the luminance of one of the colors until he or she could not see the patch flicker. The participant adjusted the luminance by using the 4 and 6 keys on the keypad on the keyboard. The 4 key increased the luminance and the 6 key decreased the luminance of the color. When the participant no longer perceived the flicker, he or she hit the Enter key. The resulting values were used as the two hues in the visual detection task described below. The two levels of the form dimension were a square and an octagon. The length of the oblique sides of the octagon was adjusted in a pilot experiment so that the participants were equally sensitive to variation in the hue and the form dimensions. Whereas the luminance of the hue dimension was adjusted for each participant, the shapes of the stimuli were constant across participants. For each participant, one feature value from each feature dimension was chosen as a target. The target values were counterbalanced between participants. The target value is noted with a subscript of 1, and the nontarget value is noted with a subscript of 0. Thus there were four types of elements: (1) a nontarget element, F 0 H 0, (2) a form target element, F 1 H 0, (3) a hue target element, F 0 H 1, and (4) a form and hue target element, F 1 H 1. Each trial contained four elements arranged in a square around the center of the computer monitor. This matrix of four elements subtended 5º of visual angle on a side. There were five trial types (see Figure 3): (1) a no-target trial composed of four F 0 H 0 elements, (2) a form target trial composed of three F 0 H 0 elements and one F 1 H 0, (3) a hue target trial composed of three F 0 H 0 elements and one F 0 H 1, (4) a coincident target trial composed of three F 0 H 0 elements and one F 1 H 1, and (5) a disparate target trial composed of two F 0 H 0 elements, one F 1 H 0, and one F 0 H 1. Figure 3. Examples of the five trial types in the present experiment.

8 1152 COHEN On those trials in which at least one target element was present, the target element was presented in a randomly chosen spatial position. The target elements were presented in random locations to discourage the participants from focusing their attention on a restricted portion of the display. Each positive trial type (trials that contained target elements) was presented with a probability of.125, and the negative trials (trials that contained no target elements) were presented with a probability of.5. Randomly presenting the trial types discouraged the participants from relying on a trial-specific strategy. The display time was manipulated between participants. Each participant was presented displays for either 33 or 83 msec. Because Braida and Durlach (1972) had shown showed that magnitude estimates and category ratings provide equivalent information in a detection task, an 18-point ordinal response scale was used in the present experiment. Eighteen categories were chosen because participants generally use between 10 and 20 numbers in a magnitude estimation task (Luce & Krumhansl, 1988). Furthermore, when a set of restricted responses is used, the results are essentially unchanged if that set is sufficiently large (Durlach & Braida, 1969). To facilitate the participant s use of the entire scale, the response screens were constructed to be a cross between a visual and a semantic scale. Specif ically, the numbers 1 through 18 were listed horizontally across the screen and the participant moved a box across the numbers to indicate his or her response. Thus, both the number and the position of the box were indicators of magnitude. There was a response screen for each dimension assessed. One response screen asked the participant to rate his or her confidence that the target form feature value was present in the display. At the top of this response screen, the following sentence was written: Please rate your confidence that the target FORM was present. Underneath this sentence, the word form was written in large letters centered horizontally on the screen. Below the word form, the numbers 1 18 were listed horizontally across the computer monitor. Over the number 1, the words very sure no target present were written. Over the number 18, the words very sure target present were written. The number 1 was framed by a box. The participant manipulated the position of the box by pressing the 4 and 6 keys on the computer keypad. The 4 key moved the box to the left; the 6 key moved the box to the right. When the box was moved to left past the rating of 1 (or to the right past the rating of 18), it wrapped around the rating scale to the number 18 (or 1). When the participant positioned the box over his or her intended rating, he or she hit the Enter key to finalize his or her choice. The second response screen asked the participant to rate his or her confidence that the hue target feature value was present. This screen was identical to the form response screen, with the exception that the word form was replaced by the word color throughout. 6 On every trial the participant responded to both response screens. The order in which the response screens were presented was randomly determined on each trial. Procedure Each participant was tested individually in a small dark room. The participant was given both verbal and written instructions. The instructions indicated which target feature values the participant was to identify. The participant was shown those features, as well as the nontarget feature values. The participant was shown self-timed examples of each trial type. When the participant indicated understanding of the task, the experiment began. Each trial began with a 500-msec blank screen, followed by the trial matrix and by a mask of yellow stars. The mask remained visible for 1 sec. Following the mask, the two response screens were presented in random order. There was no feedback. Each participant participated in three sessions over the course of 1 week. Each session consisted of 40 practice trials, followed by 480 experimental trials per session. The practice trials contained 5 trials of each of the four target trial types and 20 nontarget trials. The experimental trials contained 60 trials of each of the four target trial types and 240 nontarget trials. The trial types were presented randomly within and across sessions. The participant was allowed one self-timed break after 240 trials. Results Below are the results of the micro- and macroanalyses, followed by those of the regression analysis. Each analysis was first calculated on the individual participant s data and then concatenated across participants using a summedz meta-analysis (for a detailed description of the analysis, see Cohen, 1997; Rosenthal & Rubin, 1986; Strube, 1985; Strube & Miller, 1986). The summed-z meta-analysis was appropriate because the degrees of freedom associated with each t test were large enough for the t distribution to be a good approximation of a Gaussian. The meta-analysis was particularly well suited to the present data because it emphasized individual effects while simultaneously providing an overall summary of the data. 7 Separate analyses were performed for each condition 3 display time (33 vs. 83 msec). All reported p values indicate significance as calculated by the meta-analysis, and because multiple tests per condition were calculated, a Use of the Rating Scale The precision of the regression analysis is dependent in part on the participant s ability to apply the rating scale. If the participant uses only a portion of the scale, the power to detect perceptual integrality and perceptual dependence is reduced. In general, although most of the participants used the entire response scale, they tended to gravitate toward the ratings on the high and low ends of the scale. Specifically, the participants average form rating was (SD ), and the average color rating was 8.46 (SD ). When assessing form, only 2 participants did not use the entire 18 response criteria (those 2 participants used 14 and 16 response criteria). When assessing hue, 9 participants did not use the entire 18 response criteria, with 8 of those 9 participants using 15 or more response criteria (the 9th participant used 8 response criteria). When a target was presented, the participants used Ratings 13 and greater 89% of the time for form responses (M , SD ) and 95% of the time for hue responses (M , SD ). When a target was not presented, the participants used Ratings % of the time for form responses (M , SD ) and 85% of the time for hue responses (M , SD ). Macro- and Microanalyses Because the participants tended to gravitate toward the ratings on the high end of the scale when the target was presented, Ratings (inclusive) were used as criteria when the micro- and macroanalyses were calculated. As would be predicted by decisional separability, the results were stable across criteria. Therefore, the marginal d9 and marginal response invariance (MRI) results are reported only for Criterion 15 and are displayed in Tables 2 and 3. Marginal d9. The marginal d9 analysis assesses whether the average perceptual effect of one dimension depends

9 DIRECT ESTIMATION OF MULTIDIMENSIONAL PERCEPTUAL DISTRIBUTIONS 1153 Table 2 A Summary of the Results of the Macroanalyses as They Relate to Perceptual Separability for Display Times of 33 and 83 msec Marginal d9 Coincident Disparate Dimension Time (msec) (F 1 H 1 ) p (F 1 H 1 ) p Form 33 d9 FH n.s. d9 FH ,.001 d9 FH d9 FH d9 FH ,.001 d9 FH ,.001 d9 FH d9 FH Hue 33 d9 F0H n.s. d9 F0H n.s. d9 F1H d9 F1H d9 F0H ,.01 d9 F0H n.s. d9 F1H d9 F1H MRI Coincident Disparate No Target (F 1 H 1 ) p (F 1 H 1 ) p (F 0 H 0 ) p Form 33 p( f 1 F 1 H 0 ) 5.86 p( f 1 F 1 H 1 ) 5.88 n.s. p( f 1 F 1 H 0 ) 5.86 p( f 1 F 1 H 1 ) 5.81,.001 p( f 0 F 0 H 0 ) 5.73 p( f 0 F 0 H 1 ) 5.69, p( f 1 F 1 H 0 ) 5.88 p( f 1 F 1 H 1 ) 5.96,.001 p( f 1 F 1 H 0 ) 5.88 p( f 1 F 1 H 1 ) 5.74,.001 p( f 0 F 0 H 0 ) 5.67 p( f 0 F 0 H 1 ) 5.64,.001 Hue 33 p(h 1 F 0 H 1 ) 5.93 p(h 1 F 1 H 1 ) 5.94 n.s. p(h 1 F 0 H 1 ) 5.93 p(h 1 F 1 H 1 ) 5.93 n.s. p(h 0 F 0 H 0 ) 5.91 p(h 0 F 1 H 0 ) 5.90 n.s. 83 p(h 1 F 0 H 1 ) 5.94 p(h 1 F 1 H 1 ) 5.98,.001 p(h 1 F 0 H 1 ) 5.94 p(h 1 F 1 H 1 ) 5.95 n.s. p(h 0 F 0 H 0 ) 5.94 p(h 0 F 1 H 0 ) 5.93 n.s. Note Perceptual separability is assessed separately for coincident and disparate targets. Perceptual separability fails for a dimension (form or hue) if the compared measures differ ( p,.01). Perceptual separability fails for both coincident and disparate conditions within a dimension if the MRI no-target measures differ. on the level of the other dimension. Specifically, marginal d9 analysis assesses the relation between the means of the perceptual distributions. For marginal d9s to be equal, the following must be true: d = AB0 AB. 1 (3) When the participants identified the stimulus form, marginal d9s were equal only in the 33-msec coincident condition (first two rows of Table 2). For display times of 33 and 83 msec, the participants were more sensitive to the form target feature value in the form-only condition than in the disparate condition. Thus, the presence of a hue target feature value in a location spatially different from that d of the form target feature value inhibited detection of the form target feature value. For a display time of 83 msec, the participants were less sensitive to a form target feature value in the form-only condition than in the coincident condition. Thus, the presence of a hue target feature value in the same spatial location as the form target feature value enhanced detection of the form target feature value. When the participants identified the stimulus hue, marginal d9s were equal in the disparate condition for both display times and in the 33-msec coincident condition (third and fourth rows of Table 2). For a display time of 83 msec, the participants were less sensitive to a hue target feature value in the hue-only condition than in the coincident con- Table 3 A Summary of the Results of the Microanalyses as They Relate to Perceptual Independence for Display Times of 33 and 83 msec Time Sampling Independence Tetrachoric Correlation (msec) Stimulus p( f 1 h 1 F i H j ) Independence p r SE p 33 no target (F 0 H 0 ) , ,.001 form (F 1 H 0 ) n.s n.s. hue (F 0 H 1 ) n.s ,.001 disparate (F 1 H 1 ) n.s ,.001 coincident (F 1 H 1 ) n.s , no target (F 0 H 0 ) , ,.001 form (F 1 H 0 ) n.s ,.001 hue (F 0 H 1 ) n.s n.s. disparate (F 1 H 1 ) n.s ,.001 coincident (F 1 H 1 ) n.s ,.001 Note Perceptual independence is assessed separately for each stimulus. Perceptual independence fails if p,.01.

10 1154 COHEN dition. Thus, at 83 msec, the presence of a form target feature value in the same spatial location as the hue target feature value enhanced detection of the hue target feature value. Figure 4 shows the mean placement of the perceptual distributions, using a technique described in Kadlec and Hicks (1998). Specifically, the mean of the F 0 H 0 perceptual distribution is placed at (0,0). The remaining perceptual distributions were placed at the distance specified by the relevant marginal d9 from the F 0 H 0 perceptual distribution. For the two stimuli that lacked a marginal d9 on one axis (i.e., F 0 H 1, F 1 H 0 ), the distance on that axis was assumed to be zero. If perceptual separability holds for a dimension, the bivariate means of the perceptual distributions will fall on the dotted lines perpendicular to that dimension s axis. Because of the method used to construct Figure 4, this condition is free to fail only for the dual target distributions. Perceptual independence is represented in Figure 4 by equal likelihood contours based on the tetrachoric correlations, assuming bivariate normal distributions and equal variances. Perceptual independence holds only if the major and minor axes of the ellipse representing equal likelihood contours are parallel to the hue and form axes. Marginal response invariance. The MRI analysis assesses whether the probability of correctly recognizing one dimension depends on the level of the other dimension. Whereas marginal d9 provides information about the placement of the perceptual distributions, MRI will fail if (1) m Ai B 0 Þ m Ai B 1 or (2) s 2 A i B 0 Þ s 2 A i B 1. For MRI to hold, the following must be true (Ashby & Townsend, 1986): Figure 4. Placement and shapes of the five perceptual distributions associated with the five trial types used in two-dimensional general recognition theory space, based on marginal d9s. The dotted rectangle indicates the predicted position of the bivariate means of the perceptual distributions if hue and form are perceptually separable. The solid ellipses represent equal likelihood contours of the perceptual distributions derived from the results of the tetrachoric correlation analysis, assuming equal variance. p a b A B p a b A B ( i 0 i 0 ) + ( i 1 i 0 ) p aib0 AiB1 p aib1 AiB1. i 0, 1 = ( ) + ( ) = (4) Although there are some minimal discrepancies at the high criteria (likely due to unstable data), the results found when MRI was assessed paralleled those found for marginal d9 (the bottom four rows of Table 2). That is, when the participants identified the stimulus form, MRI failed in (1) the no-form target conditions for display times of 33 and 83 msec, (2) the form-only condition versus the disparate condition for display times of 33 and 83 msec, and (3) the form-only condition versus the coincident condition when the display time was 83 msec. When the participants identified the stimulus hue, MRI failed only in the hue-only condition versus the coincident condition when the display time was 83 msec. Marginal b. The marginal b analysis does not provide additional information, given the findings from the marginal d9 and MRI results. Sampling independence. The analysis of sampling independence assesses whether the probability of reporting both dimensions equals the product of the probabilities of reporting each dimension alone. Sampling independence is sensitive to the covariance between dimensions and is a test of perceptual independence. For sampling independence to hold, the following must be true: p( a b A B ) p a b A B p a b = [ ( ) + ( A B ) ] i 1 i j 1 0 i j 1 1 i j * [ p( a b A B ) p a b + ( A B ) ]. 0 1 i j 1 1 i j i 5 0,1 j 5 0,1 (5) Sampling independence held for all conditions except the no-target condition (first four columns of Table 3). In