Reconciling Simplicity and Likelihood Principles in Perceptual Organization

Transcription

1 Psychologcal Revew Copyrght 1996 by the Amercan Psychologcal Assocaton, Inc Vol. 103, No. 3, X/96/$3.00 Reconclng Smplcty and Lkelhood Prncples n Perceptual Organzaton Nck Chater Unversty of Oxford Two prncples of perceptual organzaton have been proposed. The lkelhood prncple, followng H. L. E yon Helmholtz ( 1910 / 1962 ), proposes that perceptual organzaton s chosen to correspond to the most lkely dstal layout. The smplcty prncple, followng Gestalt psychology, suggests that perceptual organzaton s chosen to be as smple as possble. The debate between these two vews has been a central topc n the study of perceptual organzaton. Drawng on mathematcal results n A. N. Kolmogorov's ( 1965 ) complexty theory, the author argues that smplcty and lkelhood are not n competton, but are dentcal. Varous mplcatons for the theory of perceptual organzaton and psychology more generally are outlned. How does the perceptual system derve a complex and structured descrpton of the perceptual world from patterns of actvty at the sensory receptors? Two apparently competng theores of perceptual organzaton have been nfluental. The frst, ntated by Helmholtz ( 1910/1962), advocates the lkelhood prncple: Sensory nput wll be organzed nto the most probable dstal object or event consstent wth that nput. The second, ntated by Werthemer and developed by other Gestalt psychologsts, advocates what Pomerantz and Kubovy (1986) called the smplcty prncple: The perceptual system s vewed as fndng the smplest, rather than the most lkely, perceptual organzaton consstent wth the sensory nput '. There has been consderable theoretcal and emprcal controversy concernng whether lkelhood or smplcty s the governng prncple of perceptual organzaton (e.g., Hatfeld, & Epsten, 1985; Leeuwenberg & Bosele, 1988; Pomerantz and Kubovy, 1986; Rock, 1983). The controversy has been dffcult to settle because nether of the key prncples, lkelhood and smplcty, s clearly defned. Moreover, there have been suspcons that the two prncples are not n fact separate, but are two sdes of the same con. Pomerantz and Kubovy ( 1986 ) cted Mach (1906/1959)--"The vsual sense acts therefore n conformty wth the prncple of economy [.e., smplcty], and at the same tme, n conformty wth the prncple of probablty [.e., lkelhood]" (p. 215)--and themselves have suggested that some resoluton between the two approaches mght be possble. Moreover, the close mathematcal relatonshp between smplcty and lkelhood has been wdely acknowledged n a range of techncal lteratures, n computatonal modelng of percepton (e.g., Mumford, 1992), artfcal ntellgence (e.g., Cheeseman, 1995), and statstcs (e.g., Wallace & Freeman, 1987). But ths relatonshp has not been used to demonstrate I would lke to thank Mke Oaksford and Julan Smth for ther valuable comments on the manuscrpt. Correspondence concernng ths artcle should be addressed to Nck Chater, Department of Expermental Psychology, Unversty of Oxford, South Parks Road, Oxford OX 1 3UD, England. Electronc mal may be sent va lnternet to chater@psy.ox.ae.uk. the equvalence between smplcty and lkelhood prncples n perceptual organzaton. Ths artcle shows that the lkelhood and smplcty prncples of perceptual organzaton can ndeed be rgorously unfed by usng results lnkng smplcty and probablty theory developed wthn the mathematcal theory of Kolmogorov complexty (Chatn, 1966; Kolmogorov, 1965; L & Vtany, 1993; Solomonoff, 1964). Lkelhood Versus Smplcty: The Debate Both the lkelhood and smplcty prncples explan, at least at an ntutve level, a wde range of phenomena of perceptual organzaton. Consder, for example, the Gestalt law of good contnuaton, that perceptual nterpretatons that nvolve contnuous lnes or contours are favored. The lkelhood explanaton s based on the observaton that contnuous lnes and contours are very frequent n the envronment (e.g., Brunswck, 1956). Although t s possble that the nput was generated by dscontnuous lnes or contours that happen, by concdence, to be arranged so that they are n algnment from the perspectve of the vewer, ths possblty s rejected because t s less lkely. The smplcty explanaton, by contrast, suggests that contnuous lnes or contours are mposed on the stmulus when they allow that stmulus to be descrbed more smply. Another example s the tendency to perceptually nterpret ambguous two-dmensonal projectons as generated by threedmensonal shapes contanng only rght angles (Attneave, 1972; Perkns, 1972, 1982; Shepard, 1981 ). The lkelhood explanaton s that rght-angled structures are more frequent n the envronment (at least n the "carpentered" envronment of the typcal experments subject; Segall, Campbell & Herskovts, 1966). The smplcty explanaton s that rght-angled struc- t Ths prncple s also known as prgnanz or the mnmum prncple. The term mnmum prncple comes from the mnmzaton of complexty; the term prgnanz was used wth somewhat broader scope by the Gestalt school, to nclude regularty, symmetry, and other propertes (Koffka, 1935/1963). 566

2 SIMPLICITY VERSUS LIKELIHOOD 567 tures are smpler--for example, they have fewer degrees of freedom-than trapezodal structures. There s a vast range of phenomena that appear consstent wth both lkelhood and smplcty nterpretatons. From the standpont of ths artcle--that the lkelhood and smplcty prncples are equvalent--the common coverage of the two approaches s not surprsng. More nterestng, however, are phenomena that have been taken as evdence for one vew or the other. On the present nterpretaton, such evdence cannot, of course, be taken at face value, as we shall see below. For now, let us consder a typcal example of evdence adduced on each sde of the debate. Putatve Evdence for Lkelhood Lkelhood s wdely assumed to be favored by evdence that shows that preferred perceptual organzaton s nfluenced by factors concernng the structure of the everyday envronment. For example, consder two-dmensonal projectons of a shaded pattern, whch can be seen ether as a bump or an ndentaton (see, e.g., Rock, 1975). The preferred nterpretaton s consstent wth a lght source from above, as n natural lghtng condtons. Thus, the perceptual system appears to choose the nterpretaton that s most lkely, but there s no ntutve dfference between the smplcty of the two nterpretatons. Putatve Evdence for Smplcty Cases of perceptual organzatons that volate, rather than conform to, envronmental constrants are wdely assumed to favor the smplcty account. Leeuwenberg and Bosele (1988) show a schematc drawng of a symmetrcal, two-headed horse. The more lkely nterpretaton, also consstent wth the drawng, s that there are two horses, one occludng the other. But the perceptual system appears to reject lkelhood; t favors the nterpretaton that there s a sngle, rather bzarre, anmal. These consderatons suggest that lkelhood and smplcty cannot be the same prncples; the two appear to gve rse to dfferent predctons. I now dscuss lkelhood and smplcty n turn and argue that, despte appearances, they are dentcal. Lkelhood The lkelhood prncple proposes that the perceptual system chooses the organzaton that corresponds to the most lkely dstal layout consstent wth the sensory nput. But what does t mean to say that a hypotheszed dstal layout, H~, s or s not lkely, gven sensory data,/9.9 The obvous nterpretaton s that ths lkelhood corresponds to the condtonal probablty of the dstal layout, gven the sensory nput: P(H I D). But ths step does not take us very far, because there are a varety of ways n whch probabltes can be nterpreted, and t s not clear whch nterpretaton s approprate n the present case. A frst suggeston s that the probablty can be nterpreted n terms of frequences. Ths frequentst nterpretaton of probablty (yon Mses, 1939/1981 ) s that the probablty of an outcome s the lmt of the frequency of the outcome dvded by the total number of"trals" n a repeated experment. For example, suppose that the repeated experment s tossng a far con. The lmtng frequency of the outcome, "heads," dvded by the total number of trals wll tend toward 0.5 as the number of trals ncreases. Accordng to the frequentst nterpretaton of probablty, ths s what t means to say that the probablty of the con fallng heads, on any gven tral, s 0.5. The frequentst nterpretaton of the condtonal probablty P(A I B) s smply the frequency of trals on whch B and A occur, dvded by the total frequency of trals on whch B occurs. Although the frequentst nterpretaton of probablty s used n many phlosophcal and mathematcal contexts, t does not gve a meanngful nterpretaton of probablty n the present context. Accordng to the frequentst account, the condtonal probablty of the dstal layout, gven the sensory nput, P(H~ I D), s defned as the lmt of the frequency of trals on whch both the sensory nput, D, and the dstal layout, H~, occur, dvded by the total frequency of trals on whch D occurs. But ths lmt s not well defned, because the same sensory nput wll never occur agan--and hence a lmtng frequency can never be obtaned. Of course, smlar nputs wll occur, but ths does not help---because classfyng nputs as smlar requres that some organzaton s mposed upon them, and t s organzaton that we want to explan. In short, then, the frequentst nterpretaton of probablty s defned n terms of lmtng frequences n a repeated experment; and ths s napplcable to probabltes nvolvng sensory nputs, because sensory nputs are never repeated. Therefore a dfferent noton of probablty s requred. The natural alternatve s a subjectve concepton ofprobablty. 2 Accordng to a subjectvst concepton (de Fnett, 1972; Keynes, 1921 ), probablty s a measure of degree of belef n some event or state. Condtonal probablty corresponds to degree of belef n an event or state, gven some other belef n an event or state. How can these deas be translated nto the context of perceptual organzaton? The lkelhood, P(H~ I D), s nterpreted as the degree of belef n the hypothess (H) concernng the dstal layout, gven data (D) concernng the sensory state) In order to calculate lkelhood, Bayes's theorem must be appled: where P(DIH~)P(H~) /'(n~ I D) =, ( l ) P(D) P(D) = ~ PfDII-Ij)PtI-Ij), J Bayes's theorem allows the lkelhood to be calculated from two knds of quantty: (a) terms of the form: P(Hj)--"pror" probabltes for each dstal layout; and (b) terms of the form 2 There s a further possblty, an objectvst or propensty approach to probablty (e.g., Mellor, 1971 ), but t s not clear how ths approach mght be applcable n ths case. 31 use the term lkelhood n the sense used by perceptual theorsts. In statstcs, t s frequently used to denote P(D I H ), the probablty of data gven a hypothess. Classcal samplng theory approaches to statstcal nference often nvolve maxmzng lkelhood (Fsher, 1922 ), n the statstcans sense, rather than posteror probablty, whch s the approprate sense of maxmum lkelhood for perceptual theory.

3 568 CHATER P(D[Hj)--condtonal probabltes for each sensory nput, gven the dstal layout. The lkelhood approach assumes that the perceptual system chooses the most lkely//j, (.e., the H that maxmzes Equaton 1 ). To state ths formally, and for use n the calculatons below, I ntroduce some notaton: and argmax[f(x)] = ~ f()maxmzesf(x) (2) x defnton argmn[f(x)]= ~ f()mnmzesf(x). (3) x defnton Usng ths notaton, the lkelhood prncple states that the chosen hypothess s the Hk such that k = arg max [P(H~L D)]. (4) Smplcty Applyng a smplcty crteron to perceptual organzaton requres clarfcaton on two ponts: what s assessed for smplcty? and how s smplcty measured? I address these n turn. What Is Assessed for Smplcty? The frst queston nvtes an easy answer: that the perceptual organzaton s made as smple as possble. But, taken at face value, ths means that a very smple organzaton (perhaps percevng the dstal scene to be a unform, unstructured feld) would always be a preferred organzaton. Ths possblty s, of course, ruled out by the constrant that the organzaton s consstent wth the sensory nput--and most sensory nputs wll not be consstent wth ths organzaton, because they are hghly nonunform. But ths pont tself rases dffcult questons: What does t mean for an organzaton to be consstent, or compatble, wth a perceptual nput? Can consstency wth the nput be traded aganst smplcty of nterpretaton? 4 If so, how are smplcty and consstency wth the nput to be jontly optmzed? The theoretcal account of smplcty presented below suggests how these questons may be answered. There s, however, a further, and more subtle dffculty: What rules out the smplest possble, "null," perceptual organzaton? Ths organzaton s completely consstent wth the sensory nput, snce t adds nothng to t. Mere consstency or compatblty wth the sensory nput s planly not enough; the perceptual organzaton must also, n some sense, capture regulartes n the sensory nput. Ths nave use of smplcty of perceptual organzaton as a gudng prncple s analogous to a nave use of smplcty as a gudng prncple n scence: Preferrng the smplest theory compatble wth the data would lead to null theores of great smplcty, such as "anythng whatsoever can happen" Such null theores, although smple, are unsatsfactory because they do not explan any of the regulartes n the natural world. In scence, smplcty of theory must be traded aganst explanatory power (Harman, 1965); the same pont apples for perceptual organzaton. But ths appears to mply that perceptual organzaton nvolves the jont optmzaton of two factors, and the relatve nfluence of these two factors s unspecfed. Moreover, ths concluson s unattractve because two notons, smplcty and explanatory power, must be explcated rather than just one. Fortunately, there s an alternatve way to proceed. Ths s to vew perceptual organzaton as a means of encodng the sensory stmulus; and to propose the perceptual organzaton chosen s that whch allows the smplest encodng of the stmulus. Ths vew dsallows smple perceptual organzatons that bear lttle or no relaton to the stmulus, because these organzatons do not help encode the stmulus smply. It also provdes an operatonal defnton of the explanatory power of a perceptual organzaton-as the degree to whch that organzaton helps provde a smple encodng of the stmulus. If a perceptual organzaton captures the regulartes n the stmulus (.e., f t "explans" those regulartes), then t wll provde the bass for a bref descrpton of the stmulus; f an organzaton fals to capture regulartes n the stmulus, then t wll be of no value n provdng a bref descrpton of the stmulus. Explanatory power s therefore not an addtonal constrant that must be traded off aganst smplcty; maxmzng explanatory power s the same as maxmzng the smplcty of the encodng of the stmulus. How Can Smplcty Be Measured? I have establshed what smplcty should apply to: namely, the encodng of the perceptual stmulus. But how s smplcty to be measured? The measurement of smplcty has been consdered extensvely n phlosophy (e.g., Sober, 1975) where no quanttatve measure has ganed acceptance. In psychology, theorsts concerned wth perceptual organzaton have taken the pragmatc step of dentfyng the smplcty of an encodng wth ts length. Attneave ( 1954, 1981 ), for example, explctly suggested that "what the [perceptual] system lkes s short descrptons" (Attneave, 1981, p. 417 ). Accordng to ths pont of vew, the preferred perceptual organzaton s that whch allows the brefest possble perceptual encodng. It s nterestng that the suggeston that the perceptual system has economcal encodng as an mportant goal has also been suggested n a varety of other contexts (e.g., Atck & Redlch, 1990; Barlow, Kaushal, & Mtchson, 1989; Blakemore, 1990). Moreover, as I dscuss below, ths s an approprate choce--an ndependent tradton wthn mathematcs and computer scence, Kolmogorov complexty theory, shows that the dentfcaton of smplcty wth brevty provdes a deep and mportant theory of smplcty (Chatn, 1966; Kolmogorov, 1965; L & Vtany, 1993; Solomonoff, 1964). Psychologsts have used two approaches to operatonalze the noton of brevty of encodng: Shannon's (1948) nformaton theory (Attneave, 1959; Garner, 1962 ) and the tradton known as codng theory (Hochberg & McAlster, 1953; Restle, 1979; Smon, 1972), one elaboraton of whch s structural nformaton theory (Buffart, Leeuwenberg & Restle, 1981 ). I consder these n turn. 4 Koffka ( 1935/1963 ) allowed the possblty of organzatons that are not consstent wth the perceptual stmulus, allowng dstorton to be traded wth smplcty. The emprcal evdence ndcates that such trade-offs, f they occur, may be qute small (Attneave, 1982 ).

4 Informaton theory and brevty. The nformaton-theoretc approach quantfes brevty n terms of the number of bts requred to dstngush the stmulus ( or some part of the stmulus) from a range of mutually exclusve and exhaustve alternatves, known as an nformaton source. Each alternatve, A, n an nformaton source, A, s assocated wth some probablty of occurrence, P(A~ ). The amount of nformaton, I(A ), assocated wth the choce of a partcular alternatve, A~, s called the surprsal of A and s defned I(A) = l g2 (p~a~)) Later, 1 shall consder the surprsal ofa~, condtonal on some other event, By. I denote ths by I(Al By), and the defnton parallels Equaton 5: I( A~ [ Bj) = l gz ( p( A~[ Bj) ) " The average surprsal of a source, A, s known as ts entropy, H(A), and s smply the surprsal of each alternatve, weghted by ts probablty of occurrence: H(A) = ~, P(Aj)I(Aj). J Surprsal can be vewed as a measure of brevty of encodng because of basc deas from nformaton theory, whch I now dscuss. Suppose that a sequence of alternatves s ndependently chosen accordng to the probabltes of the nformaton source, and that ths sequence of alternatves must be encoded n a bnary sequence. Let us stpulate that the encodng must be noseless (.e., the sequence of alternatves can be reconstructed wth perfect accuracy). Suppose, moreover, that the encodng proceeds by assocatng each alternatve, A~, wth a "code word"; that s, a sequence of bnary dgts (so, e.g., a partcular alternatve, Als, mght be assocated wth the code word ). A partcular sequence of alternatves s then encoded by concatenatng the correspondng code words nto a sngle bnary sequence. How should the code words be assgned to alternatves n order to mnmze the average length of the bnary strng requred to transmt the sequence of alternatves? The length of the strng encodng the sequence s the product of the length of the sequence and the average length of code words for elements n the sequence; hence we must assgn code words n order to mnmze the average code word length. Suppose that the code word for alternatve A s a bnary strng of length, l. Then the average code word length for the source A s specfed: X P(Aj)/j. J SIMPLICITY VERSUS LIKELIHOOD 569 a relatvely natural defnton, n terms of the probabltes of Let us call the mnmum value of, ths average L(A). Shannon's each stmulus p~:esented n the experment (e.g., Garner, 1962). (1948) noseless codng theorem s that ths mnmum s very But n many expermental contexts, and n natural percepton, close to the entropy of the source: the range of possbltes from whch the current stmulus s H(A) < L(A) <_ H(A) + 1. (9) drawn can be defned n nnumerably many dfferent ways, as s Crucally, ths mnmum s obtaned when the code length for an alternatve s ts surprsal (rounded up to the nearest nteger, because bnary sequences can only have nteger lengths). In symbols: (10) where the notaton rx] denotes x rounded up to the nearest nteger. Ths means that the surprsal of an alternatve can be vewed as a measure of ts code length n an optmal bnary code. Thus, surprsal can be vewed as a measure of brevty of ( 5 ) encodng. Despte the theoretcal elegance of nformaton theory n many contexts, t proves to have a number of dffcultes when appled to ndvdual perceptual stmul, as we now see. Suppose, to borrow an example from Leeuwenberg and Bosele (1988), that the stmulus conssts of a sequence of letters: aaabbbbbgg. The amount of nformaton n ths stmulus de- (6) pends on the nformaton source beng consdered. Suppose that t s assumed that each letter s chosen ndependently and that there s an equal (1/3) chance that the letter chosen wll be a, b, or g. Then the nformaton assocated wth the specfcaton of, say, the ntal a s log2( 1 / I/3) = 1og2(3) bts of nformaton. The nformaton assocated wth the specfcaton of each (7) of the 10 letters n ths sequence s, n ths case, the same. Hence the nformaton requred to specfy the entre sequence s 10 log2(3) bts. But a dfferent result s obtaned f we suppose that the letters have dfferent probabltes of beng chosen. Suppose that we assume that b s chosen wth probablty 1/2, and a and g wth probabltes I/4. Now, fve of the letters (the bs) can each be specfed wth log2( 1/J/2) = 1og2(2) = 1 bt; and fve (the as and gs) can each be specfed wth log2( 1/1/4) = log2(4) = 2 bts, makng a total of 15 bts of nformaton. Furthermore, a dfferent result agan s obtaned f t s assumed that the letters are chosen from the entre alphabet, or the entre computer keyboard, or all possble shapes of a certan sze, and so on. The larger the set from whch the stmulus s presumed to have been chosen, the more nformaton s requred to specfy t. Moreover, we mght assume that the ndvdual elements of the stmulus are not chosen ndependently (as the sequental runs n our sample sequence would tend to suggest). Perhaps the stmulus was generated by a more complex process, such as a Markov source, or a stochastc context-free grammar. Each dstnct specfcaton of the source from whch the stmulus was generated gves rse to a dfferent answer concernng the amount of nformaton n the stmulus. In short, the nformaton-theoretc approach does not measure the nformaton n a partcular stmulus per se, but rather measures the amount of nformaton n that stmulus relatve to the probabltes of all the other stmul that mght have been (8) generated. In expermental settngs, where a small number of stmul are presented many tmes, ths range of possbltes has clear even wth the smple letter sequence descrbed above. Hence, n most contexts, nformaton theory does not provde a useful measure of the brevty wth whch a partcular stmulus

5 570 CHATER can be encoded because t s not defned relatve to the stmulus alone (see Garner, 1962, for related dscusson).5 Even puttng ths dffculty asde, nformaton theory merely specfes the length of the code (the number of bts) requred to encode the stmulus, but t does not pck out any partcular code as the best code for the stmulus. Yet t s the nature, not just the length, of the code that s crucal from the pont of vew of understandng perceptual organzaton (Garner, 1974). Suppose, for example, that the sequence above was drawn from the eght equally lkely alternatves shown n Table 1. These sequences can be vewed as consstng of three "segments" of repeated letters--the frst segment beng three letters long, the second segment fve letters long, and the fnal segment two letters long. The eght alternatves can be vewed as generated by three bnary decsons, concernng whether the frst segment conssts of as or xs, whether the second segment conssts orbs or ys, and whether the thrd segment conssts of gs or zs. By nformaton theory, the number of bts requred to specfy a choce between eght equally probable alternatves s three bts. But optmal three-bt codes need not relate to the organzaton present n the perceptual stmul. Table 1 llustrates two optmal codes for the stmul. The frst s "meanngful" n the sense that each bt carres nformaton about the dentty of the letters n a partcular segment; t therefore reflects ths (mnmal) organzaton of the stmulus nto three segments. By contrast, the second code s "meanngless"; codes are arbtrarly assgned to strngs, and hence the organzaton of the stmulus s gnored. From an nformaton-theoretc pont of vew there s no dstncton between these codes; nformaton theory does not favor the code that reflects the underlyng organzaton of the stmulus over that whch does not. But from the pont of vew of perceptual organzaton, the dfference between codes that express the Table 1 "'Meanngful'and "'Meanngless" Codes for a Smple Sequence "Meanngful.... Meanngless" Stmulus code code aaabbbbbgg aaabbbbbzz aaayyyyygg 101 0(91 aaayyyyyzz xxxbbbbbgg xxxbbbbbzz xxxyyyyygg xxxyyyyyzz Note. Table 1 shows a set of eght stmul that are assumed to be equally probable. By nformaton theory, any gven stmulus can be encoded n just three bts. Two possble optmal codes are shown. The frst s "meanngful," n the sense that each element of the code can be nterpreted as specfyng part of the structure of the stmulus. Specfcally, the frst element of the code specfes whether the frst three letters are as or xs, the second element specfes whether the next fve letters are bs or ys, and the thrd element specfes whether the fnal two letters are gs or zs. The second code s "meanngless," n that the relaton between stmul and codes s chosen at random. Informaton theory does not dstngush between the code that "organzes" the stmul and the code that does not. Table 2 A "'Meanngless" Code for Heterogeneous Stmul Stmulus Code aaabbbbbgg * 1fhq (Hh8cQ l a l 101 wwwww 100 % 011 T 010 PERCEPTION Note. Table 2 renforces the pont made by Table 1, agan showng a set of eght stmul, assumed to be equally probable. Here the stmul are completely heterogeneous. The sequence a a a b b b b b g g has the same code as n Table 1, but t s not "meanngful?' Informaton theory does not recognze ths dstncton: In both Tables 1 and 2, the code used for the sequence s optmal. structure of the stmul and those that do not would appear to be crucal. Ths pont s renforced by the example shown n Table 2, of eght further stmul assumed to be equally lkely. Unlke the stmul n Table 1, these stmul cannot be organzed n any coherent way, but are completely heterogeneous. Nonetheless, the same code as before ( 111 ) s used to specfy aaabbbbbgg as n the meanngful code n Table 1. But whle the stmulus and the code are the same as before, now there s no meanngful nterpretaton of the ndvdual parts of the code as specfyng the structure of the sequence. From the pont of vew of perceptual organzaton ths s a crucal dfference, the dfference between stmul that can be organzed and those that cannot; but t s gnored by nformaton theory. Codng theory and brevty The dffcultes wth the applcaton of nformaton theory have led psychologsts to develop an alternatve approach to measurng the brevty wth whch a perceptual organzaton allows a stmulus to be encoded. Ths method s to defne what Smon ( 1972 ) calls pattern languages n whch dfferent organzatons of the stmulus can be expressed. The preferred organzaton s that whch allows the shortest descrpton of the stmulus, when measured n terms of the length of the expresson n the pattern language. Ths constrant s stated n terms of number of symbols n the descrpton (e.g., Smon, 1972) and sometmes the number of parameters (e.g., Leeuwenberg, 1969). These formulatons are equvalent accordng to the natural stpulaton that the values of each parameter are coded by a dstnct symbol. The codng theory approach may be llustrated wth the aaabbbbbgg sequence descrbed above. The codng correspondng to the null organzaton requres 10 parameters (I = 10), one for each symbol of the sequence. A spurous organzaton, dvdng the sequence (aa) (ab) (bb) (bb) (gg) can support the code 2(a)ab2(b)2(b)2(g); where 2(a) s a code n the pattern 5 Varous ngenous notons such as nferred subsets (Garner, 1974) and perceved subsets ( Pomerantz, 1981; Royer & Garner, 1966) have been used to specfy a relevant set of alternatves from whch the stmulus s chosen, although these apply only n very specfc contexts.

6 SIMPLICITY VERSUS LIKELIHOOD 571 language meanng a run of two as. But ths expresson contans 10 parameters, ether numbers or letters, and hence no economy s acheved (I = 10, as before). 6 A good organzaton, (aaa) (bbbbb) (gg), can support the code 3(a)5 (b)2(g), whch requres just sx parameters (I = 6 ). Ths ntutvely sensble organzaton thus corresponds to the organzaton that supports a short code. Ths general approach has been appled n a varety of contexts, from the organzaton of smple sequences, such as the example just consdered (Leeuwenberg, 1969; Restle, 1970; Smon, 1972; Smon & Kotovsky, 1963; Vtz & Todd, 1969), to judgments of "fgural goodness" (Hochberg & McAllster, 1953), the analyss of Johansson's (1950) experments on the percepton of moton confguratons (Restle, 1979), and fgural completon (Buffart, Leeuwenberg & Restle, 1981 ). It has also been advanced as a general framework for understandng perceptual organzaton (e.g., Attneave & Frost, 1969; Leeuwenberg, 1971; Leeuwenberg & Bosele, 1988). Approaches based on short descrpton length appear to be dogged by two problems: (a) that a fresh descrpton language must be constructed for each fresh knd of perceptual stmulus and (b) that the predctons of the theory depend on the descrpton language chosen and there s no (drect) emprcal means of decdng between putatve languages. In practce, as Smon (1972) noted, the second problem s not terrbly severe; descrpton lengths n dfferent proposed descrpton languages tend to be hghly correlated. The mathematcal theory of Kolmogorov complexty provdes a useful generalzaton of codng theory that addresses these ssues. Reconclng Lkelhood and Smplcty: I I now show that the lkelhood and smplcty prncples can be vewed as dfferent sdes of the same con. I present the dscusson n two stages. Ths secton descrbes how to effect the reconclaton, f smplcty s measured usng standard nformaton theory. Ths connecton between smplcty and lkelhood has prevously been dscussed n the context of computatonal models of vsual percepton by Mumford ( 1992; see also Grenander, ) and s well-known n the lterature on nformaton theory (e.g., Cover & Thomas, 1991 ) and statstcs (e.g., Rssanen, 1989). The next, much longer, secton, generalzes ths analyss, usng Ko!mogorov complexty nstead of standard nformaton theory, to provde a deeper reconclaton of smplcty and lkelhood. I now turn, then, to the analyss n terms of standard nformaton theory. Let us begn wth the lkelhood vew. I noted above that the lkelhood vew recommends the most probable hypothess, Ilk, about perceptual organzaton, gven the sensory data, D. In symbols (Equaton 4) ths s k = arg max [P(HID)]. Applyng Bayes's theorem (Equaton 1 ), ths mples that k = arg max [ P(DIHLP(H) ] L P(D) ] The k that maxmzes ths quantty wll also maxmze the log- arthm of that quantty (because log s monotoncally ncreasng). Ths mples that k=argm,ax{l g2[ P(D) JJ = arg max { log2[ P(D I H ) ] + logz[ P(H ) ] - log2[ P(D) ] }. Because P(D) s ndependent of the choce of/, the fnal term s rrelevant, such that: k = arg max ( log2[p(d I H ) ] + log2[p(h ) ] } = arg mn { -log2[e(dih )] - log2[p(h)] } = arg mn log2 P(D + log2. Ths gves the result: k = arg mn [I(DIH) + I(H;)], (11) where ths last step follows from the defntons of surprsal, I (Equatons 5 and 6 ). Havng consdered the lkelhood prncple, I now consder smplcty, usng nformaton theory. The length of code requred to encode data D (the stmulus) needs to be determned, usng a partcular hypothess, H, concernng perceptual organzaton. Ths code wll have two parts: frst, an encodng of the choce of hypothess, H, and second an encodng of the data D, gven H. The total code length, lto,,t~, wll be the sum of the length of the code for the hypothess, In,, and the length of the code for the data gven the hypothess, ID fn,. In symbols: Zto,,6 = ln, + loln~. (12) The smplcty strategy s to choose the k' that mnmzes the total code length,/to,ate, or, n symbols: k' = arg mn (lto~6) = arg mn (ln, + 1Dl~/). Usng Equaton 10 to convert code lengths nto surprsals (assumng that we use optmal codes, at each step), ths s k' = mn [[I(DIH)] + II(H;)q ]. (13) Asde from the roundng terms (whch can n any case be elmnated by a more sophstcated treatment), the goal of choosng a hypothess to mnmze descrpton length (13) s the same as the goal of choosng a hypothess to maxmze lkelhood ( 11 ). That s, the most lkely hypothess, H, about perceptual organzaton s the H that supports the shortest descrpton of the data, D, concernng the stmulus. Thus, the lkelhood prncple 6 Ths code also contans 10 symbols (gnorng brackets, whch are present only for clarty). In ths code, each parameter value corresponds to a sngle symbol (although ths would cease to be true for numercal values larger than 9, whch are expressed as compound symbols n base 10).

7 572 CHATER (choose the organzaton that s most lkely) and the smplcty prncple (choose the organzaton that allows the brefest encodng of the stmulus) are equvalent. It s mportant to stress that ths result shows not just that lkelhood and descrpton length are sometmes dentcal. Rather, t shows that for any problem of maxmzng lkelhood there s a correspondng "dual" problem of mnmzng descrpton lengths. Specfcally, gven any specfcaton of subjectve probabltes P(Hk) and P(D I Hk), there wll be a code that s optmal wth respect to those probabltes. The hypothess that mnmzes the length of code requred for the data, D, wll be the same as the hypothess that maxmzes lkelhood. Smlarly, any code can be vewed as an optmal code wth respect to a set of subjectve probabltes. Therefore, choosng the hypothess that mnmzes the code length of the data wll be equvalent to maxmzng lkelhood wth respect to those probabltes. Ths equvalence has not been noted n the psychology of perceptual organzaton, but n the study of computatonal models of percepton, t has been wdely exploted. For example, problems of maxmzng lkelhood can sometmes be made more tractable when pror knowledge of relevant probabltes s not avalable, by swtchng to a formulaton of mnmzng code length (Mumford, 1992). Conversely, problems of mnmzng code length can sometmes be solved by swtchng to a probablstc formulaton, where relevant pror knowledge s avalable (e.g., Atck & Redlch, 1992). It follows, therefore, that any evdence that the perceptual system chooses perceptual organzatons that follow from the lkelhood prncple (where certan assumptons are made about subjectve probabltes) can be vewed equally well as evdence for the smplcty prncple (where certan and equvalent assumptons are made about the codng language). Reconclng Smplcty and Lkelhood: II The analyss above s suggestve, but unsatsfactory n two ways. Frst, regardng smplcty, I have used the nformatontheoretc measure of smplcty, whch I have already noted s not an approprate measure of the brevty wth whch an ndvdual stmulus can be encoded. Second, regardng lkelhood, I have sdestepped a fundamental dffculty: that there are nfntely many possble dstal states (.e., nfntely many H) consstent wth any gven perceptual stmulus, D, and t s by no means obvous that these can consstently be assgned pror probabltes, P(H ). I now show how both of these problems have been addressed by the mathematcal theory of Kolmogorov complexty. Specfcally, I show how a measure of the nformaton n an ndvdual stmulus can be defned, whch can be vewed as a generalzaton and unfcaton of the nformaton theory and codng theory approaches developed n psychology; and how a coherent defnton of pror probablty for a dstal state can be specfed. I then show that the equvalence between the lkelhood and smplcty crtera holds, usng these more satsfactory notons of lkelhood and smplcty. The lterature on Kolmogorov complexty s not well-known n psychology, and I therefore sketch relevant aspects of the theory below, referrng the reader to L and Vtany's ( 1993) excellent textbook for more detals. Kolmogorov Complexty as a Measure of Smplcty Codng theory suggests a means of decdng between rval perceptual organzatons of the stmulus. Ths nvolves defnng a pattern language, expressng the rval perceptual organzatons n that pattern language, and favorng the organzaton that allows the brefest descrpton of the stmulus. By extenson, codng theory suggests a way of measurng the complexty of stmul themselves: the length of the shortest descrpton n the pattern language that encodes the stmulus. Indeed, ths noton has been used to account for subjects' judgments of the complexty of stmul by a number of researchers (Leeuwenberg, 1969; Smon, 1972; Smon & Kotovsky, 1963; Vtz & Todd, 1969). The theory of Kolmogorov complexty (Chatn, 1966; Kolmogorov, 1965; Soomonoff, 1964), developed ndependently, can be vewed as a more general verson of ths approach to measurng complexty (and t can also be vewed as a generalzaton of standard nformaton theory). I noted above two apparently unattractve features of codng theory: (a) that dfferent pattern languages must be developed for dfferent knds of stmul and (b) that the measure of smplcty depends on the pattern language used. Kolmogorov complexty avods the frst problem by choosng a much more general language for encodng. Specfcally, the language chosen s a unversal programmng language. A unversal programmng language s a general purpose language for programmng a computer. The famlar programmng languages such as PROLOG, LISP, and PASCAL are all unversal programmng languages. How can an object, such as a perceptual stmulus, be encoded n a unversal programmng language such as, for example, LISP? The dea s that a program n LISP encodes an object f the object s generated as the output or fnal result of runnng the program. Whereas codng theory requres the development of specal purpose languages for codng partcular knds of perceptual stmulus, Kolmogorov complexty theory can descrbe all perceptual stmul usng a sngle unversal programmng language. Ths follows because, by the defnton of a unversal programmng language, f an object has a descrpton from whch t can be reconstructed n any language, then t wll have a descrpton from whch t can be reconstructed n the unversal programmng language. It s ths that makes the programmng language unversal. The exstence of unversal programmng languages s a remarkable and central result of computablty theory (Odfredd, 1989; Rogers, 1967). Perhaps even more remarkable s that there are so many unversal programmng languages, ncludng all the famlar computer languages. 7 One of the smplest unversal languages, whch we wll consder below, s that used to encode programs on a standard unversal Turng machne (Mnsky, 1967). A Turng machne s a smple computatonal devce, wth two components. The frst component s a lnear "tape" consstng of squares that may contan the symbols 0 or 1 or may be left blank. The tape can be extended ndefntely n both drectons, and hence there can be nfntely many dfferent patterns of 0s and ls on the tape. The second component s a "control box," consstng of a fnte 7 Specfcally, any language rch enough to express the partal recursve functons wll be unversal (Rogers, 1967 ).

8 SIMPLICITY VERSUS LIKELIHOOD 573 number of states, whch operates upon the tape. At any tme, the control box s located over a partcular square of the Turng machne's tape. The control box has a small number of possble actons: It can move left or rght along the tape, one square at a tme; t can read the symbol on the tape over whch t s currently located; t can replace the current symbol wth a dfferent symbol; and the current state oftbe control box may be replaced by one of the fnte number of other possble states. Whch actons the control box performs s determned by two factors: the current state of the machne and the symbol on the square of the tape over whch t s located. A Turng machne can be vewed as a computer n the followng way. The nput to the computaton s encoded as the strng of ls and 0s that comprse the ntal state of the tape. The nature of the control box (that s, whch symbols and states lead to whch actons and changes of state) determnes how ths nput s modfed by the operaton of the control box. The control box mght, for example, leave the nput ntact, delete t entrely and replace t wth a completely dfferent strng of I s and 0s, or more nterestngly perform some useful manpulaton of the nput. Ths resultng strng encodes the output of the computaton, s Each control box can therefore be assocated wth a mappng from nputs to outputs, defnng the computaton that t performs. Accordng to the Church-Turng thess (Boolos & Jeffrey, 1980), every mappng that can be computed by any physcal devce whatever can be computed by some Turng machne. A unversal Turng machne s a Turng machne that s programmable. There are many dfferent possble programmng languages correspondng to dfferent unversal Turng machnes (n the same way that there are many dfferent programmng languages for conventonal computers). In the followng, one unversal Turng machne (t does not matter whch) has been chosen, whch s called U. The nput to U can be thought of as dvded nto two parts (these wll typcally be separated, for example, by blank spaces between them on the tape). The frst part s the program, whch encodes the seres of nstructons to be followed. The second part s the data, upon whch the nstructons n the program are to operate. U's control box s desgned so that t reads and carres out the nstructons n the program as appled to the data provded. In ths way, U can perform not just a sngle computaton from nput to output, but many dfferent mappngs, dependng on what s specfed by ts program. In fact, t s possble to wrte a program that wll make Ucompute the same mappng as any specfed Turng machne, and thus every mappng that can be computed at all (assumng the Church-Turng thess). So U s unversal n that t can perform all possble computatons, f t s gven the approprate program. Notce, fnally, that the program of the unversal Turng machne s just a strng of Is and 0s on the nput tape--ths s the same form as the data on whch the program operates. We shall see that ths s useful when consderng the noton of unversal a pror probablty below. For any object, 9 ncludng perceptual stmul, the defnton of the complexty of that object s the length of the shortest code (.e., the shortest program) that generates that object, n the unversal programmng language of choce. By usng a unversal language, the need to nvent specal purpose languages for each knd of perceptual stmulus s avoded, thus solvng the frst problem noted wth codng theory. Moreover, n solvng the frst problem, the second problem, that dfferent patterns languages gve dfferent code lengths, s solved automatcally. A central result of Kolmogorov complexty theory, the nvarance theorem (L & Vtany, 1993 ), states that the shortest descrpton of any object s nvarant (up to a constant) between dfferent unversal languages. Therefore, t does not matter whether the unversal language chosen s PRO- LOG, LISP or PASCAL, or bnary strngs on the tape of a unversal Turng machne; the length of the shortest descrpton for each object wll be approxmately the same. Let us ntroduce the notaton KLse(X) to denote the length of the shortest LISP program that generates object x; and KpASCAL(X) to denote the length of the shortest PASCAL program. The nvarance theorem mples that KLlse(x) and KeASCAL(X) wll only dffer by some constant, c (whch may be postve or negatve), for all objects, ncludng, of course, all possble perceptual stmul. In symbols: 3CVx(KL~sI,(X) = KeaSCAL(X) + C). (14) (where 3 denotes exstental quantfcaton and V denotes unversal quantfcaton). In specfyng the complexty of an object, t s therefore possble to abstract away from the partcular language under consderaton. Thus the complexty of an object, x, can be denoted smply as K(x); ths s known as the Kolmogorov complexty of that object. Why s complexty language nvarant? To see ths ntutvely, note that any unversal language can be used to encode any other unversal programmng language. Ths follows from the precedng dscusson because a programmng language s just a partcular knd of computable mappng, and unversal programmng language can encode any computable mappng. For example, startng wth LISP, a program can be wrtten, known n computer scence as a compler, that translates any program wrtten n PASCAL nto LISP. Suppose that ths program has length ct. Now suppose that KpASCAL(X), the length of the shortcst program that generates an object x n PASCAL, s known. What s KL~sp(X), the shortest program n LISP that encodes x?. Notce that one way of encodng x n LISP works as follows: The frst part of the program translates from PASCAL nto LISP (of length ct), and the second part of the program, whch s an nput to the frst, s smply the shortest PASCAL program generatng the object. The length of ths program s the sum of the lengths of ts two components: KpASCAL(X) + Ct. Ths s a LISP program s It s also possble that the control box wll contnue modfyng the contents of the tape forever, so that there s no well-defned output. We shall gnore such nonhaltng "lurng machnes later for smplcty. See Footnote There s an mplct restrcton, ofcourse, to abstract, mathematcal objects, both n ths general context and n the context of percepton. The perceptual stmulus s consdered n terms of some level of descrpton (e.g., n terms of pxel values, or actvty of receptors); t s the abstract descrpton that s encoded n the pattern language. It would be ncoherent to speak of the perceptual stmulus tself beng encoded. Encodng concerns nformaton, and nformaton s by defnton an abstract quantty, dependent on the level of descrpton (see Chater, 1989; Dretske,! 981, for dscusson).

9 574 CHATER that generates x, f by a rather roundabout means. Therefore KLzsp(X), the shortest possble LISP program must be no longer than ths: Kuse(X) < KeASCAL(X) + Ct. An exactly smlar argument based on translatng n the opposte drecton establshes that KeASCAL ( X) < KL~se( X) + C2. Puttng these results together, KenscnL(x) and Kzlse(X) are the same up to a constant, for all possble objects x. Ths s the Invarance Theorem (see L & Vtany, 1993, for a rgorous proof along these lnes) and establshes that Kolmogorov complexty s language nvarant. Both lmtatons of codng theory--the need to develop specal purpose languages for partcular knds of pattern and the dependence of code length on the pattern language used--are overcome. Only a sngle language need be used, a unversal programmng language, and moreover t does not matter whch unversal programmng language s chosen, because code lengths are nvarant across unversal languages. In addton to provdng a measure of the complexty of a sngle object, x, Kolmogorov complexty can be generalzed to measure the complexty of transformng one object, y, nto another object, x. Ths quantty s the length of the shortest program that takes y as nput and produces x as output and s called condtonal Kolmogorov complexty whch s wrtten K(xl y). K(xl y) wll sometmes be much less than K(x). Suppose, for example, that x and y are both random strngs ofn bnary numbers (where n s very large). Because random strngs have no structure, they cannot be compressed by any program, and hence K(x) = K(y) = n (L & Vtany, 1993). But suppose that x and y are closely related (e.g., that x s smply the same strng as y but n reverse order). Then the shortest program transformng y nto x wll be the few nstructons needed to reverse a strng, of length c3, where c3 s very much smaller than n. Thus, n ths case, K(xl y) wll be much smaller than K(x). On the other hand, K(xl y) can never be substantally larger than K(x). Ths s because one way of transformng y nto x s to frst run a (very short) program that completely deletes the nput y and then reconstruct x from scratch. The length of the former program s very small (say, c4), and the length of the latter s K(x). The shortest program transformng y nto x cannot be longer than ths program, whch makes the transformaton successfully. Therefore K(x[ y) < K(x) + c4, whch establshes that K(xl y) can never be substantally larger than K(x) (see L & Vtany, 1993, for dscusson). Condtonal Kolmogorov complexty s mportant below as a measure of the complexty of perceptual, D, gven a hypotheszed perceptual organzaton, Hr. Kolmogorov complexty also has very close relatons to Shannon's noton of nformaton. For an nformaton source, A, t can be shown (L & Vtany, 1993, p. 194) that the entropy H(A ) of the source s approxmately equal to the expected Kol- mogorov complexty of the alternatves At, whch comprse the 10 source: H(A) = Z P(Aj)I(Aj) ~ Z P(Aj)K(A:) J J = expected Kolmogorov complexty. (15) Intutvely, ths s plausble, because entropy s the expected value of surprsal I(A ), and that surprsal (rounded up to the nearest nteger) was noted earler as the mnmal code length for the alternatve A, n an nformatonally optmal code. Kolmogorov complexty s smply a dfferent measure of code length for an alternatve; but on average t has the same value as the orgnal measure. There are many other parallels between the standard noton of nformaton theory and Kolmogorov complexty, whch has gven rse to algorthmc nformaton theory, a reformulaton of nformaton theory based on Kolmogorov complexty (see, e.g., Kolmogorov, 1965; Zvonkn & Levn, 1970). Notce that Kolmogorov complexty overcomes the crucal dffculty wth the classcal noton of nformaton n the context of the study of perceptual organzaton, because t apples to a sngle object n solaton, not n relaton to the set of alternatves. It s therefore possble to vew Kolmogorov complexty as a measure of smplcty, whch s both a generalzaton of nformaton theory and a generalzaton of codng theory. It thus provdes an attractve unfcaton of the two prncpal approaches used by psychologsts to quantfy smplcty; and t overcomes the standard dffcultes wth both notons from the pont of vew of measurng complexty of perceptual stmul. More mportant than unfyng the two approaches to smplcty, however, s that t allows the reconclaton of the apparently dstnct lkelhood and smplcty prncples of perceptual organzaton. Before ths reconclaton can be demonstrated, t s necessary to provde a more detaled mathematcal treatment of the lkelhood vew, to whch I now turn. Lkelhood and Pror Probabltes I showed above how Bayes's theorem (Equaton 1 ) can be used to calculate the probablty of a partcular hypothess concernng the dstal layout, gven data concernng the perceptual stmulus. An mmedate possble concern regardng the applcaton of Equaton 1 s that the number of possble sensory stmul s very large ndeed (ndeed t s nfnte, asde from the lmts of resoluton of the sensory systems), and the probablty P(D) of any specfc stmulus, D, wll be very close to 0. Ths possble dvson by 0 s not actually problematc, however, because the numerator wll always be even smaller than the denomnator; otherwse P(H~ID) would be greater than 1, volatng the laws of probablty theory. That s, P(H~ [ D) s the rato of two very small quanttes, but the rato s well-defned. Indeed, n typcal applcatons of Bayesan statstcs, P(D) s typcally very close to 0, but no problems arse (e.g., Lndley, 1971 ). A more dffcult problem arses, however, wth respect to the pror probabltes of hypotheses, P(H~ ). Applyng Bayes's theorem requres beng able to specfy the pror probabltes of the possble hypotheses. But, as I noted earler, there are nfntely many dstal layouts (or perceptual organzatons) that are consstent wth a gven stmulus (at least for the degraded stmul studed n experments on perceptual organzaton j~ ). For ex- 10 Ths result requres the weak condton that the functon from the ndex,, of a state to ts probablty P(A ) s recursve. See L and Vtany ( 1993, p. 194) for detals. lj Gbson ( 1950, 1966, 1979) has, of course, argued that n the rch, ecologcally vald condtons of normal percepton, ths underdetermnaton of the dstal layout by the perceptual nput does not apply. Although I would argue aganst ths pont of vew, 1 smply note here that

10 SIMPLICITY VERSUS LIKELIHOOD 575 ample, a two-dmensonal lne drawng may be the projecton of an nfnte number of dfferent three-dmensonal shapes; a pattern of dots may be joned up by nfntely many dfferent curves; and so on. Prors must be assgned so that each of ths nfnte number of alternatves s assgned a nonzero pror (so that t s not ruled out a pror) and so that the sum of the probabltes s 1 (otherwse the axoms of probablty theory are volated). These constrants rule out the possblty of assgnng each hypothess an equal probablty, because the sum of an nfnte number of fnte quanttes, however small, wll be nfnte and hence not equal to 1 (the problems assocated wth such "mproper" prors have been extensvely dscussed n the phlosophy of scence and the foundatons of statstcs; e.g., Carnap, 1952; Jeffrey, 1983; and Keynes, 1921 ). Therefore, an uneven dstrbuton of pror probabltes s requred. In a perceptual context, an uneven dstrbuton of prors s qute reasonable. I have already noted that, accordng to the lkelhood nterpretaton, the perceptual system can be vewed as favorng (.e., assgned a hgher pror probablty to) certan hypotheses (such as three-dmensonal shapes contanng rght angles) over other hypotheses (such as hghly skewed trapezods). The lkelhood vew typcally (though not necessarly) takes the emprcst vew that certan hypotheses are favored because of past perceptual experence (e.g., that humans lve n a "carpentered world"). Hence the queston arses: How should prors be set for the newborn, before any perceptual experence? The obvous approach s to suggest that each hypothess s gven equal probablty. However, ths s not possble, because there are an nfnte number of hypotheses. The problem of assgnng prors to an nfnte number of hypotheses has been most ntensvely studed n the context of scentfc nference (Horwch, 1982; Howson & Urbach, 1989; Jeffreys & Wrnch, 1921; Keynes, 1921 ). Indeed, the nablty to fnd a satsfactory soluton to ths problem proved to be a serous dffculty for Carnap's (1950, 1952) program of attemptng to devse an nductve logc (see Earman, 1992; for dscusson). In the context of attemptng to solve the problems of nductve nference rased by Carnap, Solomonoff (1964) showed how prors could be assgned consstently and neutrally to an nfnte range of hypotheses and n dong so provded the frst formulaton of the prncples of Kolmogorov complexty. Solomonoff suggested that hypotheses could be neutrally assgned probabltes as follows. Frst, a programmng language s selected. For smplcty, the very smple language of the unversal Turng machne, U, s chosen. Recall that a program for U conssts of arbtrary strngs of 0s and ls on a partcular porton of U's tape. 1 now consder a two-stage process for generatng objects, x, whose a pror probablty we wsh to assgn. The frst stage nvolves generatng programs, p, for the unversal Turng machne at random. Ths smply nvolves generatng random strngs of 0s and 1 s, for example, by tossng a con. The second stage s to run each program, p, untl t halts, havng generated some object (some programs wll not halt, but we can gnore these). Solomonoff defnes the unversal a pror proba- research on perceptual organzaton typcally concerns ecologcally nvald stmul, such as lne drawngs. blty, Qv(x), of an object x as the probablty that the object produced by ths process s x. Intutvely, the unversal a pror probablty of an object depends on how many programs there are that generate t (.e., how many descrptons t has). If any of these programs s generated n the frst stage, then the xwll be produced at the second stage. In addton, t s mportant how long the programs (descrptons) of the object are: The shorter the program, the more lkely t sto be generated at random at the frst stage. Specfcally, consder a partcular program, p', for U that generates the object x; that s, U(p') = x. The length of program (.e., the number of 0s and Is t contans) s denoted by l(p'). Then the probablty of the program p' beng generated at random at the frst stage of the process above s the probablty of l(p') consecutve con tosses comng up n a partcular way: (l/2)tcp') = 2-ttf). Ifp' s generated at the frst stage, then at the second stage U runsp' and generates the object x. The above calculaton gves the probablty ofx beng generated by ths partcular program. The unversal pror probablty Qv(x) s the probablty ofx beng generated by any program. To calculate ths, we must sum over all programs, p, whch generate x (n symbols, p: U(p) = x) Thus, unversal pror probablty Qv(x) s Qv(x) = ~ 2 -I~p). (16) p:u(p)ff Although the language of a partcular unversal Turng machne, U, has been consdered, unversal a pror probablty, lke most quanttes n Kolmogorov complexty theory, s language ndependent. ~2 The ntuton behnd Solomonoff's (1964) approach to settng prors concernng sets of alternatve objects s that neutralty should consst of evenhandedness between processes (programs) that generate alternatve objects, not evenhandedness between objects themselves. Thus, objects that are easy to generate should be those that are expected, a pror. Furthermore, evenhandedness among processes means generatng programs at random; ths favors smpler processes (.e., those wth shorter programs), snce they are more lkely to arse by chance. It s here that the frst sgn of a relatonshp between a pror probablty and smplcty s seen. Recall that the applcaton of Bayes's theorem requres the specfcaton not just of pror probabltes, P(H ), but also condtonal probabltes P(DIHj). These can be specfed n a drectly analogous way to the pror probabltes, defned as follows: x Qv(xl y) = ~ 2 -ttp). (17) p:u(p,y) ~2 There are a number of techncal detals that I gnore for smplcty. For example, t s mportant that the programs for the unversal Turng machne are what s known as prefx codes. That s, no complete program s also the ntal porton (prefx) of any other program. I also gnore the queston of how, or f, to take account of those Turng machnes that do not halt. On the current defnton, the pror probabltes sum to less than one because of the nonhatng machnes. These and other techncal ssues have been tackled n dfferent ways by dfferent researchers (e.g., Solomonoff, 1978; Zvonkn & Levn, 1970), but do not affect the present dscusson. = x

11 576 CHATER Qu( xl y) s known as the condtonal unversal dstrbuton. It represents the probablty that a randomly generated program for U wll generate object x (accordng to the two stages gven earler), gven y as an nput. Intutvely, fx s probable gven y (e.g., y s a hypothess that correctly descrbes some aspect of data x), then t should be easy to reconstruct x gven y. Asde from ntutve appeal, unversal a pror probablty (and ts generalzaton to condtonal probabltes) has a large number of attractve mathematcal characterstcs that have led to t, or close varants, beng wdely adopted n mathematcs (L & Vtany, 1993). Moreover, entrely ndependent mathematcal arguments, drawng on nonstandard measure theory, converge on the same noton of unversal a pror probablty (Zvonkn & Levn, 1970). For these reasons, unversal a pror probablty has become a standard approach to assgnng probabltes to nfnte numbers of alternatves, n the absence of pror experence. It therefore seems reasonable to apply ths noton to assgnng probabltes to alternatve hypotheses concernng the dstal layout. Reconclng Smplcty and Lkelhood: H The frst reconclaton of smplcty and lkelhood reled on Shannon's (1948) noseless codng theorem, whch relates the probablty of an alternatve to ts code length. It was noted that (by Equaton 10), where lj s the length of the code for alternatve A j, gven an optmal code. The second, and deeper, reconclaton between smplcty and lkelhood also requres relatng probablty and code length, va what L and Vtany ( 1993 ) call the codng theorem (due to Levn, 1974 ), a drect analog of Shannon's result. Ths states that (up to a constant) K(x) = log2 [Q~(x)]. (18) The length K(x) of the shortest program generatng an object, x, s related to ts unversal pror probablty by the codng theorem n the same way as optmal code length lj s related to the probablty of the alternatve A j, whch t encodes. There s another analogous result that apples to condtonal probabltes, known as the condtonal codng theorem, whch states that (up to a constant) K(xly)=l g2[~l~dv,'l'xl Y)']" (19) The argument for the reconclaton of lkelhood and smplcty runs as before. As above, the lkelhood prncple recommends that we choose k so that k = arg max (P(H; I D)) Followng our prevous analyss ths mples k=argmn{l g2[p(lhh;)]+l g2[p~h~)]} " We can now apply the codng theorem and the condtonal codng theorem, to derve k = arg mn [K(DIH;) + K(H;)]. (20) Thus, choosng hypothess Hk n order to maxmze lkelhood s equvalent to choosng the H~, whch mnmzes the descrpton length of the data, D, when that data s encoded usng Hk. That s, maxmzng lkelhood s equvalent to maxmzng smplcty. The smplcty and lkelhood prncples are equvalent.~3 As I noted n dscussng the nformaton-theoretc reconclaton between lkelhood and smplcty above, every problem of maxmzng lkelhood has a dual problem of mnmzng code length. Ths also holds n terms of Kolmogorov complexty. The possblty of establshng an equvalence between smplcty and lkelhood s not merely a matter of mathematcal curosty. It has been part of the motvaton for the development of approaches to statstcal nference couched n terms of smplcty, rather than probabltes, known varously as mnmum message length (Wallace & Boulton, 1968; Wallace & Freeman, 1987 ) and mnmum descrpton length (e.g., Rssanen, 1978, 1987, 1989). These deas have also been appled n the lteratures on machne learnng (e.g., Qunlan & Rvest, 1989), neural networks (e.g., Zemel, 1993), and even to problems qute closely connected to perceptual organzaton: automatc handwrtten character recognton (Goa & L, 1989) and computer vson approaches to surface reconstructon (Pednault, 1989). It s nterestng that such a range of mportant applcatons have resulted from the reconclaton of the two psychologcally motvated prncples of smplcty and lkelhood. It s possble to speculate that the reconclaton may have potentally mportant consequences for perceptual theory and for the study of cognton more generally. 1 consder some possble mplcatons n the dscusson. Dscusson I now consder the wder mplcatons of ths analyss. Frst, I reconsder the emprcal evdence that has been vewed as favorng ether the lkelhood, or the smplcty, prncple and show how ths evdence does not dstngush between the two vews. Second, I outlne possble resdual debates concernng whch prncple should be vewed as fundamental. Thrd, I note that t can be mathematcally proved that the cogntve system cannot follow ether prncple, as tradtonally formulated, and suggest a mnor modfcaton of the prncples, to retan psychologcal plausblty. Fourth, I consder possble applcatons of smplcty-lkelhood prncples n relaton to other aspects of cognton. Fnally, I dscuss what the smplcty-lkelhood vew of perceptual organzaton leaves out. ~3 L and Vtany ( 1995 ) provde a rgorous and detaled dscusson of the mathematcal condtons under whch ths relaton holds, n the context of the relatonshp between mnmum descrpton length ( Rssanen, 1987 ) and Bayesan approaches to statstcal nference.

12 SIMPLICITY VERSUS LIKELIHOOD 577 Reconsderng the Emprcal Evdence If the smplcty and lkelhood prncples are dentcal, then evdence from experments on perceptual organzaton cannot favor one rather than the other. Indeed, ~ as noted above, a wde range of phenomena n perceptual organzaton have been nterpreted equally easly n terms of both prncples. Nonetheless, varous lnes of emprcal evdence have been vewed as favorng one vew or the other. 1 argued that ths evdence does not dstngush between the smplcty and lkelhood prncples of perceptual organzaton. Putatve evdence for the lkelhood prncple comes from preference for organzatons that "make sense" gven the structure of the natural world, but are not n any ntutvely obvous sense smpler than less "natural" organzatons, such as the tendency to nterpret objects as f they are llumnated from above. The mathematcal analyss above suggests that there must, however, be an explanaton n terms of smplcty. The smplctybased explanaton can be ntutvely understood as follows. Consder the smplest descrpton not of a sngle stmulus, but of a typcal sample of natural scenes. Any regularty that s consstent across those scenes need not be encoded afresh for each scene; rather, t can be treated as a "default." That s, unless there s an specfc addtonal part of the code for a stmulus that ndcates that the scene volates the regularty (and n what way), t can be assumed that the regularty apples. Therefore, other thngs beng equal, scenes that respect the regularty can be encoded more brefly than those that do not. Moreover, perceptual organzatons of ambguous scenes that respect the regularty wll be encoded more brefly than those that volate t. In partcular, then, the perceptual organzaton of an ambguous stmulus obeyng the natural regularty of llumnaton from above wll be brefer than the alternatve organzaton wth llumnaton from below. In general, preferences for lkely nterpretatons also gve rse to preferences for smple nterpretatons: If the code for perceptual stmul and organzatons s to be optmal when consdered over all (or a typcal sample of) natural scenes, t wll reflect regulartes across those scenes. Putatve evdence for smplcty nvolves cases of perceptual organzatons that appear to be very unlkely. Recall Leeuwenberg and Bosele's (1988) schematc drawng of what s seen as a symmetrcal, two-headed horse. People do not perceve what seems to be a more lkely nterpretaton, that one horse s occludng another. Ths appears to be at varance wth the lkelhood prncple, and Leeuwenberg and Bosele hnted that ths s evdence n favor of smplcty. But a lkelhood explanaton of ths phenomenon, where lkelhood apples locally rather than globally, can also be provded. That s, the perceptual system may determne the nterpretaton of partcular parts of the stmulus accordng to lkelhood (e,g., the fact that there are no local depth or boundary cues may locally suggest a contnuous object). These local processes wll not always be guaranteed to arrve at the globally most lkely nterpretaton (see Hochberg, 1982). Resdual Debates Between Lkelhood and Smplcty From an abstract pont of vew, smplcty and lkelhood prncples are equvalent. But from the pont of vew of perceptual theory, one may be more attractve than the other. Leeuwenberg and Bosele (1988) gve an mportant argument aganst the lkelhood prncple and n favor of the smplcty prncple: that the lkelhood prncple presupposes that patterns are nterpreted, rather than explanng the nterpretaton of those patterns. Ths s because the lkelhood prncple holds that the structure n the world explans the structure n perceptual organzaton; but the theorst has no ndependent way of accessng structure n the world, asde from relyng on the results of the prncples of perceptual organzaton. Hence, lkelhood cannot be taken as basc n explanng perceptual organzaton. Ths pont of vew has parallels n the lterature on nductve nference usng Kolmogorov complexty. For example, Rssanen (1989) argues that, although mnmzng descrpton length and maxmzng lkelhood are formally equvalent, the former s a preferable vewpont as a foundaton for nductve nference, because the lkelhood approach presupposes that the world has a certan (probablstc) structure, and the only way to access ths structure s by nductve nference. Therefore, lkelhood cannot be taken as basc n nductve nference. Ths lne of argument provdes a motvaton for preferrng smplcty over lkelhood; but t wll not, of course, be persuasve to theorsts wth a strong realst pont of vew, accordng to whch the structure of the world s ndependent of human cognton, and can be objectvely studed wthout embodyng presuppostons about the structure of the human cogntve system. A very dfferent reason to prefer one or other prncple may be derved by consderng the nature of mental representatons and algorthms used by the cogntve system. Ths ssue s usefully framed n terms of Marr's (1982) dstncton between "computatonal level" and "algorthmc level" explanaton of an nformaton-processng devce. At the computatonal level, the questons "What s the goal of the computaton, why s t approprate, and what s the logc of the strategy by whch t s carred out?" are asked (Marr, 1982, p. 25). It s clear that the prncples of smplcty and lkelhood are typcally understood at ths level of explanaton; they defne goals for the perceptual system: maxmzng smplcty or lkelhood. I have shown that these goals are equvalent, and therefore that the smplcty and lkelhood are equvalent prncples at Marr's computatonal level. By contrast, at the algorthmc level the questons "How can ths computaton be mplemented? what s the representaton for the nput and output, and what s the transformaton?" are asked (Marr, 1982, p. 25). If the smplcty and lkelhood prncples are vewed as rval accounts of the algorthmc level, then there may be a genune debate between them. An algorthmc-level elaboraton of the smplcty vew based on descrpton length s that algorthmc calculatons are defned over descrpton lengths, rather than probabltes. Specfcally, perceptual organzatons are chosen to mnmze descrpton length n some language of nternal representaton. Ths noton of descrpton length could then be nterpreted not only as an abstract measure of smplcty but also as a concrete measure of the amount of memory storage space used by the cogntve system n storng the stmulus, usng the nternal language. Evdence for the exstence of such an nternal language, along wth evdence that short codes n that language correspond to preferred organzatons, mght be taken as evdence n favor of the prorty of the smplcty vew at the algorthmc level. Accord-

13 578 CHATER ng to ths vew, perceptual organzaton would not nvolve probablstc calculatons, but the mnmzaton of the amount of storage space requred n memory. Smlarly, evdence for the nternal representaton of nformaton concernng probabltes and evdence that ths nformaton was used n algorthms for probablstc calculatons could be used n favor of the lkelhood vew. Accordng to ths vew, the cogntve system would not be mnmzng the amount of storage space requred n memory but would be conductng explct probablstc reasonng. How could smplcty and lkelhood accounts of the algorthmc level be dstngushed emprcally? Evdence dstngushng between these competng algorthmc-level accounts s lkely to be very dffcult to collect, partcularly because the most obvous lne of attack, expermental study of the structure of perceptual organzatons for varous knds of stmulus, s napplcable because of the equvalence of the two prncples at the computatonal level. Nonetheless, the two prncples can (although they need not) be nterpreted as makng dstnct emprcal clams concernng the algorthms underlyng perceptual organzaton. Overall, even the provable equvalence of the smplcty and lkelhood prncples may not preclude arguments over whch s fundamental, based ether on phlosophcal or algorthmc-level concerns. A Psychologcally Plausble Modfcaton of the Smplcty-Lkelhood Prncple In ths artcle, I have been concerned wth showng that the smplcty and lkelhood prncples are the same. I now note that t s provably mpossble that the cogntve system follows ether prncple n ts strongest form. For concreteness, I dscuss the smplcty prncple, although the same consderatons apply to the lkelhood prncple. Is t possble that the perceptual system nvarably chooses the smplest (most probable) organzaton of the stmulus? Assumng that the perceptual system s computatonal, there are strong reasons to suppose that t cannot. The general problem of fndng the shortest descrpton of an object s provably uncomputable (L & Vtany, 1993). It s mportant to note that ths result apples to any knd of computer, whether seral or parallel, and to any style of computaton, whether symbolc, connectonst or analog.14 It s also ntutvely obvous that people cannot fnd arbtrary structure n perceptual scenes. To pck an extreme example, a grd n whch pxels encoded the bnary expanson of ~r would, of course, have a very smple descrpton, but ths structure would not be dentfed by the perceptual system; the grd would, nstead, appear completely unstructured. It s clear, then, that the perceptual system cannot, n general, maxmze smplcty (or lkelhood) over all perceptual organzatons, pace tradtonal formulatons of the smplcty and lkelhood prncples. It s, nonetheless, entrely possble that the perceptual system chooses the smplest (or most probable) organzaton that t s able to construct. That s, smplcty may be the crteron for decdng between rval organzatons. To retan psychologcal plausblty, the smplcty (lkelhood) prncples must be modfed, to say that the perceptual system chooses the smplest (most lkely) hypothess t can fnd; ths wll not, n general, be the smplest possble hypothess. Notce, however, that ths revsed formulaton--that the perceptual system chooses the smplest (most probable) organzaton that t can construct--does not affect the equvalence between the smplcty and lkelhood prncples, Because there s a one-to-one relatonshp between probabltes and code lengths, a small (but not qute mnmal) code wll correspond to a probable (but not qute most probable) organzaton. So maxmzng smplcty and maxmzng lkelhood are equvalent, even when maxmzaton s approxmate rather than exact. Implcatons for Other Psychologcal Processes Could the relatonshp between smplcty and lkelhood prncples be relevant to other areas of psychology? One possble applcaton s to areas of low-level percepton n whch compresson of the sensory sgnal has been vewed as a central goal (Atck & Redlch, 1990; Barlow, Kaushal, & Mtchson, 1989; Blakemore, 1990). ts The goal of compresson s frequently vewed as stemmng from lmtatons n the nformaton-carryng capacty of the sensory pathways. However, the equvalence of maxmzng compresson (.e., mnmzng descrpton length) and maxmzng lkelhood ndcates a complementary nterpretaton. It could be that compressed perceptual representatons wll tend to nvolve the extracton of features lkely to have generated the sensory nput. Accordng to ths complementary nterpretaton, perceptual nference occurs n the very earlest stages of percepton (e.g., as mplemented n mechansms such as lateral nhbton n the retna), where neural codng serves to compress the sensory nput. The relatonshp between the smplcty and lkelhood prncples may also be relevant to the relatonshp between nference and memory. Perceptual, lngustc, or other nformaton s not remembered n a "raw" form, but n terms of hgh-level categores and relatons organzed nto structured representatons (e.g., Anderson, 1983; Hnton, 1979; Johnson-Lard & Stevenson, 1970) such as "sketches" (Mart, 1982), schemata (Bobrow & Norman, 1975), scrpts (Schank & Abelson, 1977), or frames (Mnsky, 1977 ). Two constrants on such memory organzatons suggest themselves: (a) that they allow the structure of the world to be captured as well as possble, and (b) that they allow the most compact encodng of the nformaton to be recalled, so that memory load s mnmzed. Prma face, these goals mght potentally conflct and requre the cogntve system to somehow make approprate trade-offs between them. But, as we have seen, the goal of capturng the structure of the world and the goal of provdng a compressed representaton can be seen as equvalent. 14 Strctly speakng, ths statement requres a caveat. It apples only to neural network and analog styles of computaton where states need not be specfed wth nfnte precson. Ths restrcton s very mld, snce t seems extremely unlkely, to say the least, that an nfnte precson computatonal method could be mplemented n the bran, partcularly n vew of the apparently nosy character of neural sgnals. 15 These theorsts advocate a varety of dstnct proposals concernng the objectves of percepton. They are all closely related to the goal of mnmzng descrpton length, although not necessarly couched n these terms.

14 SIMPLICITY VERSUS LIKELIHOOD 579 More generally, the relatonshp between smplcty and lkelhood prncples may be useful n ntegratng psychologcal theores that stress probablstc nference (e.g., Anderson, 1990; Fred & Holyoak, 1984; Oaksford & Chater, 1994) and those that stress the mportance of fndng compressed representatons (Redlch, 1993; Wolff, 1982). Furthermore, the possblty of vewng cogntve processes from two complementary perspectves may throw valuable lght on both knds of accounts. What the Smplcty and Lkelhood Prncples Leave Out Ths artcle has been concerned wth showng that the smplcty and lkelhood prncples are dentcal. I have not consdered whether the unfed smplcty-lkelhood prncple really governs perceptual organzaton. Ths queston s clearly a large topc for future research, and a full dscusson s beyond the scope of ths artcle. Nonetheless, two prelmnary comments are worth makng. Frst, on the postve sde, the evdence for the unfed smplcty-lkelhood prncple s the sum of the evdence that has been prevously adduced n favor of the smplcty and lkelhood prncples. Second, on the negatve sde, the smplcty-lkelhood prncple gnores a factor that may be of consderable mportance: the nterests and potental actons of the agent. The applcaton of the smplcty-lkelhood prncple can be vewed as "dsnterested contemplaton" of the sensory stmulus: The smplest encodng, or the most lkely envronmental layout, s sought wthout any concern for the specfc nterests of the percever. But percevers are not dsnterested; they are concerned wth partcular goals and actons and hence wth aspects of the envronment relevant to those goals and actons. The frog's perceptual system s, for example, geared toward the detecton of dark, fast, movng concave blobs (among other thngs), because ths nformaton allows the frog to perform actons (snappng n the approprate drecton) that satsfy ts nterests (eatng fles; Lettvn, Maturana, McCullough, & Ptts, 1959). Smlarly, the fly s senstve to correlates of optc expanson because of the mportance of ths nformaton n the tmng of landng (Poggo & Rechardt, 1976; Rechardt & Poggo, 1976). It has been suggested that many aspects of human percepton, too, may be explaned n terms of people's nterests and motor abltes (e.g., Gbson, 1979). Indeed, one of the most mportant tenets of Gbson's "drect" percepton s that agents pck up propertes of the world that afford varous actons to the agent, such as beng lfted, reached, grasped, or clmbed. The mportant pont here s that affordances are defned n terms of the actons and goals of the agent: just those factors that the smplcty and lkelhood prncples gnore. The mportance of affordances and lke notons has been wdely dscussed n many areas of percepton, but has not been a focus of nterest n the lterature on perceptual organzaton. It s possble that the nterests and actons of the agent are not relevant n organzatonal processes n percepton. On the other hand, t s possble that here, too, t s necessary to vew the percever not merely as a dsnterested observer of sensory stmul, but as usng sensory nput to determne approprate actons. Devsng expermental tests of the mportance of nterests and actons n the context of perceptual organzaton s an mportant emprcal challenge. If these factors do nfluence perceptual organzaton, then the smplctylkelhood prncple must be elaborated or replaced. Concluson I have shown that the smplcty and lkelhood prncples n perceptual organzaton are equvalent, gven natural nterpretatons of smplcty n terms of shortest descrpton length and of lkelhood n terms of probablty theory. Ths mples that the emprcal and theoretcal debate over whether perceptual organzaton maxmzes smplcty or maxmzes lkelhood s msguded. Instead, the fundamental queston s whether, or to what extent, perceptual organzaton s maxmzng smplcty and maxmzng lkelhood. References Anderson, J. A. (1983). The archtecture of cognton. Cambrdge, MA: Harvard Unversty Press. Anderson, J. A. (1990). The adaptve character of thought. Hllsdale, N J: Erlbaum. Atck, J. J., & Redlch, A. N. (1990). Towards a theory of early vsual processng. Neural Computaton, 2, Atck, J. J., & Redlch, A. N. ( 1992 ). What does the retna know about natural scenes? Neural Computaton, 4, Attneave, F. (1954). Some nformatonal aspects of vsual percepton. Psychologcal Revew, 61, Attneave, E (1959). Applcatons of nformaton theory to psychology. New York: Holt, Rnehart & Wnston. Attneave, F. ( 1972 ). Representaton of physcal space. In A. W. Melton & E. J. Martn (Eds.), Codng processes n human memory (pp ). Washngton, DC: Wnston. Attneave, F. ( 1981 ). Three approaches to perceptual organzaton. In M. Kubovy & J. R. Pomerantz (Eds.), Perceptual organzaton (pp ). Hllsdale, N J: Erlbaum. Attneave, E ( 1982 ). Pr~gnanz and soap bubble systems: A theoretcal exploraton. In J. Beck (Ed.), Organzaton and representaton npercepton (pp ). Hllsdale, N J: Erlbaum. Attneave, E, & Frost, R. (1969). The determnaton of perceved trdmensonal orentaton by mnmum crtera. Percepton & Psychophyscs, 6, Barlow, H. B., Kaushal, T. P., & Mtchson, G. J. (1989). Fndng mnmum entropy codes. Neural Computaton, 1, Blakemore, C. (Ed.). (1990). Vson: Codng and effcency. Cambrdge, England: Cambrdge Unversty Press. Bobrow, D. G., & Norman, D. A. ( 1975 ). Some prncples of memory schemata. In D. G. Bobrow & A. Collns (Eds.), Representaton and understandng: Essays n cogntve scence ( pp ). New York: Academc Press. Boolos, G. S., & Jeffrey, R. C. (1980). Computablty and logc. Cambrdge, England: Cambrdge Unversty Press. Brunswck, E. ( 1956 ). Percepton and the representatve desgn of psychologcal experments. Berkeley: Unversty of Calforna Press. Buffart, H., Leeuwenberg, E., & Restle, E ( 1981 ). Codng theory of vsual pattern completon. Journal of Expermental Psychology. Human Percepton and Performance, 7, Carnap, R. (1950). Logcal foundatons of probablty Chcago: Unversty of Chcago Press. Carnap, R. ( 1952 ). The contnuum of nductve methods. Chcago: Unversty of Chcago Press. Chatn, G. J. (1966). On the length of programs for computng fnte

15 580 CHATER bnary sequences. Journal of the Assocaton for Computng Machnery, 13, Chater, N. (1989). Informaton and nformaton processng. Unpublshed doctoral dssertaton, Centre for Cogntve Scence, Unversty of Ednburgh, Ednburgh, Scotland. Cheeseman, P. (1995). On Bayesan model selecton. In Wolpert, D. (Ed.), The mathematcs of generalzaton (pp ). Redwood Cty, CA: Addson-Wesley. Cover, T. M., & Thomas, J. A. ( 1991 ). Elements of nformaton theory New York: John Wley. de Fnett, B. (1972). Probablty, nducton and statstcs. New York: Wley. Dretske, F. I. ( 1981 ). Knowledge and theflow of nformaton. Cambrdge, MA: MIT Press. Earman, J. (1992). Bayes of bust? A crtcal examnaton of Bayesan confrmaton theory Cambrdge, MA: MIT Press. Fsher, R. A. (1922). On the mathematcal foundatons of theoretcal statstcs. Phlosophcal Transactons of the Royal Socety of London, A Fred, L. S., & Holyoak, K. J. (1984). Inducton of category dstrbutons: A framework for classfcaton learnng. Journal of Expermental Psychology: Learnng, Memory, and Cognton, 10, Garner, W. R. (1962). Uncertanty and structure as psychologcal concepts. New York: Wley. Garner, W. R. ( 1974). The processng of nformaton and structure. Potomac, MO: Erlbaum. Gbson, J. J. (1950). The percepton of the vsual world. Boston: Houghton Mffln. Gbson, J. J. ( 1966 ). The senses consdered as perceptual systems. Boston: Houghton Mffln. Gbson, J. J. (1979). The ecologcal approach to vsual percepton. Boston: Houghton Mffln. Goa, Q., & L, M. (1989), An applcaton oftbe mnmum descrpton length prncple to on-lne recognton of handprnted alphanumerals. In l lth Internatonal Jont Conference on Artfcal Intellgence (pp ). San Mateo, CA: Morgan Kaufman. Grenander, U. ( ). Lectures n pattern theory L H and II1: Pattern analyss, pattern synthess and regular structures. Hedelberg, Germany: Sprnger-Verlag. Harman, G. ( 1965 ). The nference to the best explanaton. Phlosophcal Revew, 74, Hatfeld, G., & Epsten, W. (1985). The status of the mnmum prncple n the theoretcal analyss of vsual percepton. Psychologcal Bulletn, 97, Helmholtz, H. L. E yon (1962). Treatse on physologcal optcs (Vol. 3; J. P. Southall, Ed. and Trans.). New York: Dover. (Orgnal work publshed 1910) Hnton, G. E. (1979). Some demonstratons of the effects of structural descrpton n mental magery. Cogntve Scence, 3, Hochberg, J. (1982). How bg s a stumulus? In J. Beck (Ed.), Organzaton and representaton n percepton (pp ). Hllsdale, N J: Erlbaum. Hochberg, J., & McAlster, E. (1953). A quanttatve approach to fgure "goodness." Journal of Expermental Psychology, 46, Horwch, E (1982). Probablty and evdence. Cambrdge, England: Cambrdge Unversty Press. Howson, C., & Urbach, E (1989). Scentfc reasonng: The Bayesan approach. La Salle, IL: Open Court. Jeffrey, R. C. (1983). The logc of decson. New York: McGraw-Hll. Jeffreys, H., & Wrnch, D. ( 1921 ). On certan fundamental prncples of scentfc enqury. Phlosophcal Magazne, J ohansson, G. (1950). Confguratons n event percepton. Stockholm: Almqvst & Wksell. Johnson-Lard, P. N., & Stevenson, R. (1970). Memory for syntax. Nature, 227, 412. Keynes, J. M. ( 1921 ). A treatse on probablty London: Macmllan. Koffka, K. (1963). Prncples of Gestalt psychology New York: Harcourt, Brace & World. (Orgnal work publshed 1935 ) Kolmogorov, A. N. (1965). Three approaches to the.quanttatve defnton of nformaton. Problems n Informaton Transmsson, 1, 1-7. Leeuwenberg, E. (1969). Quanttatve specfcaton of nformaton n sequental patterns. Psychologcal Revew, 76, Leeuwenberg, E. ( 1971 ). A perceptual codng language for perceptual and audtory patterns. Amercan Journal of Psychology, 84, Leeuwenberg, E., & Bosele, E (1988). Aganst the lkelhood prncple n vsual form percepton. Psychologcal Revew, 95, Lettvn, J. Y., Maturana, W. S., MeCullough, W., & Ptts, W. H. (1959). What the frog's eye tells the frog's bran. Proceedngs of the Insttute of Rado Engneers, 47, Levn, L. A. (1974). Laws of nformaton conservaton (non-growth) and aspects of the foundatons of probablty theory. Problems n Informaton Transmsson, 10, L, M., & Vtany, E ( 1993 ). An ntroducton to Kolmogorov complexty and ts applcatons. New York: Sprnger-Verlag. L, M., & Vtany, E ( 1995 ). Computatonal machne learnng n theory and praxs. (Tech. Rep. No. NC-TR ) London: Royal Hol- Ioway College, Unversty of London, Department of Computer Scence. Lndley, D. V. ( 1971 ). Bayesan statstcs: A revew. Phladelpha: Socety for Industral and Appled Mathematcs. Math, E. ( 1959 ). The analyss of sensatons and the relaton of the physcal to the psychcal. New York: Dover Publcatons. (Orgnal work publshed 1914) Marr, D. (1982). Vson. San Francsco: Freeman. Mellor, D. H. ( 1971 ). The matter of chance. Cambrdge, England: Cambrdge Unversty Press. Mnsky, M. ( 1967 ). Computaton: Fnte and nfnte machnes. Englewood Clffs, N J: Prentce Hall. Mnsky, M. (1977). Frame-system theory. In E N. Johnson-Lard & E C. Wason (Eds.), Thnkng: Readngs n cogntve scence (pp ). New York: McGraw-HU. Mumford, D. (1992). Pattern theory: A unfyng perspectve, In Joseph, A., Mgnot, E, Murat, E, Prum, B., & Rentschler, R. (Eds.), Proceedngs of the Frst European Congress of Mathematcs (pp ). Basel, Swtzerland: Brkhuser Verlag. Oaksford, M., & Chater, N. (1994). A ratonal analyss of the selecton task as optmal data selecton. Psychologcal Revew,, 101, Odfredd, E ( 1989 ). Classcal recurson theory Amsterdam: Elsever. Pednault, E. E D. ( 1989 ). Some experments n applyng nductve nference prncples to surface reconstructon. In llth Internatonal Jont Conference on Artfcal Intellgence (pp ). San Mateo, CA: Morgan Kaufman. Perkns, D. N. (1972). Vsual dscrmnaton between rectangular and nonrectangular paralleleppeds. Percepton and Psychophyscs, Perkns, D. N. (1982). The percever as organzer and geometer. In J. Beck (Ed.), Organzaton and representaton n percepton (pp ). Hllsdale, NJ: Erlbaum. Poggo, T., & Rechardt, W. (1976). Vsual control of orentaton behavor n the fly: Part ll. Towards the underlyng neural nteractons. Quarterly Revew of Bophyscs, 9, Pomerantz, J. R. ( 1981 ). Perceptual organzaton n nformaton processng. In M. Kubovy & J. R. Pomerantz (Eds.), Perceptual organzaton (pp ). Hllsdale, NJ: Erlbaum. Pomerantz, J. R., & Kubovy, M. (1986). Theoretcal approaches to perceptual organzaton: Smplcty and lkelhood prncples. In K. R.