Learning: Chapter 7: Instrumental Conditioning

Transcription

1 Learning: Chapter 7: Instrumental Conditioning W. J. Wilson, Psychology November 8, Reinforcement vs. Contiguity Thorndike (a reinforcement theorist): Law of effect: positive consequences strengthen responses; negative consequences weaken responses Cat in puzzle box cat gradually learns to solve, because successful Rs become more likely. Guthrie (a contiguity theorist): Reinforcement not necessary: S & R occur together and they become associated. Learning requires only 1 trial But what about gradual learning curve? Stimulus elements: Portions of S are associated with R on each trial (only those portions being attended) What about obvious effects of reinforcement (e.g., rat runs faster when reinforced at end of runway)? *** Reinf acts as an especially salient stimulus, and is thus available to be associated with an R. Runway alone is associated with running (last R with runway alone was running). Post-reward competing Rs are associated with runway + food. therefore, competing Rs are NOT associated with runway by itself, so rat runs in runway alone. 2. Reinforcement as motivation (Tolman) Behavior is a means to an end it is flexible. 1

2 Rs are not learned, rather S is learned about. S-S theorist (Thorndike & Guthrie were S-R theorists) Macfarlane (1930): trained rats to swim through a maze for a reward, then drained maze. Rats ran to the goal. R of swimming had not been learned. Tolman et al. 1946: two important studies: Rats learn circuitous path to a feeding station. When this path is replaced by straight arms, they run directly to the station. Rats in plus maze learn either Always Go West or Always Turn Left. First is much easier for them to learn. Learning about the location of food, not the R to make. Tolman & Honzik (1930) Rats in 14-unit T-maze. Reinf group, NotReinf group, and NotReinf-Reinf group. On Trial 11, rats in NR-R group get food reward for the first time. R rats learn maze (as indicated by gradual reduction in errors) NR rats never reduce errors much NR-R rats perform better than R rats once they are reinforced. Latent learning! Cognitive map Reinforcer was a motivator. Tolman emphasized the Learning/Performance distinction. 3. Skinner s Operant psychology Skinner was atheoretical - focussed entirely on S & R; believed the mind (and even the brain) was irrelevant to an understanding of behavior. Operants: Rs that operate on the environment (& can be made whenever the animal wants, hence they seem voluntary) Superstitious behaviors Operants are under stimulus control: S D and S 2

3 Conditioned reinforcers are important they can become S D s for additional Rs, leading to behavior chains Skinner developed cumulative recorder (on cumulative record, slope represents rate of responding) 4. Schedules of reinforcement 5. Choice Fixed Ratio (FR1 is special case: continuous reinforcement) Fixed Interval scalloped cumulative record Varied Ratio highest rate of responding Varied Interval slow, steady rate others are possible: e.g., DRL Partial Reinforcement Effect Concurrent schedules allow examination of choice behavior Herrnstein (1961) Matching Law: B 1 /(B 1 + B 2 ) = R 1 /(R 1 + R 2 ) (1) e.g., behavior occurs proportionally to the extent to which it is reinforced. Choice is everywhere even if only 1 behavior is being measured, many others are always available to the subject. Herrnstein (1970) Quantitative Law of Effect: B 1 = KxR 1 /(R 1 + R O ) (2) K: constant reflecting all possible behavior in a given situation R O : constant reflecting total reinforcement value of all other behaviors. K & R will vary from animal to animal Bouton suggests implications for understanding attraction of drugs to some people, especially those with low R O 3

4 6. Impulsiveness Animals (human and non-human) will select small soon reward over large delayed reward If choice is made well in advance of the rewards, larger one is usually chosen, even if it comes later. Rachlin suggests that pre-commitment can avoid impulsive choice. Implications for behavior on Deal or No Deal? 7. Behavioral Economics: Are All Reinforcers alike? Tinklepaugh (1928): monkeys, banana, lettuce. Tells us about substitutability, cognition. Reinforcers can be substitutes, independent, or complements. Meta-analysis of drugs as reinforcers: PCP cost goes up, alcohol substitutes Alcohol cost goes up, PCP independent! Alcohol cost goes up, cigarettes act as complements. An understanding of substitutability is necessary to an understanding of reinforcement. 8. Theories of Reinforcement Hull s Drive Reduction Drive arises from need shortage of a biological essential Drive reduction is reinforcing BUT: many reinforcers unrelated to need Premack Principle More-preferred R will reinforce a less-preferred R. Measure which of two Rs is more preferred, can safely predict that it will reinforce the other one. Sometimes adequate or appropriate measure of preference is difficult 4

5 Behavior Regulation Theory (response deprivation hypothesis) Preferred level for every R If R is prevented to a point below its preferred level, animal will engage in it more when given the opportunity. Result is that a ledd-preferred R, if prevented, will then reinforce a more-preferred R. Bliss point illustrates preferred level of each of two Rs when preference is measured. When sched of reinf constrains extent to which Rs can occur, the level of each will achieve the minimum distance to the bliss point. Behav Regul theory works well for ratio scheds, but Matching does better for interval scheds. Selection by Consequences Recent interest in application of natural selection to behavioral choice and reinforcement. Variation in behavior ensures that multiple Rs are available; consequences select the ones that survive. Remains to be seen how valuable this approach is to understanding reinforcement & onstrumental conditioning. 5