Psychology of Learning

Module 06: Operant Conditioning 1

Key Terms

Chained Schedule: A compound reinforcement schedule in which two or more simple schedules are placed in a fixed sequence, with each schedule signaled by different discriminative stimuli; completion of one schedule provides access to the next.

Concurrent Schedule: A reinforcement arrangement in which an organism is exposed to two or more different schedules simultaneously on different response options; used to study choice and preference.

Conditioned Reinforcer: A stimulus that has acquired reinforcing properties through association with a primary reinforcer; also called a secondary reinforcer. For example, the sound of a pellet dispenser becomes a conditioned reinforcer after being paired with food delivery.

Continuous Reinforcement: A reinforcement schedule in which every occurrence of the target behavior is reinforced; produces steady rates of behavior during acquisition but low resistance to extinction.

Differential Reinforcement: The process of reinforcing some responses while not reinforcing others; used to make responding selective and to establish discrimination between stimuli that signal reinforcement versus those that don’t.

Discrimination: In operant conditioning, the process by which an organism learns to tell the difference between two or more forms of a stimulus, responding to those that signal reinforcement and not responding to those that don’t.

Discrimination Learning: The process of learning that the same behavior produces different consequences depending on which discriminative stimulus is present; the organism learns to respond selectively.

Discriminative Stimulus (S^D): A stimulus that signals that a particular behavior will lead to a particular consequence; sets the occasion for responding by indicating when reinforcement is available.

Extinction: In operant conditioning, the elimination of a behavior by removing the reinforcement that previously maintained it; the behavior decreases in frequency when it no longer produces consequences.

Extinction Burst: The temporary increase in the frequency, intensity, or duration of a behavior immediately after reinforcement is discontinued; behavior typically gets worse before it gets better during extinction.

Fixed-Interval (FI) Schedule: A reinforcement schedule that provides reinforcement for the first response after a specific, constant time period has elapsed; produces a scalloped response pattern with low responding after reinforcement and acceleration as the next interval ends.

Fixed-Ratio (FR) Schedule: A reinforcement schedule that provides reinforcement after a set, constant number of responses; produces high, steady rates of responding with brief post-reinforcement pauses.

Generalization: In operant conditioning, the tendency to respond in the same way to stimuli that are similar but not identical to the original discriminative stimulus; once a behavior is reinforced in the presence of one stimulus, similar stimuli also evoke that behavior.

Instrumental Conditioning: Another term for operant conditioning; emphasizes that behavior is instrumental in producing consequences.

Intermittent Reinforcement: A reinforcement schedule in which only some occurrences of the target behavior are reinforced; also called partial reinforcement. Produces greater resistance to extinction than continuous reinforcement.

Law of Effect: Thorndike’s principle stating that behaviors followed by satisfying consequences are more likely to be repeated, while behaviors followed by annoying consequences are less likely to be repeated; the cornerstone of operant conditioning.

Law of Exercise: Thorndike’s original principle that practice strengthens associations; later discarded when Thorndike recognized that practice alone, without reinforcement, does not strengthen learning.

Law of Recency: Thorndike’s principle that the most recently demonstrated behavior is the behavior most likely to be repeated in a given situation.

Meaning-Based Generalization: Stimulus generalization that occurs when two stimuli have been paired together during learning and come to share the same meaning in terms of predicting reinforcement, even if they are not physically similar.

Negative Punishment: The removal of a pleasant stimulus or consequence following a behavior, intended to decrease the future likelihood of that behavior; also called omission training. Example: removing television privileges after misbehavior.

Negative Reinforcement: The removal of an unpleasant stimulus or consequence following a behavior, intended to increase the future likelihood of that behavior. Example: taking aspirin removes headache, reinforcing aspirin-taking behavior.

Omission Training: Another term for negative punishment; a procedure in which a pleasant stimulus is removed following a target behavior, decreasing the future probability of that behavior.

One-Trial Learning: Guthrie’s contention that associations are formed at full strength after a single pairing of stimulus and response; what appears as gradual learning reflects learning many different movements under varying conditions.

Operant Conditioning: A form of learning in which voluntary behaviors are modified by their consequences; behaviors followed by reinforcement increase in frequency, while behaviors followed by punishment decrease.

Operant Conditioning Chamber: An apparatus designed by Skinner for the systematic study of operant conditioning; contains mechanisms for delivering stimuli and recording responses. Also called a Skinner box.

Partial Reinforcement: See Intermittent Reinforcement.

Positive Punishment: The application of an unpleasant stimulus or consequence following a behavior, intended to decrease the future likelihood of that behavior. Example: receiving a speeding ticket decreases speeding.

Positive Reinforcement: The application of a pleasant stimulus or consequence following a behavior, intended to increase the future likelihood of that behavior. Example: praise after helpful behavior increases future helpfulness.

Post-Reinforcement Pause: The brief cessation of responding that occurs immediately after reinforcement, particularly evident in fixed-ratio schedules; longer pauses occur with larger ratio requirements.

Primary Reinforcer: A stimulus that has innate reinforcing properties without requiring any learning; satisfies biological needs. Examples include food, water, warmth, and sexual contact.

Progressive-Ratio Schedule: A reinforcement schedule similar to a fixed-ratio schedule, but the ratio requirement increases after each reinforcement; used to measure motivation strength or reinforcer value.

Puzzle Box: Thorndike’s apparatus for studying trial-and-error learning; animals had to perform specific behaviors to escape the box and obtain food.

Radical Behaviorism: Skinner’s philosophical position that behavior can be fully explained by environmental variables without reference to internal mental states; emphasizes the study of observable behavior and its environmental determinants.

Reinforcing Stimulus (S^R): The consequence that follows a response in operant conditioning; can be positive (added) or negative (removed) and serves to increase the future probability of the behavior.

Resistance to Extinction: The tendency for a behavior to persist during extinction; behaviors learned under intermittent reinforcement, especially variable schedules, show greater resistance to extinction than behaviors learned under continuous reinforcement.

Response (R): In the three-part contingency, the voluntary behavior that the organism performs; the operant behavior that is influenced by its consequences.

Scalloped Response Pattern: The characteristic pattern of responding produced by fixed-interval schedules, showing low rates immediately after reinforcement with gradually accelerating rates as the next reinforcement opportunity approaches.

Shaping: A form of operant learning in which an organism learns a complex behavior through reinforcement of successive approximations to the target behavior; behaviors gradually become more similar to the goal behavior through differential reinforcement.

Similarity-Based Generalization: Stimulus generalization that occurs when two stimuli are physically similar (such as lights of similar wavelength); the more similar stimuli are physically, the more generalization occurs.

Skinner Box: See Operant Conditioning Chamber.

Spontaneous Recovery: In operant conditioning, the reappearance of an extinguished behavior after a period of time, particularly when cues previously associated with the behavior are present.

Stop Action Principle: Guthrie’s idea that reinforcement works by changing the situation, thereby “capturing” or preserving the last movements made before the situation changed; reinforcement acts like a camera taking a snapshot of behavior.

Successive Approximations: In shaping, the series of intermediate behaviors that are reinforced as the organism moves progressively closer to the target behavior; each approximation is reinforced until reliable, then the criterion for reinforcement advances.

Superstitious Behavior: Behavior that develops based on an accidental temporal relationship between a response and a reinforcer; the organism acts as if the behavior caused the outcome even though no causal relationship exists.

Temporally Weighted Matching Law: A theoretical framework explaining extinction bursts based on how organisms allocate behavior between target responses and reinforcer consumption; proposed by Fisher et al. (2023).

Three-Part Contingency: Skinner’s model of operant conditioning consisting of the discriminative stimulus (S^D), the response (R), and the reinforcing stimulus (S^R); written as S^D → R → S^R.

Time-Out: A form of negative punishment in which access to reinforcing activities and stimuli is removed for a specified period following an undesired behavior; the individual must remain in a designated location away from reinforcement.

Trial-and-Error Learning: The gradual learning process observed by Thorndike in which organisms try various responses and learn which ones produce satisfying consequences; incorrect responses are gradually eliminated while successful responses are strengthened.

Variable-Interval (VI) Schedule: A reinforcement schedule that provides reinforcement for the first response after an unpredictable time interval has elapsed; the average interval is known but specific intervals vary. Produces slow, steady responding with no pauses.

Variable-Ratio (VR) Schedule: A reinforcement schedule that provides reinforcement after an unpredictable number of responses; the average ratio is known but specific requirements vary. Produces high, steady rates of responding with no post-reinforcement pauses and extreme resistance to extinction.