Psychology of Learning

Module 05: Classical Conditioning 2

Part 3: Theoretical Accounts of Classical Conditioning

Looking Back

In Parts 1 and 2, we explored challenges to traditional views of classical conditioning. Taste aversion learning violated principles about CS-US timing and number of pairings while revealing preparedness—organisms’ innate tendencies to associate certain stimuli more readily with certain outcomes. Cue competition effects (blocking, overshadowing, potentiation) demonstrated that stimuli don’t condition independently—they compete for or facilitate associative strength based on predictive value. Drug tolerance revealed compensatory CRs that oppose rather than mimic URs. Extinction effects (facilitated reacquisition, renewal) showed that extinction creates new learning rather than erasing original associations. These complex phenomena demanded sophisticated theoretical accounts. Simple stimulus-response association theories couldn’t explain why organisms learn some associations but not others, why prior learning blocks new learning, or why contexts influence whether extinguished responses return. In this part, we examine three theoretical perspectives attempting to explain what organisms actually learn during classical conditioning and the physiological mechanisms underlying this learning.

Pavlov’s Stimulus Substitution Theory

Because Pavlov was a physiologist, he was naturally interested in what changes might occur in the brain during conditioning. His stimulus substitution theory proposed a neural mechanism for how CSs acquire the power to elicit CRs (Pavlov, 1927).

The Basic Mechanism

When an experimenter presented meat powder to a test animal, Pavlov theorized that the meat powder activated a US center in the brain. Once activated, the US center automatically and innately activated a UR brain center, which in turn automatically activated neural pathways needed to produce the response. This represents the natural, reflexive connection between US and UR—the unconditioned reflex hardwired into the nervous system.

When a CS, such as a ticking metronome, was presented, it also activated a brain center—the CS center. Initially, this CS center had no connection to response centers. But through repeated CS-US pairings, Pavlov proposed that connections formed between the CS center and other brain centers.

S-S Versus S-R Associations

Two possibilities emerged for how the CS comes to elicit responses. In S-S association (Stimulus-Stimulus), the CS brain center becomes associated with the US brain center; when the CS activates its center, this activation spreads to the US center, which then activates the UR center, producing the response. The CS substitutes for the US by activating its representation. In S-R association (Stimulus-Response), the CS brain center becomes directly associated with the response center, bypassing the US center entirely; the CS elicits the response directly without activating US representations.

Although he lacked research data, Pavlov appeared to favor the S-S interpretation. He believed conditioning involved learning about stimulus relationships—that the CS signaled the US—rather than just forming direct stimulus-response connections.

Rescorla’s Evidence for S-S Associations

Decades later, Rescorla (1973) provided elegant experimental support for S-S associations. In his experiment, the US (loud noise) naturally led to suppression of lever pressing (UR). A light (CS) was paired with the noise until the light produced conditioned suppression (CR).

During Phase 2, one group was habituated to the noise—they heard it repeatedly without shock until they no longer responded to it. The control group received no Phase 2 treatment. Then both groups were tested with the light alone.

Prediction: If conditioning involves S-S associations (light activates noise representation, which produces suppression), habituation to noise should eliminate light’s effectiveness. If conditioning involves S-R associations (light directly elicits suppression), habituation to noise shouldn’t matter.

Results: Suppression occurred only in the control group. The habituation group showed no conditioned suppression to the light. This supports S-S associations—the light activated a representation of the noise, and when that representation was weakened through habituation, the CR disappeared (Rescorla, 1973).

However, Pavlov’s theory couldn’t account for several phenomena discovered later: blocking (why doesn’t the second CS condition?), overshadowing (why does salience matter?), or extinction effects. More sophisticated theories were needed.

The Rescorla-Wagner Model: Learning from Surprise

The Rescorla-Wagner model is a theory of classical conditioning in which the strength of conditioning depends on the surprisingness of the US—specifically, when there is a discrepancy between what is expected and what actually occurs. This mathematically based model attempts to predict conditioned responding on a trial-by-trial basis (Rescorla & Wagner, 1972).

The model revolutionized thinking about conditioning by proposing that organisms don’t simply form associations through contiguity. Instead, during classical conditioning, organisms acquire information about the meaning of signals. They learn to predict future events based on current cues.

The Core Principle: Prediction Error

The model assumes learning occurs when there’s a discrepancy between what is expected and what actually occurs. If a dog expected meat powder following a metronome sound, it would be surprised when the metronome sounded without meat powder appearing. When the US is surprising, signals for that US are relevant and need to be remembered. Hence, learning occurs in “surprising” conditions.

Mathematically, the degree of surprise is expressed as the difference between the expected US and the received US. A rat may learn that a tone (CS) is followed by shock (US). After several CS-US pairings, the rat expects shock when the tone sounds. If shock occurs as expected, no surprise—minimal learning. If shock doesn’t occur, surprise—new learning (extinction). If a stronger shock occurs, surprise—more excitatory conditioning.

The Rescorla-Wagner Equation

The model expresses these ideas mathematically: ΔVi = αiβj(λj – ΣVi). In this equation, ΔVi represents the amount of conditioning on a given trial (change in associative strength); αi represents the salience of the CS (how noticeable it is, ranging from 0 to 1); βj represents the salience of the US (how intense it is, ranging from 0 to 1); λj represents the total amount of conditioning possible (maximum associative strength the US can support); and ΣVi represents the amount of conditioning already present (sum of all CSs’ associative strengths).

The key term is (λj – ΣVi)—this represents surprise or prediction error. When expected US (ΣVi) is less than actual US (λj), positive prediction error drives learning. When expected exceeds actual, negative prediction error produces extinction or inhibition.

Six Rules of the Rescorla-Wagner Model

Rules 1-3 determine what type of conditioning occurs.

• Rule 1: If actual US strength > expected US strength, excitatory conditioning occurs (the US is better than expected—learn to predict it more strongly).
• Rule 2: If actual US strength < expected US strength, inhibitory conditioning occurs (the US is worse than expected—learn that it won’t occur).
• Rule 3: If actual US strength = expected US strength, no conditioning occurs (perfect prediction—nothing new to learn).

Rules 4-6 determine how much change occurs.

• Rule 4: Larger discrepancies produce more conditioning; bigger surprises drive faster learning.
• Rule 5: More salient CSs condition faster; noticeable cues gain associative strength more rapidly.
• Rule 6: With compound CSs, expectations equal total strength of all elements; associative strengths sum when multiple cues are present.

Explaining Acquisition

On Trial 1, when a rat sees a light followed by food, it has no expectation (ΣVi = 0). The actual US (λj = 1 food pellet) greatly exceeds expectation, producing substantial excitatory conditioning. On Trial 2, a weak expectation exists. The difference between expected and actual remains large, but smaller than Trial 1. With each trial, conditioning increases, expectation rises, and prediction error shrinks. Eventually, expectation equals the actual US—no more surprise, no more learning. Conditioning reaches asymptote.

Explaining Blocking

Blocking occurs because no learning happens when expected US equals actual US. First, a tone is conditioned to asymptote (tone predicts food perfectly, ΣVi = λj). Then compound (tone + light) is paired with food. The tone already produces maximum expectation (ΣVi = λj), so adding light doesn’t create surprise. Because (λj – ΣVi) = 0, no conditioning occurs to the light. No surprise means no learning—the light is redundant.

Explaining Extinction

During extinction, expected US > actual US, producing inhibitory conditioning. Initially, the conditioned light produces expectation of 1 food pellet (ΣVi = 1), but actual outcome is 0 pellets (λj = 0). This creates negative prediction error (λj – ΣVi = -1), producing inhibitory learning. The light develops inhibitory associations that oppose its original excitatory strength. This explains why extinction doesn’t erase learning—it adds inhibitory learning on top of excitatory learning.

Explaining Overshadowing

Overshadowing occurs because conditioning depends on CS salience (Rule 5). When a compound stimulus (weak light + strong noise) is presented, initially there’s no expectation. Food arrival produces excitatory conditioning to both components. But the strong CS gains more conditioning per trial due to higher α value. As trials continue, both gain associative strength, but the salient stimulus gains more. When total expectation (ΣVi) reaches actual US (λj), learning stops. The strong CS captured most associative strength, overshadowing the weak CS.

The Overexpectation Effect

The Rescorla-Wagner model makes a counterintuitive prediction: the overexpectation effect. In Phase 1, rats receive separate trials where light → food and tone → food. Both CSs acquire full conditioning (each predicts 1 pellet). In Phase 2, the compound (light + tone) is presented followed by 1 pellet. What happens?

The rats expect 2 pellets (light predicts 1, tone predicts 1, expectations sum). They receive 1 pellet. Expected US > actual US, so inhibitory conditioning occurs to both! Testing shows weakened CRs to both stimuli. This surprising prediction has been confirmed experimentally—pairing two fully conditioned CSs together with the same US actually weakens their individual strength (Rescorla, 1970).

Contemporary Evidence for the Rescorla-Wagner Model

Recent comprehensive reviews confirm the enduring value of the Rescorla-Wagner model for understanding fear learning. Yau and McNally (2023) reviewed extensive evidence supporting two key insights from the model. First, learning to fear and learning to reduce fear are both governed by a common, signed prediction error—the same computational principle underlies acquisition, extinction, and other fear learning phenomena. Second, this prediction error drives variations in the effectiveness of the shock US that are causal to whether and how much fear is learned or lost during a conditioning trial. The model’s experimental approaches continue to provide powerful behavioral tools for advancing mechanistic understanding of fear learning, even as researchers identify neural findings that will require future theoretical refinements. The prediction error concept from the Rescorla-Wagner model has also proven foundational for understanding dopamine neuron function, creating important bridges between behavioral learning theory and neuroscience.

The Comparator Hypothesis: It’s All Relative

The comparator hypothesis proposes that the strength of the CR depends on comparing the CS’s association with the US to the strength of other stimuli’s associations with the US. Rather than absolute associative strength, what matters is relative predictiveness—how well does the CS predict the US compared to alternative predictors? (Miller & Matzel, 1988).

The basic premise is that learning occurs when two or more stimuli are contiguous (close in time or space). Unlike the Rescorla-Wagner model, which emphasizes learning through prediction error, the comparator hypothesis focuses on performance—how associations are expressed during testing.

Three associations form during conditioning: (1) CS and US (direct association), (2) CS and Context (the CS occurs in an environment), and (3) Context and US (the US also occurs in that environment).

Stimuli other than the CS are comparator stimuli. When the CS is presented during testing, it elicits a representation of the US (direct path) and a representation of the context (because they were paired). The context representation then elicits its own representation of the US (indirect path).

The memory of the US elicited by the CS is direct; the memory of the US elicited by context is indirect. The CR depends on comparing these two pathways. If the CS predicts the US much better than context does, strong CR. If context predicts the US nearly as well as the CS, weak CR.

Spaced Versus Massed Trials

The comparator hypothesis makes unique predictions about trial spacing. Yin, Barnet, & Miller (1994) administered classical fear conditioning to two groups with the same number of CS-US pairings. One group received massed trials (very little time between trials); the other received spaced trials (time elapsed between trials).

Both groups form equivalent and strong CS-US associations. CS-context associations are similar for both. But the spaced-trials group experiences context without US between trials, forming context-no US associations. The massed-trials group doesn’t experience context without US, maintaining strong context-US associations.

Prediction: Spaced-trials animals should show stronger CRs because the CS is a better predictor relative to context. For massed-trials animals, context also strongly predicts US, reducing the CS’s relative advantage.

Results confirmed this prediction—spaced trials produced stronger CRs. The Rescorla-Wagner model struggles with this finding because CS-US associations are equivalent for both groups. The comparator hypothesis succeeds by focusing on relative rather than absolute predictiveness (Yin et al., 1994).

Physiological Mechanisms of Classical Conditioning

Research with Aplysia

Eric Kandel’s Nobel Prize-winning research used Aplysia (sea slugs) to identify cellular mechanisms. In primitive creatures, weak stimulation of neuron 1 or neuron 2 caused them to fire. Before conditioning, neuron 1 firing didn’t cause neuron 3 to fire (though neuron 3 showed weak polarization changes). Neuron 2 firing caused neuron 3 to fire—this was the unconditioned reflex.

Conditioning procedure: Firing neuron 1 (CS) followed by firing neuron 2 (US) led to neuron 3 firing (UR). After repeated pairings, firing neuron 1 alone caused neuron 3 to fire (CR). This connection lasted about 20 minutes. Conditioning required neuron 1 before neuron 2 and a short interval between them—just like behavioral conditioning!

Kandel found the exact opposite chemical changes from habituation. Classical conditioning created increased neurotransmitter release from the sensory neuron—more chemical communication strengthened the synaptic connection (Kandel, 2001).

Research with Mammals

Mammalian research reveals greater complexity. Neural pathways for CRs often differ from those for URs—the CR isn’t just the UR pathway activated by a new stimulus; it involves distinct circuits. Many brain structures contribute to simple CRs; classical conditioning isn’t localized to one region but distributed across multiple areas. Different conditioning phenomena involve different brain locations; acquisition, extinction, and blocking recruit different neural circuits. Different CRs involve different brain locations; eyeblink conditioning relies heavily on cerebellum, fear conditioning depends on amygdala, and taste aversion involves insular cortex. Individual neurons have been identified whose activity correlates with CR acquisition; these “conditioning neurons” change firing patterns as learning progresses.

Human Research

Human results parallel other mammals. The cerebellum plays crucial roles in eyeblink conditioning—patients with cerebellar damage can’t acquire eyeblink CRs. Different types of conditioning activate different brain regions, revealed through fMRI and PET imaging. The amygdala is critical for fear conditioning; the hippocampus for contextual conditioning. This distributed organization reflects conditioning’s complexity and ecological importance.

Learning in the Real World: Phobias & Treatment

Classical conditioning plays important roles in phobia development. Imagine being trapped in an abandoned refrigerator as a child. Being shut inside (CS) resulted in oxygen deprivation (US), producing intense fear (UR). This experience can create lasting claustrophobia—intense, irrational fear of enclosed spaces (CR).

Flooding Treatment

Flooding is a technique for treating phobias by presenting patients with highly feared cues that aren’t removed until fear subsides. For claustrophobia, patients might be shut inside a small closet resembling the original experience. This produces great fear, but patients don’t experience oxygen deprivation (no US). When patients exit and feel relief (fear subsides), extinction proceeds and phobias diminish (Marks, 1987).

Systematic Desensitization

Systematic desensitization is a behavioral technique based on classical conditioning that treats phobias by combining relaxation training with gradual exposure to phobia-related stimuli. Patients begin with least-threatening situations, progressing to more intense exposures only when completely comfortable at each level (Wolpe, 1958).

For claustrophobia, patients might start in a large room (20 by 20 feet) for 10 minutes. Only when anxiety-free do they progress to smaller rooms, eventually reaching closet-sized spaces. The key is advancing only when the current situation produces no anxiety.

The Renewal Problem

Spontaneous recovery means phobias can regain strength over time even after successful extinction. The renewal effect poses even greater challenges. If extinction occurs in a clinical context but the phobia developed elsewhere (like a backyard for spider phobias), the CR returns when patients encounter the original context.

Treatment strategies address renewal. First, conduct extinction where patients will encounter feared stimuli; instead of extinguishing spider fear in a laboratory, conduct treatment in patients’ homes where they’ll actually encounter spiders. Second, extinguish fear in multiple contexts, assuming effects will generalize to new situations; if fear is extinguished in clinic, home, and workplace, patients are better protected against renewal in novel contexts.

Understanding renewal as context-dependent memory retrieval (occasion setting) explains why phobia treatment requires careful attention to environmental contexts where learning and testing occur.

Looking Forward

We’ve examined three theoretical accounts of classical conditioning: Pavlov’s stimulus substitution theory proposing neural connections between brain centers, the Rescorla-Wagner model emphasizing prediction errors and surprise as drivers of learning, and the comparator hypothesis focusing on relative rather than absolute predictiveness. Each theory offers insights while facing limitations explaining all conditioning phenomena. We’ve explored physiological mechanisms from Kandel’s work with Aplysia revealing synaptic changes to mammalian research showing distributed brain systems supporting different types of conditioning. We’ve seen how understanding conditioning principles enables effective phobia treatments, though renewal effects remind us that extinction learning is context-dependent. This completes our exploration of classical conditioning—from Pavlov’s accidental discovery of psychic secretions through taste aversion challenges, cue competition effects, drug tolerance, sophisticated theoretical models, and clinical applications, we’ve seen how a simple form of learning reveals deep principles about how organisms predict and prepare for their environments. These principles form the foundation for understanding all associative learning.