A participant chooses a symbol that usually pays ten points. This time it pays nothing. The result is disappointing, but disappointment alone is not the quantity a reward-learning model calls prediction error. The difference depends on what the participant expected immediately before the outcome, how that expectation was learned, and whether ten points had value to this participant. A task that records only the outcome cannot recover all of those ingredients.
Measuring reward prediction error means estimating a mismatch between received and expected reward under an explicit model, then asking whether behavior or a physiological signal tracks that estimate. It is distinct from measuring pleasure, the vigor with which someone pursues a cue, or a general reaction to surprise. This methods guide explains task design, model fitting, neural interpretations and common confounds. Its aim is a defensible claim about a specific learning signal, not a single biomarker of reward sensitivity.
Define the error before estimating it
In a simple learning model, an outcome is compared with a current estimate of value. An unexpectedly good outcome produces a positive signed error; an unexpectedly bad one produces a negative signed error. If the outcome matches the estimate, the error is near zero even if the reward is pleasant. The model can use that error to update future value. The exact equation, value scale and timing must be stated because different models can assign different errors to the same trial.
Schultz, Dayan and Montague connected formal reward-learning ideas with phasic dopamine responses in influential work. Later recordings by Bayer and Glimcher tested quantitative relationships between expected and received reward in primate midbrain neurons. These studies provide strong motivation for measuring error signals, but they do not imply that every dopamine response in every context equals the same scalar variable. The task, organism and recording method delimit the claim.
A researcher must distinguish prediction of reward occurrence from prediction of magnitude, probability and timing. A task can violate one while preserving another. Receiving the expected amount later than expected may generate a timing-related response. Receiving an expected reward in an unusual sensory form may create surprise without changing reward value. The trial schedule should make the intended dimension independently identifiable.
Value is not always equal to nominal points. Ten points may matter more near a threshold or when they can be exchanged for money. Fatigue, satiety and instructions can alter the subjective scale. If a model uses objective point values, report that choice and test whether it fits the participant’s decisions. Calling a residual “reward prediction error” does not make its input value valid.
Design a task with learnable expectations
The most common design asks participants to choose among options with probabilistic outcomes. Reward contingencies may be stable, reverse, or drift over time. To infer trial-by-trial expectation, participants need enough experience to form beliefs, while the researcher needs enough variation to separate expected value from actual outcome. If every good outcome follows the same symbol and every bad outcome another, reward, familiarity and motor response can become confounded.
Include trials where the same outcome arrives under different expectations. Five points after an option that usually pays one point is different from five after an option that usually pays nine. That comparison is closer to the construct than comparing five points with zero while ignoring expectation. A design can also vary probability while holding outcome size fixed. Such contrasts help distinguish signed error from outcome magnitude and from unsigned surprise.
Randomization alone is insufficient. Participants may infer patterns the experimenter did not intend, or may not learn the intended contingencies. A manipulation check can ask for expectations periodically, but frequent questions can change learning and attention. Sparse ratings or an independent test phase can help. The study should explain whether the fitted expectations are constrained by choices, explicit reports or both.
Reversal tasks are informative but complicated. When the better option changes, errors may reflect surprise, exploration, frustration and a shift in attention. A participant can keep choosing an old option because of habit, uncertainty or low learning rate. An estimated negative error on a reversal trial is therefore a model quantity, not a direct report of felt loss.
Model fitting makes the estimate conditional
A reinforcement-learning model typically includes parameters such as learning rate and decision noise. The learning rate determines how much each error updates value. Decision noise describes how tightly choices follow estimated values. Different combinations can produce similar behavior, especially with few trials or weak reward differences. Fit quality, parameter recovery and model comparison are part of the measurement argument.
Researchers should simulate data from plausible parameter values, fit their proposed model, and see whether the parameters can be recovered under the actual task design. If two models produce nearly identical choices, a neural correlation with one model’s errors cannot settle which mechanism generated behavior. Alternative models may include different learning rates for positive and negative outcomes, changing states, or explicit inference about reversals. Choose rivals before using a signal to favor the preferred account.
A model can fit group choices well while assigning unstable individual errors. Trialwise estimates may be noisy for participants who make few informative choices. Hierarchical fitting can improve estimation but introduces assumptions about the group distribution. Report those assumptions, uncertainty and sensitivity to priors or parameter bounds. A single best-fit trace can make a fragile inference look precise.
Signed and unsigned quantities deserve separate predictors. A signed error distinguishes better from worse than expected. Absolute error describes surprise or salience regardless of direction. In some tasks, unexpected losses and unexpected gains both capture attention. If a signal rises for both, calling it a signed reward prediction error would be misleading. Test whether the analysis can resolve their contributions.
Another identification problem is that the fitted error can be strongly correlated with the outcome itself. If high-value rewards are usually surprising and low-value outcomes are usually expected, a response to reward magnitude will resemble a response to prediction error. The design should include the same outcome after different cue-based expectations and, where feasible, the same expected value followed by different outcomes. Analyze whether outcome magnitude, expected value and signed error can be estimated separately in the actual trial schedule, not merely in the model’s equations. O’Doherty and colleagues’ temporal-difference analysis illustrates the value of specifying cue and outcome events; Bayer and Glimcher’s quantitative tests likewise depend on variation in expectation and receipt. If the regressors remain nearly inseparable, report a reward-related response rather than assigning it uniquely to an error computation.
Neural and physiological readouts
Single-neuron recording can reveal activity at the time of cues and outcomes with high temporal resolution. Bayer and Glimcher’s primate data are especially relevant to quantitative error claims. But even a close correlation between firing and a model regressor is conditioned on the task and on how neurons were selected. Population diversity and context dependence matter. A study should report how many neurons or recording sites were analyzed and how the error predictor was estimated.
Human fMRI often uses model-derived errors as trialwise regressors. O’Doherty and colleagues examined temporal-difference-like signals in human reward learning, offering an example of connecting computational estimates to brain activity. The blood-oxygen-level-dependent response is slow and indirect. Cue, action and outcome signals can overlap in time, especially with short intervals. A fitted regressor in striatum is evidence of covariation under the model, not a direct measurement of dopamine release.
Pharmacological or stimulation manipulations can strengthen causal questions but introduce their own breadth. A drug that affects dopamine may alter motivation, movement, attention and learning simultaneously. Pessiglione and colleagues studied dopamine-dependent prediction errors and reward seeking in humans, yet their results do not make every behavioral difference under dopamine manipulation an error-signal effect. The interpretation depends on the contrast and model.
Peripheral measures such as pupil dilation and skin conductance can respond to unexpected outcomes, but they often reflect arousal or orienting rather than signed reward value. They are useful when the construct is surprise or attention. If used for reward prediction error, the study needs contrasts that separate positive from negative mismatch and reward from nonreward surprise.
Distinguish neighboring reward processes
Hedonic impact concerns the pleasure or displeasure of an outcome. Incentive salience concerns the motivational pull a cue can acquire. Reward prediction error concerns a mismatch that can update expectations. These processes interact but should not be treated as interchangeable. An outcome can be highly pleasurable and entirely expected, producing little error. A cue can powerfully attract behavior even when the later outcome is less pleasurable than anticipated.
Zald and Treadway’s review of reward processing and psychopathology warns against collapsing valuation, anticipation, learning and hedonics into a single hypo- or hyperreward label. That framework is useful here because a clinical group difference on one task may arise from altered expectations, outcome valuation, exploration or choice noise. A model’s error regressor cannot identify the clinical mechanism unless those alternatives are measured.
Subjective happiness can also respond to expectations and recent errors. Rutledge and colleagues modeled momentary well-being in a probabilistic reward task, showing why an emotional report is related to but not identical with a learning signal. A person’s rating after an outcome may integrate recent history, current earnings and expectations. It should not be used as a direct substitute for trialwise reward prediction error.
Prediction error itself can refer to other domains, including sensory and interoceptive mismatches. A surprising sound or bodily sensation is not automatically a reward error. The word “prediction” is shared, but the predicted variable and update rule differ. A good article or study names the domain every time the distinction matters.
Reporting choices that expose confounds
Report the complete task schedule: probabilities, magnitudes, cue timing, response window, outcome delay and any reversals. State the model equation and how initial values were set. Show whether participants learned the contingencies and whether model simulations reproduce choices. Include at least one plausible rival model, and report parameter and trialwise-estimate uncertainty.
For neural analyses, show how cue, action and outcome regressors were separated. Explain preprocessing, region selection and correction for multiple comparisons. If the region was selected using the same contrast later interpreted, note the circularity risk. Report whether the signal tracks signed error, unsigned surprise, value, outcome magnitude or several correlated predictors.
When comparing groups or interventions, test whether the measurement model operates similarly. Different choice strategies can make fitted learning rates incomparable. A group may explore more without having a different error computation. Posterior predictive checks and parameter recovery should be examined within each group, not only in the pooled sample.
Finally, phrase conclusions at the level supported. “This striatal response covaried with model-derived signed error in this probabilistic task” is stronger than “dopamine learning is impaired” when neurotransmitter activity was not directly recorded. A careful report can still be interesting. It tells the reader exactly what evidence would change the interpretation.
A useful robustness check is to repeat the analysis under reasonable alternative assumptions about initial value and learning rate. If a neural result exists only under one fragile setting, the claim should be described as model sensitive. If several plausible models yield similar trialwise errors, the data may support a broad mismatch interpretation but not a unique computational mechanism. Report both possibilities. The same discipline applies when task instructions are changed: a participant told that outcomes are random may form different expectations from one told that symbols are learnable. The instruction is part of the measurement situation, not a trivial preface.
What the evidence does not show
The cited work does not establish a unitary reward prediction error biomarker across species, tasks and recording methods. A model-derived error is only as valid as its assumptions about expectation and value. A BOLD response does not directly measure dopamine, and a pupil change does not identify the sign of a reward error without a suitable contrast. Faster learning after unexpected reward does not prove which neural system computed the update.
Nor does a group difference in fitted parameters diagnose a person or locate a single cause of psychopathology. Learning, pleasure, effort, choice noise and incentive pull can all change reward-task behavior. The Annual Reviews framework supports keeping those components separate, while specific measurement claims require task-matched primary evidence. The methodological checklist here is Institute synthesis, not a validated clinical test.
Return to the missing points
The participant who expected ten points and received zero may have a large negative error under a simple model. But the experimenter must first know whether ten was genuinely expected, whether zero had the same meaning across participants, and whether the next choice changed in a way the model predicts. A neural signal would add evidence only after outcome, surprise and movement are separated.
Reward prediction error is powerful because it links a quantitative expectation to an observed outcome and a later update. Its measurement is strongest when each link is visible. That discipline keeps surprise, pleasure and wanting in view as neighboring processes without letting any one of them silently stand in for the error itself.
Research references
- Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599. View source.
- Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47(1), 129–141. View source.
- O’Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38(2), 329–337. View source.
- Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature, 442, 1042–1045. View source.
- Rutledge, R. B., Skandali, N., Dayan, P., & Dolan, R. J. (2014). A computational and neural model of momentary subjective well-being. PNAS, 111(33), 12252–12257. View source.
- Zald, D. H., & Treadway, M. T. (2017). Reward processing, neuroeconomics, and psychopathology. Annual Review of Clinical Psychology, 13, 471–495. View source.