At a meeting, a team hears that a project has a seventy percent chance of meeting its deadline. The estimate sounds precise. Yet one person based it on a detailed schedule, another on memories of similar projects, and a third on a colleague’s confidence. Before the team chooses what to do, it has a judgment problem: how likely is the outcome, what evidence bears on it, and how much confidence is warranted? Choice will come later. Judgment is the work of forming an assessment that a choice might use.
What judgment means
In the psychology of judgment and decision making, judgment is an assessment, estimate, classification, or prediction concerning a state of the world. It may be explicit, as in an assigned probability, or implicit in saying that a plan seems safe, a person appears reliable, or one explanation fits better than another. A judgment can concern a current fact, a past event, or a future outcome. It can be expressed in a number, category, ranking, or ordinary language. The form matters because it shapes how easily the assessment can be examined or compared with evidence.
Judgment is not identical to decision-making. A decision combines beliefs about possible outcomes with preferences, obligations, constraints, and an action. Two people can agree that a proposed treatment has a particular chance of benefit yet choose differently because they weigh costs differently. Conversely, two people can choose the same option while holding different estimates of what will happen. Fischhoff and Broomell’s review separates judgment about consequences, preference over consequences, and choice that combines the two. That distinction prevents the common mistake of calling every disagreement about action a disagreement about facts.
Judgment is also not the same as knowledge. It often occurs where evidence is incomplete, costly to gather, or difficult to interpret. A claim may turn out to be true without having been well justified at the time, and a careful probability estimate may be reasonable even when the less likely outcome occurs. To evaluate a judgment fairly, ask what question it answered, what information was available, and what standard of accuracy applied. Hindsight alone can make a sensible estimate look foolish.
Questions come before answers
A judgment starts with a target. “Will this work?” is too vague to assess well. Work for whom, over what interval, under which conditions, and by which measure? A project judged successful by its launch date may be judged unsuccessful by durability or accessibility. The target should be clear enough that different observers know what evidence would count. This is a practical implication of decision-science research, not a claim that every human judgment can be reduced to a single metric.
A person can judge frequency, probability, magnitude, category membership, cause, or quality. These tasks require different evidence. A frequency estimate asks how often something occurs in a defined set. A probability estimate asks how uncertain a particular outcome is. A causal judgment asks what would have happened under a different condition. An evaluative judgment may also contain a value standard, such as fairness or usefulness. Calling all of these “intuition” obscures the question being answered and the possibility that some parts are empirical while others are ethical.
Framing can alter what people see as the question. Tversky and Kahneman showed that descriptions of equivalent outcomes can change preferences, including when outcomes are presented as gains or losses. Their work concerns choice as well as judgment, but its relevance here is that the representation of consequences and probabilities is not neutral. The wording of a question may direct attention to one reference point. A careful judge checks whether a changed description is adding information or merely making one aspect more salient.
How people form judgments under uncertainty
People rarely calculate every possible explanation or outcome from first principles. They use memory, patterns, rules of thumb, expertise, statistical information, and the testimony of others. Tversky and Kahneman’s classic analysis described representativeness, availability, and adjustment from anchors as economical ways to judge under uncertainty. Representativeness uses resemblance to a familiar type; availability uses ease of recalling examples; anchoring begins with an initial value and adjusts. Such processes can be useful, but they can also produce systematic errors when the cue is a poor guide to the target.
Suppose a recent failure is vivid in a manager’s mind. Its availability may raise the manager’s estimate of a new project’s failure risk, even if the wider record shows a lower rate. The problem is not memory itself. Examples can be valuable warnings. The problem arises when vividness substitutes for a relevant denominator, or when the manager cannot say how similar the old and new situations are. A base rate, when available and appropriately matched, gives the example a context.
An initial number can anchor an estimate even if it was selected for reasons unrelated to the phenomenon. Yet not every starting point is arbitrary. A historical average or well-grounded model can be a useful reference. The right question is where the number came from, how closely it matches the present case, and whether adjustment was adequate. A slogan such as “ignore anchors” offers little help when some anchors carry real information.
Gigerenzer and Gaissmaier’s review cautions against treating simple heuristics as necessarily inferior to complex calculation. Where information is uncertain, samples are small, or the environment has a useful structure, a rule that ignores some data can perform well. Whether a heuristic helps depends on the task and environment. The research therefore supports testing strategies against outcomes rather than assuming that more information or more computation always improves judgment.
Expertise and its boundaries
Experienced people can recognize informative patterns quickly. Fast judgment is not automatically careless. Kahneman and Klein, writing from traditions often presented as opposed, agreed that skilled intuition is more credible where the environment contains learnable regularities and the judge has had sufficient opportunity for clear feedback. Expertise should be assessed in relation to a domain and its feedback, not conferred by a title alone. A person may be expert in one kind of case while encountering a genuinely new situation in another.
Consider a technician who has repeatedly seen a distinctive equipment fault and learned which signals precede it. Their rapid assessment may condense years of useful experience. The same technician’s intuition about a rare organizational failure may have little comparable training history. This does not demean expertise; it asks where the expertise came from. Frequency of exposure, validity of cues, quality of feedback, and changes in the environment all matter.
Confident language is not evidence of expertise. Confidence can be warranted, but it can also reflect familiarity with a story, social status, or the pressure to sound decisive. The study of metacognition treats confidence as a judgment about one’s own performance or correctness. Confidence and accuracy can be related without being identical. An expert’s credibility grows when estimates can be checked against outcomes under comparable conditions, not merely when explanations sound fluent after the fact.
Institutions can make this harder. A prediction may be made collectively, revised informally, and never compared with the eventual outcome. The lack of feedback then protects weak practices from scrutiny. Equally, a noisy outcome can punish a good estimate. Good evaluation requires many appropriately defined cases and attention to uncertainty, not a single dramatic success or failure.
Judgment in a world of models and tools
Formal models can combine indicators consistently and can provide a baseline against which human estimates are compared. A meta-analysis by Grove and colleagues, in studies of health and behavior, found that mechanical data-combination methods were on average more accurate than clinical combinations in the cases examined. It did not establish that any software system is automatically reliable, or that human interpretation should be removed from all settings. Model quality depends on the target, data, validation, and how the output is used.
Human judgment remains important in defining the question, examining whether the data represent the present case, noticing constraints outside a model, and deciding what evidence could change the estimate. It is possible to rely too heavily on a tool, but it is also possible to reject useful structure merely because it is impersonal. The contrast between “human intuition” and “machine objectivity” is too coarse. A person can use a simple checklist carefully; a complex model can encode a poor target or be deployed in a changed environment.
Generative AI adds another layer: fluent explanations and estimates can be produced faster than their evidential basis can be inspected. The Institute’s synthesis is that AI-supported work needs a visible distinction between generated suggestions, human assessments of their evidence, and the eventual choice. That synthesis is a design proposal, not an effect established by the broad judgment literature. Its effectiveness would need task-specific testing rather than inference from the speed or fluency of a system’s output.
A useful question for any aid is what part of the process it supports. Does it retrieve facts, estimate a probability, flag an inconsistency, express uncertainty, or recommend an action? These functions are not interchangeable. If a system supplies a forecast, a human still needs to understand its reference class and limits. If it recommends a choice, the underlying values and trade-offs need separate scrutiny. Keeping judgment and choice distinct makes tool accountability clearer.
Assessing the quality of a judgment
Accuracy is one criterion, but its meaning depends on the task. A categorical claim can be checked against a later fact when a reliable outcome is available. A probabilistic claim needs evaluation across multiple comparable predictions; one event occurring does not prove that a seventy percent estimate was too low or too high. An estimate may be well calibrated when events given a particular probability occur at about that frequency over many cases. It may also be useful for distinguishing high-risk from low-risk cases. Calibration and discrimination are different properties.
Good judgment also needs an evidential trail. What observations were considered? Which were missing? Was a comparison class available? Were alternative explanations named? A judgment can be revised responsibly when new evidence arrives without making the earlier estimate fraudulent. The ability to update is a strength if the reasons are explicit. Refusing to update because a position has become public can degrade the estimate even while it creates an appearance of consistency.
Some outcomes arrive late or cannot be observed cleanly. Counterfactuals are especially hard: if an institution took action after a warning, the feared event may not happen, but that does not prove the warning was mistaken. Evaluating the judgment then requires more than matching a prediction with an outcome. The assessment may require a model of what would have happened without the intervention, and such models carry uncertainty. This is why outcome-based scoring is valuable but not exhaustive.
Where people disagree, separating factual assessment from preference clarifies the dispute. One participant may think an event unlikely; another may agree on the probability but regard the possible harm as unacceptable. A third may distrust the evidence source. Each disagreement calls for a different response. More data may help the first, a discussion of values the second, and source scrutiny the third. A single instruction to “use better judgment” hides these differences.
What the evidence does not show
Research on heuristics does not show that human judgment is uniformly irrational. The same shortcut can be effective or misleading depending on the environment. Research on expertise does not show that all intuition is reliable, nor that deliberation always improves accuracy. Mechanistic and laboratory findings should not be turned into universal rules for every profession or culture. The classic framing studies identify important effects under studied tasks, not an automatic forecast of any specific person’s choice.
The comparative evidence on statistical and human prediction does not establish that every deployed model is fair, current, transparent, or more accurate for a new population. It compares specified methods on specified tasks. The possibility of a useful model cannot settle which outcomes should matter or what costs are acceptable. Likewise, a person’s confidence does not measure correctness directly. Confidence can be investigated, but a confident statement is not an independent source.
No single observed outcome proves whether a probabilistic judgment was good. A seventy percent event can fail to happen without making the estimate logically inconsistent. Conversely, a true statement can have been a poorly supported guess. The broad literature does not validate a claim that AI automatically erodes judgment. Questions about AI assistance need task-specific evidence on accuracy, calibration, dependence, and consequences.
Keeping judgment open to correction
Judgment is the assessment placed between evidence and action. It can be fast or slow, personal or model-supported, expressed as a category or a probability. Its quality depends on a well-defined question, relevant evidence, an appropriate comparison, awareness of uncertainty, and opportunities for correction. The team at the meeting can improve its seventy percent estimate by asking what deadline counts, what similar projects did, and what information would change the figure. Only then is it ready to consider a choice.
That separation does not guarantee agreement or perfect forecasts. It does make disagreement more intelligible. When a prediction, a value, and an action are bundled into one declaration, errors are hard to find. When the assessment can be examined in its own right, people can challenge an assumption, revise a probability, or keep a sound estimate while choosing differently. Judgment is most useful not when it feels final, but when it is clear enough to be tested and changed.
Research references
- Fischhoff, B., & Broomell, S. B. (2020). Judgment and decision making. Annual Review of Psychology, 71, 331–355. View source
- Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131. View source
- Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211, 453–458. View source
- Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62, 451–482. View source
- Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64, 515–526. View source
- Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19–30. View source