A radiologist says she trusts an AI tool, then rejects its recommendation on a difficult scan. A colleague reports little trust but accepts nearly every suggestion during a busy shift. Which person has better calibrated trust? Neither quotation is enough. A trust rating describes an attitude under a particular question. A clinical decision reflects evidence, alternatives, time and responsibility. Calibration concerns whether reliance changes appropriately when the system is more or less likely to be right, and whether people can tell the difference in the cases they face.
Measuring that relationship requires more than asking whether someone likes an AI system. The researcher needs a defined task, a defensible account of system quality, opportunities for both warranted acceptance and warranted rejection, and measures that do not confuse confidence with performance. This entry treats calibrated trust as a construct-validity problem. It compares questionnaires, behavioral reliance and outcome-based tests, while keeping the ethical and institutional setting visible.
What exactly is being calibrated?
Lee and See’s influential review of trust in automation argues for appropriate reliance rather than the highest possible trust. A person should use an aid when its capabilities fit the task and withhold reliance when they do not. This is a relational idea: the right level of reliance depends on the system’s actual performance, the human’s own knowledge, the cost of different errors and the information available at the moment. A globally high or low trust score cannot encode all of those conditions.
Vereschak, Bailly and Caramiaux surveyed empirical methods used to study trust in AI-assisted decisions. Their work shows how much a result depends on the experimental protocol: the task, participant, AI behavior, timing of questions and behavioral measures. A study can report trust but ask a participant only once after a series of outcomes. Another can track repeated choices but never check whether those choices were warranted. The two findings are not interchangeable.
There is also a difference between trust in an AI model, trust in an interface and trust in the institution deploying it. A resident may believe that a model predicts well while doubting whether an agency will correct errors. A worker may regard the organization’s approval process as credible but find the model’s uncertainty display confusing. If a questionnaire simply asks “Do you trust the AI?”, respondents may answer different questions. A measurement plan should name the target and context before selecting an instrument.
Calibration is sometimes described as matching subjective confidence to system accuracy. That comparison can be useful, but it is not the same as appropriate reliance. A person may correctly estimate that a system is 80 percent accurate overall and still fail to detect the particular cases in which it errs. Conversely, a person may have an imperfect verbal estimate yet make good case-by-case choices using reliable cues. Researchers should distinguish global belief calibration, local error discrimination and behavioral reliance.
What a questionnaire can and cannot measure
Jian, Bisantz and Drury developed an empirically derived scale for trust in automated systems. Its vocabulary has been used widely because it gives researchers a consistent way to ask about perceived trustworthiness and related feelings. Such a scale measures a reported attitude under its item wording. It does not directly show whether a participant will accept a correct recommendation, reject an incorrect one or seek more evidence when stakes rise.
Recent validation work by Scharowski and colleagues is especially instructive. In a preregistered online study of autonomous-vehicle and chatbot scenarios, the authors evaluated the Trust between People and Automation questionnaire in AI contexts. Their analysis found that the originally proposed single-factor structure did not fit well and that a two-factor distinction between trust and distrust improved fit. That is evidence about the instrument’s structure in those scenarios, not a declaration that every possible trust scale must use two factors or that a good questionnaire score proves safe behavior.
A scale can be internally consistent while measuring the wrong target for a decision study. Participants may answer consistently about how reassuring a system feels, but the researcher may want to know whether they recognize failures in a specialized task. Convergent and discriminant validity matter: does the score relate to expected constructs without collapsing into satisfaction, familiarity or general optimism? The exact wording, language adaptation and timing should be reported. A translated item may change meaning even when the numeric response options remain the same.
When the question is a change in trust, researchers should ask when it occurred. A rating collected after a visible failure may reflect a reaction to that failure rather than a stable prior attitude. Repeated ratings can trace updating, but the questions themselves may prompt participants to inspect the aid more critically than they would in ordinary use. A one-item slider is easy to administer and hard to interpret. It can be useful as a momentary report if it is not treated as a validated diagnosis.
Behavioral measures need a truth table
A common behavioral measure is whether a person accepts the AI recommendation. Raw acceptance is not calibration. If the system is highly accurate on the presented cases, frequent acceptance may be prudent. If it is wrong on a minority of high-cost cases, the same acceptance rate may conceal serious over-reliance. The minimum useful table crosses system correctness with human acceptance or rejection, and it records the person’s final accuracy as well as the recommendation’s accuracy.
An initial judgment before AI exposure adds information. If a person begins correct, sees a wrong suggestion and changes to wrong, the aid caused a harmful switch in that trial. If a person begins wrong and accepts a correct suggestion, it helped. If both human and system are wrong, final agreement may tell us little about trust. A study that reports only the final team score hides these pathways. Bansal and colleagues’ work on human-AI team performance is a reminder that a team’s outcome can differ from either component’s standalone accuracy; explanations and collaboration design must be tested rather than assumed beneficial.
Researchers can calculate rates of appropriate acceptance and appropriate rejection, but a binary label still assumes a clear ground truth. Some cases have uncertain or contested outcomes, and some errors are more costly than others. A medical triage task, a public-benefit decision and a low-stakes classification exercise cannot be collapsed into one generic trust score without losing what counts as an appropriate choice. The scoring rule should be justified before looking at results.
One can also observe whether participants inspect source material, request a second opinion or revise after new evidence. These behaviors may indicate active oversight, but none is inherently virtuous. An extra click can be ceremonial, and repeated checking can be wasteful when the answer is straightforward. A behavioral measure should connect a process to decision quality and task constraints rather than rewarding visible skepticism for its own sake.
Reliance is not identical to trust. A worker may accept the model because policy requires it, because no alternative is available, or because rejecting it demands an explanation that the shift does not allow. A resident may refuse an accurate tool because they do not consent to its use. Both choices have reasons that a trust questionnaire might miss. Interview and workflow evidence can clarify interpretation without replacing the behavioral comparison.
Design a calibration task
A defensible task includes cases where the AI is right and cases where it is wrong, with error types that resemble those expected in deployment. It tells researchers how system reliability was determined and whether the distribution of cases matches the real setting. If the study oversamples dramatic failures to create measurable variation, that choice should be disclosed. It may make participants unusually wary and change the meaning of their behavior.
The human needs a real opportunity to discriminate. If the only information is a model score, rejecting a wrong recommendation may be a guess. If source documents are available but buried behind several screens, the task measures navigation and time as well as trust. Researchers should record what evidence the participant saw and what they were told about the model. The central question is whether the participant could reasonably make a better decision with the information provided.
System confidence displays deserve separate testing. A well-calibrated model probability is not automatically a well-understood signal to a human. People may read “80 percent confidence” as a promise about one case, a long-run frequency or a measure of the model’s internal certainty. The test should examine interpretation, not merely display the number. Appropriate reliance depends on what users understand about performance in the relevant subgroup and situation.
Repeated trials allow researchers to observe learning. A participant may start with a high trust rating, encounter several failures and adjust. Another may overreact to a conspicuous error and discard an otherwise useful tool. Order effects matter: early mistakes can anchor later use, while late mistakes may be discounted after many successes. A study should report the sequence and avoid presenting one end-of-task rating as though it described the entire interaction.
Task difficulty must be balanced. People are more likely to seek advice on difficult cases, which can make reliance appear associated with poor outcomes even when the advice helps. Randomizing advice availability across comparable cases or comparing initial and final decisions can clarify the effect. The study should also report whether participant expertise varies and whether some cases exceed the expertise expected in the deployed role.
Confounds that mimic calibration
A participant who rejects every recommendation from an inaccurate model can look well calibrated in a deliberately hostile test. That behavior may fail when the model becomes useful. A participant who accepts every recommendation from an almost perfect model can also look good, but the design has not tested failure recognition. A meaningful measure needs both kinds of opportunity and enough examples to estimate each rate with uncertainty.
Demographic or contextual distribution shift is another confound. A model may perform well overall and poorly for a specific group. A global reliability number gives the human an inadequate basis for local reliance. Measuring calibration therefore requires evaluating model quality by the conditions that matter for the task. It is unfair to blame a user for failing to detect a systematic weakness that the organization concealed.
Interface design can change behavior while the underlying trust attitude stays similar. Defaults, phrasing, visual prominence and the effort required to see evidence all affect acceptance. Buçinca and colleagues tested cognitive forcing functions intended to reduce over-reliance in AI-assisted decisions. Such interventions may improve some choices but add effort or frustrate users. An evaluation needs both decision outcomes and burden. More friction is not automatically better judgment.
Social desirability can distort self-report. A professional may know that “I always check the AI” is the expected answer even when the workflow makes that impossible. Anonymity, neutral wording and observed behavior can help, but the organization also has to make checking feasible. A study that publicly scores workers’ trust may measure defensiveness and surveillance response as much as the intended construct.
Finally, the word “trust” can invite moral judgments. A person who distrusts a tool after a bad experience may be responding rationally to local evidence. A person who uses it frequently may be protecting time for other work. The researcher should avoid treating the center of a scale as an ideal by definition. Calibration is a relation between beliefs, available evidence, system performance and chosen action, not a personality virtue.
Validity across settings and time
Construct validity asks whether the measure captures calibrated trust rather than generic approval, technology affinity or compliance. The strongest studies triangulate a stated belief, case-level reliance and the accuracy of both person and system. Each measure answers a different question. Disagreement among them is not necessarily a failure; it may reveal institutional constraints or a questionnaire that misses the decision mechanism.
Internal validity asks whether observed reliance changed because of the AI recommendation. An initial decision, random assignment and balanced items help. Ecological validity asks whether the task resembles the workplace or service in which the result will be applied. A quick online classification task can isolate mechanisms but not reproduce the accountability, appeal routes and fatigue of public administration. A field study sees those conditions but cannot easily isolate every cause.
Reliability matters at both instrument and task levels. A questionnaire should show an interpretable structure in the language and population where it is used. A behavioral task needs enough varied cases to distinguish a pattern from chance. Reporting a single person’s “calibration score” from two wrong-AI trials invites overinterpretation. Group-level differences can be informative while individual ranking remains unstable.
Fast-changing AI systems add a time problem. An update to the model, prompt, dataset or interface changes the reliability that users are asked to calibrate against. A measure validated last year in a chatbot scenario may not carry unchanged into a new professional workflow. Longitudinal studies should record version changes and consider whether users were informed. Trust cannot be calibrated to performance that neither researchers nor users have characterized.
A European oversight example
Article 14 of the EU AI Act addresses human oversight of high-risk AI systems and explicitly names the risk of automatic or excessive reliance. This gives European organizations a reason to study how people use recommendations, but the law does not prescribe one trust questionnaire or establish that a high score is desirable. Legal obligations and psychological measurement should be connected carefully, with the latest authoritative text reviewed for each deployment.
A Nordic agency evaluating an AI-assisted case review could compare human-only and assisted decisions on adjudicated cases, including errors that resemble known model weaknesses. It could ask staff how trustworthy the system seems, observe when they verify source facts, and document whether the organization allows disagreement. It should report correct and incorrect acceptance, correct and incorrect rejection, unequal error patterns and time costs. This is an Institute proposal for study design, not a validated regulatory checklist.
The people affected by decisions also have standing. A worker’s well-calibrated response cannot compensate for an invalid model, missing appeal process or opaque eligibility rule. An applicant may need a route to contest an outcome even if the average human-AI team performs well. Measuring reliance should support accountability rather than convert it into a psychological burden placed solely on the last person to click.
What the evidence does not show
A questionnaire score does not prove that a user will rely appropriately on AI. A high acceptance rate does not prove over-trust, and a low acceptance rate does not prove wisdom. These interpretations require system accuracy, case difficulty, available evidence and a comparison of outcomes. The 2025 validation findings for one trust questionnaire in two AI scenarios do not automatically invalidate every older use of the scale or establish one universal replacement.
Neither the classic trust-in-automation review nor the AI-methods survey provides a single threshold at which trust becomes “calibrated” for all tasks. A cognitive forcing study does not show that adding friction always helps, especially when delays or administrative work have costs. Human-AI team performance in experimental tasks does not certify a public institution’s deployment. Each step from laboratory measure to policy conclusion needs its own evidence.
The Institute’s proposed truth table and evaluation questions synthesize the cited measurement literature. They are not a validated instrument. A local project must define ground truth, error costs and the rights of affected people; preregister its outcome where feasible; and revise the design when the model or workflow changes. If there is no feasible path to independent evidence, the task may not support meaningful calibrated reliance at all.
Return to the two radiologists
The radiologist who rejects one AI suggestion may be accurately identifying a model failure. She may also be discarding useful advice because an earlier error remains vivid. Her colleague may accept good recommendations efficiently, or may be following a workflow that leaves no time to check them. A stated level of trust cannot settle those alternatives. The relevant record contains the case, the system’s performance, the human’s prior judgment, what evidence was available and the final outcome.
Calibrated trust is best treated as a disciplined relationship among knowledge, action and accountability. Good measurement makes the relationship visible without asking one number to stand for a person. It helps institutions learn when an AI system earns reliance, when it does not, and what people need in order to tell the difference.
Research references
- Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. View source
- Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53–71. View source
- Vereschak, O., Bailly, G., & Caramiaux, B. (2021). How to evaluate trust in AI-assisted decision making? A survey of empirical methodologies. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2), Article 327. View source
- Scharowski, N., et al. (2025). To trust or distrust AI: A questionnaire validation study. Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. View source
- Bansal, G., et al. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. View source
- Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1). View source
- European Parliament and Council. (2024, consolidated 2026). Regulation (EU) 2024/1689, Article 14: Human oversight. View source