A caseworker asks an AI tool to summarize a long application. The summary is clear and the recommendation looks plausible. The caseworker has ten minutes, three more files and a queue of people waiting. If the summary omits a qualifying circumstance, a person may lose a service. Saying that a human made the final decision does not tell us whether the human had time, evidence and authority to question the machine.
Decision hygiene for AI-assisted choices is the practice of arranging a decision so that important errors can be noticed before they become consequences. The term is an Institute synthesis, not a validated protocol. It includes defining the decision, checking the evidence, comparing alternatives, understanding when the AI is uncertain, recording reasons and preserving a route to correction. Good hygiene does not mean refusing useful automation. It means using assistance in a way that leaves judgment testable and accountable.
Begin with the decision, not the answer
An AI output can look complete before the decision problem has been specified. A manager may ask for a hiring shortlist without first deciding which skills matter, which evidence is permissible or what the role actually requires. A clinician may receive a diagnostic suggestion before hearing the patient’s account. A teacher may obtain a score before defining what the work was intended to show. In each setting, the output can become an anchor for a question that was never carefully framed.
A useful first step is to write the decision in ordinary language. What action is being considered? Who is affected? What would count as a good outcome, and what would make the choice reversible? Which facts are known, uncertain or disputed? The answer may reveal that the task is a search for missing information rather than a choice among fixed alternatives. AI can help retrieve or organize evidence, but it cannot decide which institutional responsibility disappears when an output is fluent.
The stakes should determine the procedure. A low-cost personal preference can tolerate a quick suggestion. A decision about employment, care, education or a public benefit needs more careful evidence handling and an appeal path. This is a proportionality principle, not a claim that every decision requires a committee. Time, attention and review capacity should be concentrated where errors matter most.
Reliance has two failure directions
People can accept an incorrect AI suggestion or reject a useful one. Research on appropriate reliance tries to distinguish these outcomes rather than ask whether users “trust AI” in general. Schemmer and colleagues proposed a two-dimensional way of measuring whether people follow correct advice and resist incorrect advice. Their experiment with 200 participants studied how explanations influence that pattern. The task and measure matter: a single overall agreement rate can conceal both kinds of mistake.
In a clinical simulation, Wang and colleagues asked clinicians to interpret knee imaging for anterior cruciate ligament rupture with and without AI assistance. The AI could help or mislead. Their study reported both improved diagnostic performance with assistance and a serious automation-bias problem when misleading advice influenced clinicians. The result concerns a specific imaging task and participant group. It neither proves that clinicians should abandon AI nor that a “human in the loop” label guarantees independent judgment.
Rejecting useful advice can also carry costs. A careful user might distrust a well-tested system because of an earlier bad encounter or because the interface hides its valid evidence. The target is calibration: knowing when the tool has relevant competence and when its output needs independent examination. Calibration depends on the task, the model’s performance in the relevant population, the quality of its information and what the user can observe. It is not a fixed personality trait.
Why a polished explanation is not a safety check
A system may explain a recommendation in smooth language even if its premise is wrong. The explanation can make a decision feel more justified without demonstrating that the underlying evidence supports it. Buçinca, Malaya and Gajos tested cognitive forcing functions designed to make users think before accepting AI advice. In their experimental context, such functions reduced overreliance compared with easier acceptance paths, but they also imposed costs in effort or speed. The point is not to make every screen laborious. It is to ensure that friction is applied where independent thought can change a consequential result.
For a summary of a benefits application, the interface could show each claim beside its source passage. It could highlight information the model did not process or ask the reviewer to state which eligibility rule applies before displaying a recommendation. These are possible design interventions, not universal solutions proven by the cited experiment. They need testing with real users, realistic workloads and actual error patterns. A source passage can also be misunderstood; mere display of a citation does not establish comprehension.
An explanation should answer a question relevant to the decision. “The model considered many factors” is not enough. “This conclusion depends on income during these dates, and the record for one month is missing” gives the reviewer something concrete to verify. Even that may be insufficient if the worker cannot obtain the missing record or challenge the system’s rule. Useful transparency joins evidence to the user’s authority and next action.
Critical thinking changes shape around generative AI
Lee and colleagues surveyed 319 knowledge workers about 936 examples of using generative AI at work. Higher confidence in the AI was associated with less self-reported critical-thinking effort, while greater task-specific self-confidence was associated with more. Respondents described critical thinking shifting toward verification, integration and stewardship of outputs. These are self-reports about experiences, not a randomized test proving that AI caused a decline in a measured cognitive skill.
The distinction matters because “AI makes people stop thinking” is too broad. A person may spend less effort drafting a routine sentence and more effort checking whether the result is accurate, fair and useful. Another person may accept an answer without examining its basis. The same tool can change different parts of a task in different ways. An evaluation should ask what thinking moved, what remained with the user and whether the final decision improved.
Verification itself can become superficial when the workload grows. A reviewer may open a cited page without reading it, or compare only the most prominent statement while missing an exception. An institution should make source checking feasible by allocating time and clear responsibility. Otherwise “verify the output” can become a ritual that transfers liability to the person with the least power to change the process.
Build a sequence that can catch mistakes
A workable decision sequence begins before the AI is consulted. Record the purpose, criteria and uncertainty. Then ask the tool for a bounded contribution, such as summarizing passages, proposing alternative interpretations or identifying information gaps. Compare that contribution with source material and at least one independent account where stakes justify it. Finally, document the decision’s reasons, residual uncertainty and route for review. This sequence is Institute synthesis drawn from research on reliance and risk management; it should be adapted and tested for each domain.
One useful question is whether the answer would change if the AI recommendation were hidden. A reviewer can make a preliminary assessment before seeing the output, then compare. That is not always efficient or possible, and it can create redundant work. It may be appropriate for a sample of high-risk cases or for quality audits rather than every low-stakes choice. The design should compare the cost of extra review with the cost of errors, including harms borne by people outside the organization.
Another question is whether the user can describe a credible alternative. If an AI-ranked list contains only one plausible candidate, a decision-maker may feel that alternatives do not exist. Requiring a second option may expose a hidden assumption, but a forced alternative can become empty paperwork. The goal is to examine the choice set: who was excluded by data, policy, access or the initial prompt? A diverse option set is meaningful only if it is grounded in valid evidence and genuine authority to choose.
For teams, the record should show who supplied data, who checked it, who decided and who can correct the outcome. NIST’s voluntary AI Risk Management Framework distinguishes governance, mapping, measuring and managing risks and calls for defined human-AI roles. A policy can therefore specify review thresholds and escalation paths before a case arrives. That preparation is more reliable than assuming a busy professional will invent an oversight system in the moment.
Accountability cannot be outsourced to a prompt
An organization may say the human remains responsible while providing no training, no time to inspect evidence and no authority to depart from the tool’s recommendation. That is nominal oversight. Meaningful oversight requires a person to understand the task, access relevant information, recognize uncertainty and make a correction that the institution will honor. It also requires governance of the AI system itself: validation, monitoring and a way to report failures.
The NIST framework is a risk-management resource, not proof that adopting its vocabulary makes a system safe. The application matters. A public agency could define when an AI summary may be used, how source records are linked, what kinds of cases require specialist review and how applicants can contest mistakes. A school could decide that an AI suggestion may help a teacher prepare feedback but cannot be the sole basis for a student’s consequential grade. These are policy choices requiring legal and local review, not facts demonstrated by the framework alone.
Documentation should be usable. An audit log that records only “human approved” cannot reveal whether the person saw the decisive evidence or simply clicked through. A better record distinguishes model version, input sources, recommendation, reviewer action, reason for a departure and later correction. Yet more documentation can burden workers and expose private information. The design must collect what is needed for accountability, protect it appropriately and test whether the record actually helps identify errors.
Moral and organizational context matter
Not every AI-assisted choice is a factual classification problem. Some involve competing interests, fair treatment and obligations to people who are not present. Kouchaki and Smith’s review of moral decision-making in organizations emphasizes that such decisions unfold through time and in institutional settings. A narrow accuracy score can miss whether the organization chose the right objective in the first place.
Suppose an AI scheduling tool reduces average waiting time by shifting inconvenient appointments to people who have fewer options to complain. The output may be statistically accurate about attendance yet morally poor because the objective ignores unequal burdens. Decision hygiene must therefore ask whose perspective shaped the target. A human reviewer may need to question the policy rather than merely check the prediction. This is an Institute application of organizational ethics, not an empirical finding about that specific fictional scheduling system.
Participation from affected people can reveal errors that a technical audit misses: confusing notices, inaccessible appeal routes, missing context and practical consequences. Their accounts should not be treated as optional decoration after deployment. At the same time, participation alone does not validate a model. It should sit beside quantitative performance tests, subgroup analysis and institutional responsibility for repair.
Evaluate the human-AI decision, not only the model
A model can be accurate on a benchmark while a combined human-AI workflow performs poorly. Users may accept wrong answers, disregard correct ones, misunderstand explanations or face severe time pressure. Evaluate the whole process against a relevant comparison: human alone, AI alone where appropriate, and several forms of assistance. Measure correct decisions and errors, but also time, uncertainty, disagreement, reasons for reversal and unequal effects across groups.
Test with realistic cases, including exceptions. A system that performs well on common examples may fail when a person has an unusual history or incomplete documentation. A useful evaluation distinguishes missing data, contested evidence and model uncertainty. It should report how many cases were escalated and what happened afterward. If the system improves speed but leaves high-impact errors uncorrected, the tradeoff cannot be hidden inside an average score.
Review should continue after launch. Inputs, policy, language and user behavior change. People may learn to accept the tool more readily, or to work around it. A system trained on one population may be used in another. Monitoring needs a route for affected people and staff to surface mistakes, plus authority to pause or change the workflow. No single prelaunch test can establish permanent reliability.
What the evidence does not show
The knowledge-worker survey does not measure a general loss of intelligence or prove that generative AI causes weaker critical thinking. The cognitive-forcing experiment does not establish that adding delay is always beneficial. The clinical imaging study does not generalize automatically to every specialty, tool or practitioner. The appropriate-reliance experiment offers a useful measure, but one task cannot settle what all users should do with all AI advice.
NIST’s framework is guidance for managing risk, not a certification of a particular deployment. Moral-decision research cannot supply a mechanical answer to competing values. Even a human review step may fail if workers lack time, information or a genuine right to disagree. A clean record of decisions is evidence about process only if it captures the work that actually happened.
The Institute’s practical sequence is a proposal to be evaluated. Its value depends on whether it improves decisions for the people affected, under real workloads and with a functioning route to correction. Researchers and institutions should be prepared to revise it when a design that looks careful on paper proves unworkable in use.
Return to the application
The caseworker should be able to see which source passages support the AI summary, what is missing and which rule governs the case. The person applying should have a clear path to add context or challenge an error. Supervisors should know whether the allotted time makes such review possible. If the tool speeds up a decision but makes these questions harder to answer, it has not improved the decision as a whole.
Decision hygiene is an arrangement of attention and responsibility. It keeps evidence close to the choice, gives uncertainty a place in the record and makes correction possible after the first answer. AI can contribute speed and pattern finding. The human institution must still decide what counts as a justified action and bear responsibility for what follows.
Research references
- Lee, H.-P., Sarkar, A., Tankelevitch, L., et al. (2025). The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. CHI 2025, Article 1121. View source
- Schemmer, M., Kühl, N., Benz, C., Bartos, A., & Satzger, G. (2023). Appropriate reliance on AI advice: Conceptualization and the effect of explanations. Proceedings of IUI 2023, 410–422. View source
- Wang, D.-Y., et al. (2023). Artificial intelligence suppression as a strategy to mitigate artificial intelligence automation bias. Journal of the American Medical Informatics Association. View source
- Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1). View source
- National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). View source
- Kouchaki, M., & Smith, I. H. (2025). Moral decision-making in organizations. Annual Review of Organizational Psychology and Organizational Behavior, 12, 45–72. View source