Human-AI interaction

A caseworker asks an AI system to summarize a long application. The result is fluent and quick, but a missing paragraph changes the recommendation. The caseworker might notice, accept, correct or never see the omission. What happens next depends on the model, the interface, the time allowed, the evidence shown and the worker’s authority. Human-AI interaction is the study and design of those exchanges between people and AI systems across a task. It includes how people form expectations, give inputs, interpret outputs, act on errors and learn from repeated use. It is broader than prompt writing and narrower than every social consequence of automation. The central question is how a system and its setting shape what people can understand and do.

Expectations and the meaning of an output

People infer competence from presentation. A clear answer can feel more trustworthy than a hesitant one even when the underlying evidence is weaker. Conversely, a system that displays uncertainty may be rejected despite better calibration. Human-AI interaction research examines the match between actual performance and people’s reliance, rather than treating trust as a desirable quantity to maximize. If a tool is wrong on a case type, appropriate behavior may be to withhold reliance; if it performs well under known conditions, blanket rejection may waste a useful aid.

Explanation can help, but its purpose must be clear. A short reason may help a caseworker locate the source of a recommendation. A confidence score may signal uncertainty if it is calibrated for the task and understood by the user. A long technical trace may add work without enabling a decision. An explanation can even create a false sense of understanding when it is persuasive but incomplete. The test is whether it helps the person detect relevant errors and choose a sound action, not whether the interface contains explanatory words.

The same output has different meanings in different workflows. An AI-generated draft offered as a starting point is not the same as an automatically filed final decision. The caseworker’s ability to compare the summary with the application changes the interaction; so does a deadline that makes comparison impossible. Interfaces should be evaluated together with the work they require. A system can look transparent in a demo and still fail where staff are rushed or source documents are inaccessible.

Combination does not guarantee better performance

It is tempting to say that a human plus AI combines the best of both. Vaccaro, Almaatouq and Malone’s systematic review and meta-analysis compared human-only, AI-only and combined performance across experimental studies. Their results show heterogeneous outcomes rather than automatic synergy. The combination can help in some task conditions and perform worse than the stronger solo performer in others. The review calls attention to relative strengths and the interaction that lets one partner contribute where the other is weak.

That finding changes how a pilot should be designed. Comparing an AI-assisted team only with an unaided team may miss whether a well-tuned AI alone would have performed better. Comparing average accuracy alone may hide serious errors in rare cases. A meaningful evaluation identifies the alternative workflow, the outcome that matters, the cost of mistakes and the population represented. It also asks how users decide when to consult or override the tool.

Automation bias is one failure mode: a person may accept a wrong recommendation or miss information because the system did not flag it. The opposite, discarding a correct recommendation without examination, can also matter. Neither pattern should be inferred from one agreement rate. A person may agree because the evidence supports the AI, because the task is hard, because they lack time or because they overestimate the model. The study must expose the information and available choices before diagnosing the mechanism.

Interventions can create new costs

Buçinca, Malaya and Gajos tested cognitive forcing functions in AI-assisted decision tasks. Requiring people to engage with a case before seeing advice can reduce overreliance in particular conditions. The result does not mean every extra click or delay improves judgment. Friction can be useful when it creates a meaningful check and harmful when it consumes scarce attention without adding evidence. The person must have enough information, time and authority to use the pause.

Consider a workflow that hides the AI recommendation until the caseworker writes a preliminary view. This can protect independent reasoning, yet it may be burdensome for routine low-stakes cases. A different design might highlight where a summary omits text and allow direct comparison. Which design works depends on error rates, task difficulty and the worker’s knowledge. The right outcome is not maximum effort. It is better decisions with costs and risks made visible.

A good interface should also support recovery. If a user corrects an output, can the correction be retained for the current case? Is an audit trail clear enough for a reviewer? Can the user return to the source rather than only to the generated summary? Amershi and colleagues include feedback and graceful correction among their interaction concerns. These features must be tested in the real process. A correction affordance that is hidden or ignored is not effective control.

Generative systems change the cognitive task

With generative AI, work often shifts from producing a first draft to evaluating, adapting and verifying one. Lee and colleagues surveyed knowledge workers about critical-thinking effort in generative-AI-assisted tasks. They found associations involving confidence in the AI, confidence in one’s own ability and reported effort. The survey is about self-reported experience; it does not demonstrate that AI permanently weakens critical thinking or measure an individual’s actual skill decline. Its value is to identify how the nature of the task may change.

Someone who previously wrote a report from notes may now inspect a draft. That can free time for comparison and revision, or it can lead to superficial acceptance. The outcome depends on whether sources are visible, whether the person knows what to check and whether the organization rewards careful review. A fluent draft increases the amount of text available for evaluation; it does not automatically increase the amount of evidence. The interface should make provenance and uncertainty practical rather than decorative.

Generative systems also make authorship and responsibility harder to locate. If the caseworker edits the summary, who verifies the original extraction? If an AI suggests a reason for denying support, who owns the justification? The system can contribute text without bearing professional or legal responsibility. The human role must therefore be more than a signature after the fact. A responsible process gives the reviewer access to primary information and a real route to disagree, and records what changed because of that review.

People and systems adapt over time

Repeated use changes expectations. A worker may learn which cases the AI handles well and where it fails; they may also become less likely to inspect familiar-looking outputs. Model updates can change behavior without the interface changing. An evaluation performed once at launch may cease to describe the current tool or workforce. Monitoring should include errors, corrections, cases where users decline advice and instances where an output is accepted despite conflicting evidence.

Feedback loops can run beyond one user. If a system’s suggestions shape what staff record, later training data may reflect those suggestions. If a recommender changes what people see, their responses become evidence generated under the recommender’s influence. These possibilities are context-dependent; they should be investigated rather than assumed in every product. The important design question is whether the institution can detect when the interaction itself changes the data used to judge success.

Glickman and Sharot tested feedback loops in perceptual, emotional and social-judgment tasks. Their experiments show that biased AI judgments can shift human responses and, under their conditions, amplify bias through repeated interaction. This is stronger evidence for a mechanism than a mere anecdote about users trusting a machine. It is still not evidence that every deployed AI system produces the same loop. A public agency would need to examine its own model, users and decision history before attributing a changing pattern to feedback. The practical implication is to preserve independent outcome measures and inspect whether apparent improvements depend on behavior the system itself helped create.

Longitudinal evaluation should also record when people stop using a feature and why. Abandonment may indicate poor fit, an inaccessible interface or appropriate recognition of errors. Increased use may indicate benefit or organizational pressure. Neither adoption nor rejection is a simple proxy for quality. Studying the interaction means following what the person attempted, what the system supplied and what happened afterward. Those observations help prevent a popularity metric from standing in for a claim about human capability.

Access and power also shape learning. A novice may be required to use a tool but not taught how to challenge it. An expert may have a correction route that a customer does not. A disabled user may encounter an interface whose alternatives are difficult to navigate. Calling the system “human-centered” should mean testing these different positions. Averages across users can conceal the people with least opportunity to recover from an error.

What to evaluate in practice

Begin with the specific task and a credible comparison. For the application summary, measure whether important facts are preserved, how often users detect omissions and how long correction takes. Include realistic caseloads and the source documents people would actually have. Compare a human-only workflow, an AI-only output where appropriate, and the combined process. Measure the cost of severe mistakes separately from the average time saved.

Then study the interaction path. Where does the user’s attention go? What information is available before a decision? When the AI is wrong, do users notice and recover? When it is right, do they use it appropriately? A satisfaction survey can reveal frustration but cannot replace task outcomes. A log can show clicks but not why they occurred. Interviews, observations and controlled tests answer different parts of the question. The Institute’s proposal is to combine them around the consequential decision, not to worship one metric.

Finally, give affected people a route to contest outcomes. The applicant should not have to know which model produced a summary before asking for a correction. The caseworker should be able to trace the summary to source text. Managers should be able to suspend use if error patterns emerge. These are design and governance recommendations, not an empirical claim that one interface layout has been proven best everywhere.

What the evidence does not show

Research does not show that human-AI teams always outperform either partner alone. It does not show that trust should always rise, that every explanation improves decisions or that more friction necessarily reduces mistakes. Surveyed reductions in perceived effort do not establish cognitive decline. A laboratory benefit for one forcing function cannot be assumed in a public-service workflow with different stakes and time pressures.

The handoff’s Nature Psychology seed could not be confirmed from the accessible publisher record during this review; this entry uses verified, matched sources instead. Even those sources have limits: design guidelines are not universal effectiveness trials, a meta-analysis combines heterogeneous tasks, and self-report cannot establish behavior. The Institute’s workflow suggestions are synthesis for testing, not a certified standard or substitute for domain-specific law and professional judgment.

Return to the caseworker’s desk

The caseworker sees the omitted paragraph because the interface keeps the source beside the summary and the workflow allows time to compare them. They correct the draft and record why. That outcome is not proof that “AI worked” or that “humans stayed in control” in every case. It is one observable interaction that a team can evaluate and improve.

Human-AI interaction is about the quality of a relationship between people, systems and tasks. Its science asks when assistance changes performance and how design changes reliance. Its ethical demand is that people affected by a consequential output retain meaningful ways to understand, question and correct it. A good system leaves those routes visible when the answer looks most convincing.

Research references

Amershi, S., et al. (2019). Guidelines for human-AI interaction. Proceedings of CHI 2019. View source
Vaccaro, M., Almaatouq, A., & Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour, 8, 2293–2303. View source
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 188. View source
Lee, H.-P., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025). The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. Proceedings of CHI 2025. View source
Glickman, M., & Sharot, T. (2025). How human-AI feedback loops alter human perceptual, emotional and social judgements. Nature Human Behaviour, 9, 345–359. View source

More than a prompt and an answer