Two students finish the same problem set.
The first used a chatbot that supplied complete solutions whenever they became stuck.
The second used a system that responded with hints, asked for another attempt and refused to reveal the final answer too early.
Both submit more correct work than they would have produced alone.
A week later the tool is removed.
Now the educational question begins.
Generative AI has made an old distinction newly urgent: assisted performance is not the same outcome as durable learning.
Current evidence does not support one universal verdict on AI assistance. It shows that outcomes depend heavily on system design, task, learner, timing and what later assessment asks the learner to do.
The debate is therefore not well represented by “AI helps learning” versus “AI makes students lazy.”
A better scientific question is: Which parts of the learning process are being supported, which are being replaced, and what capacity remains when conditions change?
The strongest cautionary evidence: performance up, later performance down
A 2025 field experiment published in PNAS studied nearly 1,000 high-school mathematics students in Turkey.
Students were assigned to a control condition or to one of two GPT-4-based systems. GPT Base resembled a standard chat interface. GPT Tutor was engineered with teacher-provided solutions, common errors and prompts that encouraged hints rather than simply giving the answer.
While the tools were available, performance improved dramatically. GPT Base increased practice grades by 48% relative to control; GPT Tutor by 127%.
Then the systems were removed.
On a later unaided exam, the GPT Base group performed 17% worse than control. The negative effect was largely mitigated in GPT Tutor, though the tutor group did not show a positive exam advantage over control.
This study deserves weight because it is a large field experiment with a clear post-assistance criterion.
It demonstrates that an intervention can produce impressive task performance while simultaneously impairing subsequent unaided performance under those conditions.
It does not demonstrate that all generative AI use harms learning.
The strongest counterexample: scaffolded AI can support later comprehension
A different 2025 randomized controlled study, published in Computers & Education, examined 117 higher-education students learning to interpret visual learning analytics.
Researchers compared passive generative-AI agents, proactive agents using scaffolding questions, and standalone scaffolding.
The proactive AI condition produced stronger comprehension, and the benefits persisted beyond the intervention.
This result matters because it prevents false certainty.
If one study shows that unrestricted answer provision can undermine later independent performance and another shows that scaffolded AI can improve later comprehension, the findings are not necessarily inconsistent.
The systems required different cognitive work.
The task domains differed.
The comparison conditions differed.
The outcome measures differed.
The debate should therefore move from “AI or no AI” to interaction architecture.
Assistance changes the locus of effort
Learning requires activity, but not all effort is equally useful.
A system can remove clerical burden while leaving conceptual reasoning intact. It can supply examples while requiring the learner to explain why they work. It can generate feedback while preserving the learner’s need to retrieve, choose and revise.
Or it can perform the very operations the curriculum intends to develop.
A writing assistant might improve grammar while the learner still constructs the argument.
It might also construct the argument, select the evidence, write the explanation and critique the final text.
Both cases involve AI assistance.
They are not educationally equivalent.
The Institute’s synthesis is to track locus of cognitive work: which operations remain with the learner, which are shared, and which are fully offloaded.
That framework organizes existing evidence without pretending the literature has validated one universal taxonomy.
“Cognitive effort” is not automatically good
It would be easy to respond by maximizing human effort.
That would be a mistake.
Learning design has always used scaffolds. Worked examples can reduce unnecessary search. Calculators can free attention for higher mathematics. Spellcheck can prevent surface errors from consuming all of a writer’s working memory.
The educational problem is not effort reduction itself.
It is removing generative processes that are important for the target capacity.
If the goal is to learn argumentation, asking AI to correct punctuation may be harmless or helpful. Asking it to generate the entire inferential structure may reduce relevant practice.
If the goal is to learn vocabulary, instant translation may bypass retrieval. If the goal is to communicate accurately in a high-stakes moment, instant translation may be exactly the right tool.
The target defines the boundary.
Performance and learning should be measured separately
The field needs designs with at least two outcomes.
One outcome captures performance with the tool.
Another captures the capacity of interest after assistance changes.
The second measure might be delayed recall, novel transfer, an unaided problem, error detection in a new AI output, or performance with a less capable tool.
Without that separation, studies can report a productivity gain and call it a learning gain.
Yan, Greiff, Lodge, Gašević and colleagues’ 2025 Nature Reviews Psychology Comment makes precisely this point. The Comment is conceptual, not an experiment, but it provides a useful lens for reading the emerging trials.
The PNAS field study then shows why the lens matters empirically.
AI can alter metacognitive calibration
Assistance changes not only what a learner can do.
It can change what the learner believes they can do.
When an answer appears smoothly in a shared workspace, authorship boundaries can become psychologically blurry. The learner may remember the final understanding better than the moments where the system supplied missing structure.
The PNAS study found evidence that students did not necessarily recognize the learning cost associated with reliance on GPT Base.
This connects AI use to metacognitive calibration.
A learner needs to distinguish at least four judgments:
How good is this output?
How capable is the AI in this domain?
How capable am I of verifying it?
How capable would I be of performing the task if the tool failed?
Those judgments can diverge sharply.
A person can be accurate about the output and inaccurate about their own verification ability.
Verification itself requires learned capacity
One popular solution is: “Use AI, but always check the answer.”
Check with what?
Verification is not a magic operation performed by willpower. It requires enough domain knowledge, source literacy and error sensitivity to know what to inspect.
If assistance systematically replaces the process by which users would have developed that background knowledge, verification can become ceremonial.
The person rereads a fluent answer and confirms that it still sounds fluent.
This is why some internal knowledge remains important even in highly assisted environments.
It provides the priors and anomaly detectors needed to know when external output deserves scrutiny.
The Institute calls this capacity worth retaining. The phrase is synthesis, not an established construct.
Expertise changes the answer
AI assistance can have different effects for novices and experts.
Experts possess schemas that let them evaluate suggestions, detect subtle errors and use generated alternatives as raw material. Novices may lack the knowledge needed to distinguish an elegant error from a correct explanation.
At the same time, novices often benefit most from well-designed scaffolding.
This creates a design tension.
The learner who most needs support may also be least able to evaluate unconstrained output.
Guardrails, curated sources, progressive disclosure and prompts requiring attempts can therefore matter more early in learning. As expertise develops, the same system might allow greater delegation.
This is a hypothesis consistent with broader instructional science and current AI studies. The exact progression requires direct testing.
Motivation and access complicate the debate
AI can reduce barriers that have nothing to do with the target learning process.
A multilingual learner may use AI to clarify instructions. A student with dyslexia may use text-to-speech or rewriting support. Someone returning to education after years away may use conversational explanation to ask questions they are embarrassed to ask publicly.
Removing these supports in the name of “cognitive struggle” could reduce access rather than improve learning.
Conversely, systems that make every task effortless can reduce persistence or opportunities for successful retrieval.
Equity therefore cannot be treated as an afterthought.
The relevant question is whether support removes an irrelevant barrier or the target cognitive process.
Sometimes it does both.
That is exactly why simple bans and simple adoption slogans are scientifically inadequate.
Assessment design can accidentally reward offloading
If assignments reward only polished final output, learners are rational to use whatever tools improve final output.
Institutions cannot then infer independent capability from the product.
One response is to diversify evidence.
A course can include AI-assisted work, oral explanation, delayed retrieval, process notes, transfer problems or comparison tasks where students critique model outputs. None of these measures is perfect. Together they can reveal different parts of capability.
The goal should not be surveillance of every keystroke.
It should be alignment between the capability claimed and the evidence collected.
Long-term learning is still an evidence gap
Most generative-AI education studies remain short relative to the timescale on which expertise develops.
A post-test after days or weeks is stronger than immediate output but still does not tell us what happens after a year of routine assistance.
Will learners develop new forms of expertise around delegation and verification?
Will foundational skills atrophy?
Will curricula shift so that some previously internalized knowledge becomes unnecessary while other capacities become more important?
These are empirical questions.
Strong opinions currently outrun longitudinal evidence.
A minimum evidence design for claims about AI and learning
The debate becomes clearer if researchers agree on a minimum evidentiary architecture.
First, measure performance while assistance is available. This tells us whether the system actually helps with the immediate task.
Second, include a later criterion that matches the claimed learning outcome. For retention, use delay. For transfer, change the problem. For independent competence, reduce or remove assistance. For collaborative competence, keep the tool but introduce model errors, ambiguous cases or situations requiring verification.
Third, record enough of the interaction to know what work the learner performed. Two people can receive the same AI output while engaging with it differently: one may interrogate and revise it, another may copy it. Tool assignment alone is a weak proxy for cognitive process.
Fourth, measure prior knowledge. Novices and experts may benefit from different forms of support, and an average effect can conceal that divergence.
This architecture will not settle every question, but it prevents the most basic category error: calling improved assisted output “learning” without testing what changed in the learner.
Joint competence deserves its own outcome category
The argument is often framed as a choice between unaided human skill and tool dependence.
There is a third outcome: joint competence.
A person may become better at decomposing a problem for an AI system, spotting weak evidence, choosing when not to delegate, combining multiple model outputs, or recovering when the tool fails. Those are learnable capabilities even if they do not improve unaided performance on the original task.
They should not be smuggled into the category of independent mastery, but neither should they be dismissed as mere convenience.
This matters for professional education. A future lawyer, designer, analyst or clinician may genuinely need to work with AI. The curriculum therefore has at least two questions: what must this person still be able to do without the system, and what must they learn to do with it?
The balance will differ by domain because failure costs differ.
The long horizon may reveal effects short studies cannot see
Most current experiments are necessarily short.
That creates uncertainty in both directions. Repeated AI support could erode opportunities to practise foundational operations and produce cumulative dependency. It could also accelerate exposure to high-quality feedback, expand the number of problems a learner attempts, or enable adaptive practice that compounds positively over months.
Short post-tests cannot adjudicate those possibilities.
Longitudinal studies need to track not only scores but trajectories: how the division of labour changes, whether learners need less prompting, whether verification improves, whether conceptual errors recur, and what happens when the technology itself changes.
This is especially important because generative AI systems are moving targets. A finding about a 2024 model and interface may not transport cleanly to a 2027 tutoring environment. Durable theory will have to describe mechanisms, not brand names.
What the evidence does not show
Current evidence does not show that generative AI is inherently good or bad for learning.
The PNAS mathematics experiment does not prove that every unconstrained chatbot use causes harm.
The proactive-agent RCT does not prove that scaffolding always produces durable learning.
Immediate performance gains do not prove learning.
A later unaided test does not capture every valuable form of human-AI competence.
Cognitive effort is not intrinsically beneficial.
AI-assisted accessibility should not be confused with inappropriate offloading.
And there is not yet strong longitudinal evidence showing what years of routine generative-AI use do to broad human capability.
What the debate can say now
Several conclusions are better supported than a yes-or-no verdict.
First, studies should distinguish assisted performance from later learning.
Second, interaction design matters. Systems that preserve learner attempts, retrieval, explanation and error correction can produce different outcomes from systems that simply provide answers.
Third, learning goals matter. The capacity worth retaining must be specified before offloading can be evaluated.
Fourth, metacognitive calibration matters because users may misattribute joint performance to independent mastery.
Fifth, evidence should include transfer or post-assistance measures when durable learning is the claim.
The question underneath the question
“Does AI improve learning?” sounds like a technology question.
It is partly a curriculum question.
What are human beings still expected to know, notice, remember, create and judge when powerful external intelligence is always nearby?
Different answers will produce different studies and different tools.
The evidence does not require us to romanticize unaided cognition.
It does require us to stop confusing a better artifact with a more capable learner.
The future of learning will likely be assisted.
The responsibility is to design assistance so we can still say, with evidence, what has changed inside the learner as well as what became possible around them.
Research references
Bastani H, Bastani O, Sungu A, Ge H, Kabakcı Ö, Mariman R. Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences. 2025;122(26):e2422633122. https://doi.org/10.1073/pnas.2422633122
Yan L, Martinez-Maldonado R, Jin Y, et al. The effects of generative AI agents and scaffolding on enhancing students’ comprehension of visual learning analytics. Computers & Education. 2025;234:105322. https://doi.org/10.1016/j.compedu.2025.105322
Yan L, Greiff S, Lodge JM, et al. Distinguishing performance gains from learning when using generative AI. Nature Reviews Psychology. 2025;4:435–436. https://doi.org/10.1038/s44159-025-00467-5
Lee HP, Sarkar A, Tankelevitch L, et al. The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. CHI 2025. https://doi.org/10.1145/3706598.3713778