A student works ten equation problems of one kind and gets nearly all of them right. A week later, a test mixes equations that require different methods. The student now has to decide which method belongs to each problem before calculating. The earlier success may have reflected knowing the method for a labeled block, not recognizing when to use it. Interleaved practice could help with that choice, but only if the comparison isolates the relevant feature of the schedule.
Measuring interleaving means testing what happens when examples or practice problems from different categories are alternated rather than presented in blocks. The key construct is the sequence of categories or problem types and the cognitive demand it creates. Interleaving often also spaces repetitions of each category and can require retrieval of earlier procedures. Those neighboring features are not the same intervention. This Methods & Measures guide explains how to specify the sequence, select outcomes that test discrimination and transfer, and avoid attributing every mixed-practice benefit to one mechanism.
What is mixed, and for whom?
Interleaving is a relation among examples, not a property of one item. A study must define the categories being alternated. In art learning, they may be painters whose styles need to be discriminated. In mathematics, they may be problem types that call for different strategies. In medical education, they may be diagnostic categories. If all examples are drawn from one underlying category, shuffling their surface details is not necessarily interleaving in the sense tested by category-learning research.
A blocked schedule presents examples of category A together, then B, then C. An interleaved schedule might cycle A, B, C, A, B, C. But there are many variants. A random sequence can accidentally produce short blocks; an alternating sequence may be predictable; a curriculum can mix only two categories or many. The number of transitions, the similarity of neighboring categories and the interval between repeated examples may all affect learning. Researchers should report the actual sequence or enough detail to reconstruct it.
The learner’s knowledge changes the task. A novice may not yet know what category an example belongs to and can be overwhelmed by alternation before basic features are understood. A more experienced student may benefit from deciding among similar methods. A laboratory result using unfamiliar paintings need not transport directly to a complex course where each “item” is a long lecture. Measurement should specify prior knowledge and what learners could reasonably compare during each transition.
Immediate practice accuracy can be misleading. Blocking often makes the next method obvious: after several similar problems, the student can repeat a procedure without deciding whether it fits. Interleaving may lower performance during practice because the learner must retrieve or select the method. A later mixed test can reverse the apparent ranking. A study that judges success only from practice accuracy can reward ease during study instead of durable learning.
Category-learning experiments
Kornell and Bjork compared blocked and interleaved presentation of paintings by different artists. Learners later identified the artist of new paintings more accurately after interleaving in their studied conditions, despite often believing that blocking had helped them more. The contrast is informative because the test required category generalization to new examples, not mere memory for a repeated picture. It also reveals a metacognitive problem: subjective ease during study is not a dependable outcome measure.
Birnbaum and colleagues investigated why interleaving might help inductive learning. Their experiments examined discrimination and retrieval accounts rather than assuming that temporal spacing alone explained the effect. A mixed sequence can place unlike categories near each other, making differences more visible. It can also require the learner to recall a category representation after intervening examples. Both processes may operate, and their relative importance depends on the material and test.
Kang and Pashler tested painting-style learning with designs intended to separate temporal spacing from discriminative contrast. Their result favored the role of interleaved contrast under those conditions. This does not show that spacing never matters. It shows why “the artist’s examples were farther apart” is insufficient as an explanation when a design also changes which other artists’ examples appear between them. A strong experiment manipulates the two features independently where possible.
Brunmair and Richter’s meta-analysis found that similarity moderates interleaved learning effects. That conclusion fits a discrimination account: alternation may be especially useful when categories are confusable enough that learners need to notice distinguishing features. Yet it is not a rule that every pair of similar topics should always be mixed from the beginning. Too much complexity, insufficient exposure or an unsuitable test can alter the outcome. Meta-analytic averages should guide hypotheses, not replace task analysis.
Mathematics asks a different question
In mathematics, the learner often has to select a procedure after reading a problem. Rohrer and Taylor studied shuffled versus massed mathematics practice and found benefits on later performance in their tested material. Later classroom studies brought mixed practice into authentic instruction. Those studies matter because learning in a course involves more than recognizing a painter’s style: students must identify a problem type, recall a strategy, execute it and check an answer.
A field experiment by Rohrer, Dedrick and Burgess with seventh-grade students examined interleaved and blocked practice across several weeks and a later unannounced test. The authors argued that benefits were not confined to superficially similar problem types and could involve strengthening the link between a kind of problem and its strategy. That is a specific claim about a studied curriculum and assessment. It does not imply that every math topic, sequence length or age group will show the same result.
Classroom experiments bring ecological strength and confounds. Teachers may adjust explanations in response to difficulty, students may seek outside help, and homework completion varies. A study should report how the schedules were implemented, whether total practice time and number of problems were equal, and how the final test sampled problem types. If the interleaved group received more review across the term, the contrast may blend category mixing with distribution over time.
A later mixed test is often the most relevant outcome because real problems rarely arrive labeled by the last lesson. But a blocked final test can answer a different question: did students learn the procedures when the method was obvious? Comparing both can show whether a schedule improved method selection, execution, or both. A transfer test with novel surface forms adds another constraint. The outcome should be chosen before results are known to match the promised learning benefit.
Separate interleaving from spacing
Spacing means distributing repeated encounters with material over time. Interleaving means alternating categories. An A-B-C-A-B-C sequence spaces each A, B and C repetition relative to AAA-BBB-CCC. It also inserts contrasting categories between repetitions. Without another condition, a benefit cannot be assigned uniquely to alternation or to the longer lag. A comparison can use blocked materials spaced across sessions, or manipulate within-session category order while matching the overall time between encounters as closely as the task allows.
Exact matching can be difficult. Changing order alters lag, recency and local contrast simultaneously. Researchers should describe which variables were matched and which remained confounded. They can measure the actual lag between repeats and include it in analysis, though statistical adjustment alone may not recover a clean causal contrast if lag and sequence are tightly coupled. The question is not whether a study is perfect but whether its conclusion is proportional to the design.
Spacing may support memory for individual examples or procedures through repeated retrieval over time. Interleaving may support discrimination among confusable categories by juxtaposing them. These mechanisms can reinforce one another, but they also make different predictions. If learners improve mainly at identifying which category a novel example belongs to, a discriminative explanation gains plausibility. If improvement appears mainly for recalling old items after delay, a spacing explanation may be stronger. The test can be designed to separate these outcomes.
A systematic review of distributed practice and retrieval practice in health-professions education addresses neighboring methods, but it is not direct evidence about category alternation. Interleaving experiments and meta-analysis are needed for the main claims here, while the education review reinforces the need to distinguish these techniques.
Separate interleaving from retrieval practice
Retrieval practice asks a learner to bring information to mind rather than merely reread it. Interleaved problems often force retrieval because the previous problem used a different method. Yet a learner could retrieve a method in a blocked schedule too, and an interleaved display of examples could occur without any active response. Mixing the order and requiring retrieval are distinct experimental factors.
A design could compare interleaved and blocked sequences while keeping the response requirement the same, then separately vary whether learners solve, classify or merely study examples. Solving may demand both method selection and procedure retrieval. Classification can isolate recognition of the appropriate method to a degree, though it still involves category knowledge. Restudy tests whether juxtaposition helps without overt retrieval. The pattern across these conditions can clarify what aspect of practice produced the later result.
The final test must likewise be specified. An untimed classification test asks whether learners discriminate categories. A worked-solution test asks whether they can execute a procedure. A delayed free-recall test asks something else. If researchers call any improvement “interleaving,” they hide the skill that improved. The useful claim is concrete: in this schedule, learners became better at selecting among these problem types on this later test.
Similarity, complexity and prior knowledge
Similar categories can create confusion during blocked study because each block offers too few opportunities to compare near neighbors. Interleaving puts contrasts closer together. But if categories are very dissimilar, the next example may be easy to identify without comparison, so the extra switching can add little. If categories are extremely complex and novel, learners may need some initial blocked exposure to form a basic representation before interleaving becomes productive. This is a design hypothesis, not a universal prescription.
The meta-analysis by Brunmair and Richter supports the importance of moderators, particularly category similarity, but effect sizes across studies vary with material and procedure. A researcher should describe how similarity was established rather than call topics “similar” by intuition alone. Independent ratings, confusion matrices from a pilot, or pretest misclassification patterns can help. The same pair of categories may be confusable to novices and easy for experts.
Item variability also matters. If each category is represented by nearly identical examples, success may reflect memorizing a superficial cue. If examples vary too widely, the learner may have too little common structure to infer. A test with new examples is useful because it asks whether a category representation transfers beyond studied items. The source studies on paintings and problem types use such logic, but the proper novelty level depends on the educational goal.
Motivation and effort can shift the result. Interleaving often feels difficult and may discourage a learner who expects smooth performance. If one group spends longer on problems or receives more feedback, the schedule comparison becomes less clean. Researchers should record time on task, completion, feedback and beliefs about learning. A lower practice score can be a sign of desirable challenge, but it can also indicate that the student lacked prerequisites. The later outcome decides whether the challenge helped.
Measurement design and analysis
First identify the unit of randomization. In a laboratory, participants may be assigned to one schedule. In a classroom, lessons, homework sets or schools may be assigned, raising issues of clustering and teacher effects. If students see each other’s materials, contamination can reduce contrast. The design should document the actual exposure rather than assume the planned schedule was followed.
Second equate content. Groups should encounter the same categories and a comparable number of examples, with equivalent feedback and total study time where feasible. If interleaving receives more items or extra review, the study cannot isolate order. If blocked students finish earlier, the researchers must decide whether to equalize time or let the natural schedule difference stand, and then state the estimand: equal content, equal time, or real-world implementation.
Third select outcomes that match the mechanism. New-item categorization tests discrimination. Method-choice questions test selection. Worked problems test both selection and execution. Delayed tests address retention; transfer tests address generalization. A single overall score can conceal that interleaving helped only one component. Reporting by category, item type and error type makes the result more interpretable.
Fourth examine sequence properties. Count transitions between categories, spacing of repetitions and run lengths. A supposedly interleaved random schedule might contain many same-category neighbors, while a blocked schedule could include spaced review at the end. The labels should reflect realized exposure. Pre-registering how order will be generated and what deviations are allowed prevents post hoc relabeling.
Finally, distinguish performance from inference. A statistically significant schedule difference establishes a contrast under the tested conditions, not the unique psychological pathway. Mediation claims about discrimination, retrieval or spacing require direct measures and designs that vary those candidates. The Institute’s measurement checklist synthesizes the cited research; it is not a validated educational assessment.
What the evidence does not show
Interleaving studies do not show that mixing every subject all the time improves learning. The benefit depends on what categories are mixed, how similar they are, what learners already know, how much practice they get and how learning is tested. Immediate difficulty does not guarantee later gain. A schedule may raise cognitive demands beyond what a novice can use.
A mixed-practice advantage does not by itself identify interleaving as the cause when repetitions were also spaced or when learners retrieved more often. Nor does success on a delayed mixed test prove transfer to every novel setting. Laboratory category studies, classroom mathematics studies and health-professions curricula differ in item length, stakes and implementation. Results should be described within those bounds.
A review of distributed and retrieval practice cannot validate an interleaving claim by association. The strongest conclusion is narrower: when category order is manipulated while important exposure features are documented, later discrimination or problem selection may improve under identifiable conditions. A more specific mechanism claim needs comparisons that separate category contrast from spacing, retrieval and feedback.
Return to the equations
The student’s success on a homogeneous page showed that she could apply a method when the page effectively named it. To test whether interleaving helps her recognize which method belongs to an unfamiliar problem, a study would hold problems and feedback comparable, vary their order, and give a delayed mixed test that includes new examples. It would report both method-choice and calculation errors. That result would be much more useful than a generic claim that mixing is good.
Measuring interleaving is ultimately about precision in both schedule and outcome. The word names an ordering of categories, not a complete theory of learning. When the ordering is explicit and rival explanations are tested, researchers can learn when alternation sharpens discrimination and when it merely makes practice harder.
Research references
- Kornell, N., & Bjork, R. A. (2008). Learning concepts and categories: Is spacing the “enemy of induction”? Psychological Science, 19(6), 585–592. View source
- Birnbaum, M. S., Kornell, N., Bjork, E. L., & Bjork, R. A. (2013). Why interleaving enhances inductive learning: The roles of discrimination and retrieval. Memory & Cognition, 41, 392–402. View source
- Kang, S. H. K., & Pashler, H. (2012). Learning painting styles: Spacing is advantageous when it promotes discriminative contrast. Applied Cognitive Psychology, 26, 97–103. View source
- Brunmair, M., & Richter, T. (2019). Similarity matters: A meta-analysis of interleaved learning and its moderators. Psychological Bulletin, 145, 1029–1052. View source
- Rohrer, D., & Taylor, K. (2007). The shuffling of mathematics problems improves learning. Instructional Science, 35, 481–498. View source
- Rohrer, D., Dedrick, R. F., & Burgess, K. (2014). The benefit of interleaved mathematics practice is not limited to superficially similar kinds of problems. Psychonomic Bulletin & Review, 21, 1323–1330. View source