Review

This file reads the literature on how a set of exercises is ordered and supported, strand by strand. The strands are: existing reviews, seminal work, then one strand per sub-question, practice, and Q/A sites. "(secondary)" marks a number read in an abstract or summary, not in the paper. Grey sources under 10 of 20 are named "example only" (sources).

0. Existing reviews

  • Firth 2021: systematic review of interleaving for concept learning; 26 studies, 17 meta-analysed; memory g up to 0.65, transfer up to 0.66; laboratory studies of undergraduates dominate (secondary).
  • Chen 2021: a systematic review arguing that spacing works through rest from effort and interleaving through discriminative contrast, two mechanisms (abstract).
  • Murray 2025: meta-analysis for mathematics; spaced against massed practice g = 0.28 (27 studies, 53 effects), 0.24 inside courses (17 studies), 0.43 for material taught in isolation (secondary).
  • Perry 2021 (EEF, grey tier 1, read first-hand): among studies run in schools, 12 on interleaving, 11 of them in mathematics, about five delivered by teachers or with their involvement. Also: 22 on worked examples, all mathematics and science, 7 on incorrect or incomplete examples, 8 delivered by regular teachers.
  • Atkinson 2000 turns worked-example research into design principles; Muldner 2022 reviews worked examples in programming and counts Parsons puzzles as incomplete examples (both abstract).
  • Du 2020: Parsons problems come in many variants, and their effect remains uncertain for lack of replicated studies; Ericson 2022 reviews the variants and claimed benefits and designs studies to test them (both abstract).
  • Homework: Cooper 2006, every design 1987 to 2003 flawed, a consistent positive influence; in the correlational studies, the correlation was stronger in grades 7 to 12 (secondary). Also: Bas 2017, d = 0.229 (secondary); Fan 2017, a 30-year meta-analysis for mathematics and science (title and venue only).
  • Bishop 2024: online against paper homework, 18 studies, effect 0.06 (CI -0.19 to 0.32), not significant (secondary).
  • Aleven 2003: learners in interactive environments do not use on-demand help effectively (secondary); Aleven 2016 updates it (strand 5).
  • Linked from the core: Brunmair 2019 (interleaving g = 0.42), Renkl 2004 (fading), Barbieri 2023 (worked examples), Costa 2022 (4C/ID).
  • How this review differs: it reads the set as one document, puts order, support, solutions and volume side by side, and adds adoption counts and Q/A practice.

1. Seminal work

  • Sweller 1985: worked examples as a substitute for part of algebra problem solving (title and venue only); the IES guide describes an algebra study of four example-problem pairs against eight problems (Pashler 2007).
  • van Merriënboer 1990: the completion strategy, modifying and extending given programs, against writing new ones, in high school (abstract; the result is not stated there).
  • Paas 1992: worked and completion problems beat conventional problems for transfer in statistics, worked problems most, by the author's abstract; ERIC's summary says the reverse (46 students, secondary).
  • Rohrer 2007: the standard textbook set (one lesson's problems) against the shuffled set (problems from many lessons); spaced and mixed both won at one week (secondary).
  • Parsons 2006 introduced the puzzle format for first programming courses; Bjork 1994 named desirable difficulties (both title and venue only).

2. Q1: blocked, interleaved and mixed review in classrooms

  • Rohrer 2009 (review): mixed review joins two features, spacing across sets and mixing within a set; each helps, for different reasons (secondary).
  • Rohrer 2015 (read in full): 9 seventh-grade classes, 126 students, 10 assignments over about three months, solutions shown after each.
    • Interleaved against blocked: 80% against 64% one day later, 74% against 42% thirty days later; d = 0.42 and 0.79.
    • The authors' informal survey: mixed review made up under 15% of textbook practice problems.
  • Rohrer 2014: 140 seventh graders, nine weeks, test two weeks later: 72% against 38%, d = 1.05, with dissimilar problem types (secondary).
  • Rohrer 2020 (core, cluster trial, d = 0.83) is linked; Rohrer 2020c argues little long-term gain from more than a few same-type problems in a row (secondary).
  • In tutors: one interleaved homework session helped at a delayed session, most for low-skill students (Ostrow 2015); interleaving task types beat interleaving representations for 158 pupils (Rau 2013) (both secondary).
  • Nemeth 2023: 236 third graders, 14 lessons; prior knowledge helped in the blocked condition, need for cognition in the interleaved one, with no significant difference between them (secondary).
  • Chemistry: mixed sets of identical questions beat sets grouped by chapter, the gap widening over three sessions (Gulacar 2022). Also: an uncategorized homework cohort scored higher on four tests and the final, significant on two (Cox 2024, snowball; cohorts, not randomised) (both secondary).
  • Emeny 2021: Year 7, two classroom experiments; spacing over three weekly sessions beat massing a month later, and massed students were overconfident (secondary).
  • Hartwig 2022: 193 and 175 undergraduates mostly judged the least spaced, least interleaved schedules most effective (secondary).
  • Null and mixed classroom results are in against: Nigeria (van der Haar 2023), calculus (Gorgievski 2024), statistics (Crissinger 2015), STEM quizzes (Bego 2024).
  • Saxon Math builds on daily cumulative mixed practice: its grade 7 text holds 1,491 blocked and 3,016 interleaved problems (Rohrer 2020b, read first-hand).
    • First grade: Saxon and Math Expressions 0.30 SD above Investigations, 0.24 SD above SFAW (Agodini 2010, secondary); WWC 2017 found mixed effects (5 studies, 8,855 students). WWC 2016 found none for Algebra I (198 students) (both secondary).
    • These test a whole curriculum, not mixed review alone.

3. Q2: from worked example to problem

  • Pairs: example then problem beats problems only, problem then example does not (van Gog 2011, linked in reading-order).
    • With identical items the example-first advantage held during learning and vanished at test (van Gog 2011b, 16 per group, secondary).
    • Example-problem was more efficient for 61 gifted and 65 other pupils (Coppens 2019); order had no significant effect for 156 middle-school students learning inquiry skills (Ganaiem 2025) (both secondary).
    • van Harsel 2019: 124 technical students; examples only beat example-problem pairs and problems only, and example-problem did not beat problem-example (secondary).
  • Fading: removing steps beats example-problem pairs on near transfer; far transfer unclear (Renkl 2002, three samples, secondary).
    • Atkinson 2003: backward fading with prompts to name each step's principle; medium to large near and far transfer, no extra time (secondary).
    • Kissane 2008: financial-services trainees; fading beat pairs and problem solving, more at delay and on transfer (secondary).
    • Salden 2009: in a tutor, adaptive fading beat fixed fading, which beat problem solving, in the lab and a classroom (secondary).
    • Jennings 2020: help whose examples grow more similar to the problem (fade in) beat the reverse (fade out); near-identical examples invite copying (secondary).
  • Learners' choice: free to choose, they picked worked examples on under 40% of trials and seldom before a problem, but used partial examples well (Foster 2018, secondary).
  • Completion problems: Paas 1992 (strand 1). The IES guide says that fading examples into problems, early steps given and later ones left to the learner, also seems to benefit learning (Pashler 2007, grey tier 1, read first-hand). Its recommendation 2 is rated moderate.
  • Parsons problems: as effective as writing the equivalent code and more efficient, plain (Ericson 2017, as Ericson 2018 reports it) and adaptive (Ericson 2018, abstract).
    • The efficiency gain disappeared when the expected solution was unusual (Haynes 2021, abstract); distractor lines lowered learning efficiency for young novices (Harms 2016, title only).
    • Hou 2025 (read in full): 118 students; Parsons puzzles on a help request kept them practising 22.7 minutes against 15.8 with complete solutions; no post-test was run.
  • Classrooms: Perry 2021 found 22 worked-example studies, all mathematics and science, with fading named as an implementation.

4. Q3: difficulty and variability

  • Paas 1994 (core) found varied worked examples transfer better; Likourezos 2019 bounds it: high variability helped more experienced adults and hurt novices (103 and 56 adults, secondary).
  • Jansen 2013b: 207 children on an adaptive program set to different success rates; the higher the success rate, the more problems attempted and the larger the gain (secondary).
  • Pardos 2009: in five randomised three-item sets of a tutor, item order mattered in two (secondary).
  • Tian 2021: textbooks give some decimal problem types far less often; children err more on the rarer types, whatever their complexity (secondary).
  • Lithner 2003: students solving textbook exercises chose strategies by surface features, without the mathematics (secondary).
  • NCETM 2020 (grey, read first-hand): vary one feature at a time; variation is not variety; practise the thinking, not the procedure.
  • This search found no study comparing easy-to-hard order with mixed difficulty inside one exercise set.

5. Q4: hints, answer keys, full solutions and copying

  • Help seeking: Roll 2011 (58 and 67 students) improved help use with feedback, transferring to new content; Aleven 2016 found it did not raise domain learning (secondary).
  • Hints in programming: next-step hints with explanations raised immediate performance; a later task without hints improved only with self-explanation prompts added (Marwan 2019, secondary).
  • Gaming: Baker 2008 links clicking through hints to motivation and affect (secondary); Vanacore 2024 found gamers may gain from delayed hints, without closing their gap (secondary).
  • Generated hints: Pardos 2024, N = 274; generated hints alone beat a no-help control, equal to human hints, but failed quality checks on 32% of problems before a fix (secondary).
  • Bastani 2025: about a thousand high-school students; open model access raised practice grades 48% and lowered later exam grades 17%; a hint-only tutor raised practice 127% and mostly avoided the harm (secondary).
  • Solutions given in advance: one finance course scored lower in the year students had the solutions manual (Rystrom 2005, two cohorts, secondary).
  • Copying: Palazzo 2010 detected copying rising toward deadlines; copying analytic answers went with a 2 standard deviation decline over the semester in relative score on similar exam problems. Surveys reported about a third less copying than detected (secondary).
    • Glass 2020: 2,433 students, 12 courses; the share not benefiting from homework rose from 14% (2008) to 55% (2017), and self-reported copiers did not benefit. Also: Chen 2022 found no widespread rapid copying in one remote physics course (both secondary).
    • Lancaster 2021: requests on Chegg in five STEM subjects rose 196.25% from 2019 to 2020; Pickering 2024: over half of audited units had content there, half answered within 1.5 hours (both secondary).

6. Q5: how much practice

  • Rohrer 2006: 216 students; 10 problems split over two sessions doubled scores at four weeks; 3 against 9 problems in one session made no difference at one or four weeks (secondary).
  • Gorgievski 2024: 50% more problems per set did not change final-exam scores (secondary).
  • Trautwein 2007: assigning homework relates to achievement at class level; time on homework does not capture it (secondary).
  • Online homework with hints: ASSISTments, 43 schools, 2,850 seventh graders, g = 0.18 (Roschelle 2016, read in full); larger for low prior achievement (Murphy 2020, secondary).
  • Kontur 2015: homework completion went with better exams only for high-aptitude physics students, negatively for low aptitude, copying ruled out (arXiv abstract). Also: Rieger 2016 found the correlation for all students in two larger courses (514 and 744), clearest for lower aptitude (secondary).

7. Practice

Graded convention, untested, unless a study measured the effect. Counts are in state of the art.

  • Textbooks: blocked sets with a short mixed review dominate; 9.7% of 13,505 problems in six texts were interleaved (Rohrer 2020b).
  • OER textbooks print answers to odd-numbered exercises only: OpenStax Calculus volume 1, chapter 1, answers odd-numbered exercises only, 1 to 335, except 147 to 153 (OpenStax 2026).
  • Online homework: WeBWorK (since 1994, over 35,000 problems; WeBWorK 2026). ASSISTments claims over a million students since 2019 (ASSISTments 2026, example only), and its skill builders end on a streak of correct answers, three in the trial's example (Roschelle 2016).
  • Programming: Project Euler says solving one problem opens a harder one, and counts 151,219 members with 25 or more solved (Project Euler 2026, example only); Runestone e-books embed Parsons problems (CSEd 2017d).
  • Guidance: IES (Pashler 2007) and EEF (Perry 2021) recommend examples alternated with problems; NCETM 2020 recommends variation sets.
  • Solution sites: Chegg reported 3.6 million subscription subscribers in the fourth quarter of 2024, down 21% year over year (Chegg 2025).

8. Q/A sites

Mathematics Educators, Computer Science Educators and Academia Stack Exchange, searched 2026-10-29: 29 threads, 10 under 10 of 20. Practitioner opinion; one answer carries data.

  • Q4 dominates: whether to publish solutions, and how to set homework when answers are online.
    • Common practice: homework ungraded or worth little, solutions published or withheld until submission, a short proctored quiz on similar items (MathEd 2014a; MathEd 2017a; MathEd 2023a; Academia 2023a). Some make homework optional and only exams count (Academia 2018a).
    • Against full solutions: students cannot yet judge a proof and need hints (MathEd 2016b); authors ask teachers not to post solutions (Academia 2021a). Give guides to the process, not code dumps (CSEd 2018a, example only).
    • For them: review before exams, fair access since old exams circulate, and self-study, as in Schaum's Outlines (MathEd 2014b; MathEd 2014c; MathEd 2021a; MathEd 2022b, example only). Solutions are not required and are usually given in some form (Academia 2017a).
    • Practitioners ask for evidence and find none: on sheets with and without worked solutions (MathEd 2021b, example only), on exercises without answers (MathEd 2019a, example only).
  • Copying is assumed: teachers name Chegg, Symbolab and Mathway (MathEd 2020a), students say every MasteringPhysics item is on Chegg (Academia 2019a, example only). Also one teacher asks students at random to explain what they handed in (Academia 2016a).
  • Q1 and Q3: integration homework framed as choosing the technique (MathEd 2015a); interleaving's spread asked about, with no count given (MathEd 2022a, example only). Starred problems signal expectations (MathEd 2014f); struggle helps when productive (MathEd 2014e); trick problems reward prior exposure (CSEd 2018b, example only).
  • Q5 and Q6: one instructor's unpublished data, homework done against test average, R² = 0.20, N = 149 (MathEd 2016a). A calculus class with about 15 exercises per section, three sections a week (MathEd 2014g, example only). Self-learners told to work odd problems because their answers are printed (MathEd 2024a, example only). Online homework judged by the practice it gets done (MathEd 2014d, example only).
  • Q2: Parsons problems are used on paper and in e-books, with no data (CSEd 2017d); students can pass exercises without being able to explain them (CSEd 2017e).