Baseline: what a shape of exercise set must beat
This file names the baseline that a proposed exercise-set shape must beat, and how the two would be compared.
The named baseline
| Role | Baseline | Why this one |
|---|---|---|
| To beat | B1, the blocked sheet: one section per problem type, each section a run of same-type problems after its lesson, answers to half the items or full solutions at the end | the form of most practice in print: 9.7% of 13,505 problems interleaved in six texts (Rohrer 2020b); odd-numbered answers at the back (OpenStax 2026); the control condition of every interleaving trial (Rohrer 2015) |
| To match on the first problems of a type | B2, the example-problem pair: a worked example, then one problem of the same type, then whole problems | the best-supported opening (van Gog 2011, linked); recommended at moderate evidence by the IES guide (Pashler 2007) |
- B1 is what the shapes must beat; B2 is what their opening must not fall below.
- A shape that beats B1 only by adding examples has not shown that its order matters; the comparison holds the problems fixed and changes only order and support, as Rohrer 2015 did.
The candidate shape, to test against B1
- For a new type: one example-problem pair, then one or two completion problems with the last steps left (fading; Atkinson 2003).
- A short block of same-type problems, four in Rohrer 2015's illustration.
- Interleaved review: problems of earlier types, drawn from several earlier sets (spacing and mixing; Rohrer 2009).
- For each problem: a hint, then a final answer, then a worked solution, each folded or on a separate page.
- Steps 1 to 3 rest on measured parts; their combination and step 4 are convention, untested.
How a comparison would be measured
| Outcome | Instrument | Prior study to borrow from |
|---|---|---|
| Delayed performance | unannounced test of mixed problems, one day and 30 days after the last set, scored blind | Rohrer 2015 (d = 0.42 and 0.79) |
| Strategy choice | share of test problems where the correct strategy is chosen, scored apart from execution | Rohrer 2014 (dissimilar types) |
| Transfer | near and far transfer problems | Atkinson 2003; Renkl 2002 |
| Time on task | minutes per set, logged | Ericson 2018 (efficiency) |
| Judgement of learning | predicted score against actual | Emeny 2021 (overconfidence after massing) |
| Help use and copying | order of opening hint, answer, solution; abnormally short attempts | Roll 2011; Palazzo 2010; Chen 2022 |
- Design: classes randomised to B1 or the shape, same problems, same teachers where possible; at least several weeks of sets, as in Rohrer 2015 (about three months) and Rohrer 2014 (nine weeks).
- Effect sizes: Cohen's d or Hedges' g per outcome, with the school or the class as the second level of a two-level model; Roschelle 2016 used schools.
- Moderators to record: prior knowledge (expertise reversal, Tetzlaff 2025 in the core; Likourezos 2019), subject outside mathematics (Perry 2021).
- The cumulative end-of-year test belongs in the design: the one year-long trial found a short-term gain and no cumulative one (van der Haar 2023).
A checklist in the baseline's own terms
Per section of B1, from the textbook form:
- Lesson, then a run of problems of that lesson's type.
- Problems sometimes graded or starred by difficulty (MathEd 2014f).
- A mixed review block of five to ten earlier problems (the form Rohrer 2015 describes).
- Answers to odd-numbered problems at the back; full solutions in a separate manual.
A shape may drop any of these on purpose; the shape says which, where a reader would look for it.