Protocol: how the pool of sources was built

Every search, count and rule of the flashcards review, so it can be repeated. All searches ran on 2026-10-29, by one reviewer, and the stopping rule (section 6) was written down before the first. Shared rules: evidence README.

1. Why grey literature

For flashcards, Garousi 2019, Table 4 gets a yes on all seven questions.

# Question Answer for flashcards
1 Complex, not solvable from formal literature alone? yes: the formulation rules come from practice (Wozniak 1999)
2 Lack of volume or quality of evidence? yes: no study of one fact per card, or of cloze against question cards, was found
3 Context important? yes: second-language words, medical facts and mathematics use cards differently
4 Corroborate research with practice? yes
5 Challenge practice with research, or the reverse? yes: shared decks dominate in medicine, where the largest controlled comparison on text material favours learner-made cards
6 Would a synthesis serve both communities? yes
7 Large practitioner volume? yes: 68.3% of 560 US medical students used Anki in 2020 (Halperin 2024)

2. Questions

Id Question
Q1 How is a card formulated (one fact, cloze against question and answer, direction, context, images), research against practitioner rules?
Q2 Do learner-made cards beat ready-made ones?
Q3 How do learners use flashcards (dropping, stopping, cramming), and what does it cost?
Q4 How much should a card hold; do similar cards interfere; do cards teach understanding or only facts?
Q5 What is measured about texts with embedded prompts (the mnemonic medium)?
Q6 What do practice and adoption look like (Anki, Quizlet, SuperMemo, shared medical decks), with counts and dates?
Q7 How do cards generated from notes compare with written ones?

Scheduling algorithms are not a question here; they are linked to the core's benchmark (Open Spaced Repetition 2026a).

3. Existing reviews (Garousi Guideline 2)

  • Searched first, with strings F22 (reviews of spaced repetition and flashcards) and F23 to F25, and by checking each formal string's list for reviews.
  • Found: 11 reviews and syntheses: Nakata 2011, Webb 2020, Lei 2022, Versteeg 2020, Xu 2024c, Öcel 2025, Nguyen 2021, Teymouri 2024, Kornell 2007, Bjork 2013, Kang 2016; all read by abstract. One handbook chapter was located, not opened (Nakata 2019b).
  • Two meta-analyses of the generation effect (Bertsch 2007; McCurdy 2020) and one of text generation (Schindler 2023) are used in strand 3.
  • Only Lei 2022 compares who makes the cards, by counting studies; none compares card formulations. They are placed in review.md strand 0.

6. Stopping rule

  • Stated before searching: effort bounded, the 20 most cited (Semantic Scholar) or most relevant (ERIC) per string, with saturation at k = 5 strings in a row that add no new included source.
  • Formal: evidence exhaustion where a string returned fewer than 20 records; effort bounded otherwise.
  • First pass: the last string to add a source was X1, and X2 to X8 (seven strings) added none, so saturation was reached.
  • Retries: the retried Semantic Scholar list of F23 then added two sources (Lei 2022, Nakata 2019b); F27, F34 and F35 added none.
  • Two more strings, F38 and F39, were run to test saturation and added none; with them, five strings in a row added nothing, and the search stopped.
  • Grey web: the first result page per string; G6 and G7 added nothing new, and G3 added nothing readable.
  • Q/A: the top 30 per string and site; stopped after FC14 (section 5).

7. Selection

  • Include: a source on how cards are worded, who makes them, how learners use and keep them, embedded prompts, generated cards, or adoption. The source is in English and readable on the day at least by abstract or, for a book, by a catalogue page.
  • Exclude:
    • a study that only re-measures retrieval or spacing (linked to the core, not reviewed): for example Janes 2020, which applies successive relearning (Rawson 2013, held by the core), Price 2025 and the schedulers' papers;
    • scheduling algorithms (SM-2, FSRS, half-life regression), linked to the core's benchmark;
    • single-case studies of flashcard drills for children with disabilities, and studies of children learning to read words;
    • works of another topic, passed on in handoff;
    • a systematic review of six study strategies in STEM courses, received by handoff from the exercise-sets review (ERIC EJ1517222, abstract read): it asks whether the strategies work, the core's question;
    • Rawson 2007, on self-evaluation of key-concept definitions, received by handoff from the instructional-text review: no abstract reached, title only, so not used; Golding 2012, handed over by the same review, was already included;
    • a record whose abstract could not be read and whose title does not settle relevance.
  • Grey sources must also pass section 8.

8. Quality of grey sources (Garousi Guideline 11, Table 7)

  • Outlet tier: books and encyclopedias of record = tier 1 (1); company pages, project documentation, Q/A threads, wiki pages = tier 2 (0.5); personal essays and blogs = tier 3 (0).
  • Scores, with subtotals, are in sources.

9. Extraction and synthesis

  • Extracted per source: kind, year, the claim, the evidence offered, any number with its location (marked "(secondary)" when read in an abstract), and the question it answers.

10. Limits

  • One reviewer; no second rater on this topic's grey sources.
  • Most formal sources were read by abstract only; Deng 2015 and Lu 2021 were read in full. Publisher pages of Wiley, APA and Taylor and Francis were not readable.
  • Semantic Scholar refused six strings on the day (marked); its counts are word matches, not phrase matches, and are noisy upper bounds.
  • One OpenAlex request was made by mistake, to read the abstract of Kornell 2009; it returned the same text as Crossref, and no count was taken.
  • The mnemonic-medium data of the author's working notes could not be read (pages returned 404), so only the 2019 essay is used.
  • App-store and Quizlet pages could not be read; adoption counts rest on surveys and repository counts.
  • This search did not find a study of one fact per card, of cloze against question cards in a spaced program, or of an essay with prompts against the same essay without. That is not proof that none exists.