Baseline: what this repository's rules are compared against

The review is finished when the baseline is named. This file names it and says how a comparison would run.

Named baseline

  • Checklist: Lampson 1983 and its revision Lampson 2020 (hints, goals STEADY, techniques AID); the alternative checklist is the AWS Well-Architected Framework with six pillars (AWS 2026p).
  • Check: Newcombe 2015, a precise model of the design checked before code exists.
  • Taught method: Primer 2026, four steps from use cases to scaling, the most adopted by stars.
  • Failure benchmark: the failure classes of Yuan 2014 and Alquraan 2018, as the things a design must survive.

What is compared

  • This repository's rules are about writing the design down: state load, failure model and consistency assumptions beside the design.
  • The baseline has no rule about the document; it has rules about the system. The comparison is therefore of what a reader finds, not of a rule list.

How, as a proposal (not run)

  • Reader tasks: "find what the design assumes about failure" and "find a flaw in a seeded design", on designs with known flaws from Alquraan 2018.
  • Conditions: the design written with the repository's rules, and the same design written in the primer's four steps.
  • Design: crossover, so each reader sees both documents on different designs; order balanced.
  • Measures: flaws found, time, and wrong beliefs about the failure model; effect sizes with intervals.
  • Control for the reader's experience; report the sample.

What would count as a failure of the rules

  • Readers find no more seeded flaws with the rules than with the primer's steps.
  • Readers state more wrong assumptions about the failure model.

Unknown

  • No study of design documents for running systems was found, so no effect size exists to power the comparison.