Evidence against and open controversies

This file gathers the evidence against the rules on system diagrams: null results, disputed studies, and practice that a study contradicts.

Against drawing rules from theory

  • The Physics of Notations is a theory; its applications are rarely verifiable (van der Linden 2019).
  • It is not precise enough to apply objectively to UML use case diagrams (Störrle 2013, abstract).
  • Cognitive dimensions give a vocabulary, not a test (Green 1996).
  • A test of one principle, semantic transparency, with a key present found no gain in accuracy or speed (Santos 2018, 57 novices).

Against layout rules

  • Earlier experiments found little effect of layout; Eichelberger 2009 is quoted as finding none (Störrle 2014).
  • Störrle 2011 found small accuracy gains (+6.5%) beside large preference gains (+28.2%). A rating is not a reading outcome.
  • Sharif 2010 found a helpful layout that broke general aesthetics.
  • The Störrle studies come from one group, with students, on paper or screen exercises; Störrle 2014 and 2016 re-analyse the same 2012 data. Störrle 2016b pools them with the 2011 data, so they do not confirm one another.
  • The size limit moves with the counting rule and the data set (a centre of 44, 49 or 65 elements in three data sets, Störrle 2016b), so one fixed number overstates the finding.
  • The size correlation is medium (r = -0.270) and shrinks for experts and for the "elite" group (Störrle 2016b).

Against UML as the baseline

  • Scanniello 2018 suggests analysis-phase models may harm comprehension.
  • Arisholm 2006 found no time saving and a high cost of keeping diagrams current.
  • Practitioners rarely draw to the standard (Baltes 2014, Petre 2013).
  • The C4 owner advises keeping UML if it works (Brown 2026c).

What the Q/A threads add against UML as the baseline

  • SESE 2010g, SESE 2012i, SESE 2017g, SESE 2023a, SESE 2015e and SESE 2015d (details in review.md 4) agree that practitioners draw UML selectively or not at all. They are opinions, not counts.

Against counting adoption

  • Stars and downloads measure installation, not understanding (Mermaid 2026, npm 2026).
  • Hebig 2016 found most UML diagrams created once at project start, which suggests drift.
  • Engineers in a C4 adoption name limited control over layout and expect maintenance trouble (Jongeling 2025).
  • Diagrams go stale and are generated where possible (SESE 2019b, SESE 2017e); Q/A sites hold little on the newer tools, so they add no adoption count.

Against the evidence on this page

  • Several results here were read only in abstracts and are marked (secondary); eleven formal sources were not read at all (sources.md).
  • Vendor and C4 pages are their owners' claims about their own method; eleven of 51 grey sources score below 10 of 20.
  • AWS 2026b admits its default grey falls below the contrast it cites, so a guide can break its own rule.
  • One reviewer scored the grey sources; no second rater checked them.