Against: limits, controversies and where the review is weak

The evidence against a rule or a reading, kept here and not deleted. Terms are in frame.

Shape: against

Against trusting a report written from inside

A case study is by the standard's own scope a study with observation; an experience report is excluded from it (Ralph 2020). A case written by its participants is outside what that standard checks.
The baseline's checklist assumes a researcher apart from the case. Its items on triangulation and on the author's intentions (items 48 and 49) are the hard ones when the author was in the case.
Experience reports have a known failure, the "how we did it" account with no lessons, and another, reporting only success (ICSE 2003).
Company write-ups of agent work are the owner's claim about itself and report no interventions (Carlini 2026).

Commits are not equal units: large commits differ in intent from small ones (Hindle 2008, secondary).
A history is not a record: commits move, are rewritten and are hidden (Bird 2009, secondary). Our own case shows it: 434 commits summed over seven repositories, 415 distinct by author time and subject.
Labelled data is a biased part of the whole (Bird 2009b, secondary).
A repository is not a project, and a project may work elsewhere (Kalliamvakou 2014).
A release is a tag in history, not a published release on the host; the program cannot see the second.

The one randomised trial found a slowdown of 19% for experienced developers on their own repositories, against a forecast saving of 24% (Becker 2025). A controlled task can show a speed-up (Peng 2023, secondary). The two designs differ in people, tasks and tools; they do not contradict each other.
Two datasets disagree on acceptance because they cover different populations (Li 2025b; Watanabe 2025, secondary).
Time per task was self-reported (Becker 2025).

One reviewer, 42 sources, 22 read by abstract and 5 located only.
The searches were few: four formal strings, of which one failed and one returned zero for a wrong query. Most formal works were recalled, which favours well-known work.
Web pages were read through a summary; their numbers are secondary.
A search for reviews of participant-written AI reports found none; that is a result of this search, not a claim about the world.

A sample of one. The standard lists it as an invalid criticism of a case.
The lack of a replication package for qualitative data (Ralph 2020). The measures of the first case have a program.