Compare with each part alone
Claim: a claim that a person with a model does better sets the combination against the person alone and the model alone.
- Grade: measured for the finding that a combination does worse, on average, than the better part alone (106 experiments, secondary); convention for the rule that every claim carries the three arms.
- Evidence: Vaccaro 2024, in against and state of the art (abstract read); Dell'Acqua 2023, in against (abstract read).
- Check:
three-arms(heuristic: a table that names one arm names all three). - Reader styles: none.
What it asks
- A table or a figure that reports a gain has three arms: the person alone, the model alone, and both.
- The task is named as a decision or as content creation, because the result differs by type.
- A source that has only the combination is an upper bound on one side, not a test of the claim.
Why
- Across 106 experiments, the combination did worse than the best of either alone, Hedges' g of -0.23 (secondary); it lost on decision tasks and gained on content creation (Vaccaro 2024).
- The combination gained when the person was the better alone, and lost when the model was (Vaccaro 2024).
- Outside the model's reach, workers using it were 19 percentage points less likely to be correct (Dell'Acqua 2023, secondary).
- Single-arm gains exist, measured on writing and coding tasks (Noy 2023; Peng 2023). They are upper bounds, since they do not set the combination against the better part.
Limits
- The authors of the meta-analysis name possible publication bias and varied designs.
- The effect is an average; a given task can lie on either side.
- The rule that all three arms appear is a convention. The measured part is the average loss, not the rule.
- The check reads tables only. It cannot tell a claim of gain from a glossary, so it warns.
- The family has no case yet measured in three arms; the cases member holds that work.