TY  - JOUR
AU  - Millière, Raphaël
AU  - Rathkopf, Charles
TI  - Anthropocentric bias in language model evaluation
JO  - Computational linguistics
VL  - .
SN  - 0891-2017
CY  - Cambridge, MA
PB  - MIT Press
M1  - FZJ-2025-04990
SP  - 1 - 10
PY  - 2025
AB  - Evaluating the cognitive capacities of large language models (LLMs) requires overcoming not only anthropomorphic but also anthropocentric biases. This article identifies two types of anthropocentric bias that have been neglected: overlooking how auxiliary factors can impede LLM performance despite competence (auxiliary oversight), and dismissing LLM mechanistic strategies that differ from those of humans as not genuinely competent (mechanistic chauvinism). Mitigating these biases requires an empirical, iterative approach to mapping cognitive tasks to LLM-specific capacities and mechanisms, achieved by supplementing behavioral experiments with mechanistic studies.
LB  - PUB:(DE-HGF)16
DO  - DOI:10.1162/COLI.a.582
UR  - https://juser.fz-juelich.de/record/1048885
ER  -