Using cognitive psychology to understand GPT-like models needs to extend beyond human biases

Binz and Schulz’s evaluation of Large Language Models (LLMs) via cognitive tasks propelled the adoption of psychological frameworks for testing LLMs’ knowledge. Binz and Schulz found that GPT3 (generative pretrained transformer-3) often responded to cognitive tasks analogously to humans, showcasing human-like biases. However, the authors mentioned that their results could occur not only because of similarities between the cognitive processes at work within LLMs and humans but also because LLMs may report human-made content, which “might have been part of its training set”.