Word associations have been extensively used in psychology to study the rich structure of human conceptual knowledge. Recently, the study of word associations has been extended to investigating the knowledge encoded in LLMs. However, because of how the LLM word associations are accessed, existing approaches have been limited in the types of comparisons that can be made between humans and LLMs. To overcome this, we create LLM-generated word association norms mod- eled after the Small World of Words (SWOW) human-generated word association norms consisting of over 12,000 cue words. We prompt the language models with the same cues and participant profiles as those in the SWOW human-generated norms, and we conduct preliminary com- parative analyses between humans and LLMs that explore differences in response variability, biases, concreteness effects, and network properties. Our exploration provides insights into how LLM-generated word asso- ciations can be used to investigate similarities and differences in how humans and LLMs process information.
LLM-Generated Word Association Norms / Abramski, Katherine; Lavorati, Clara; Rossetti, Giulio; Stella, Massimo. - 386:(2024), pp. 3-12. (Intervento presentato al convegno HHAI tenutosi a Malmo nel 10, Giugno 2024) [10.3233/faia240177].
LLM-Generated Word Association Norms
Abramski, Katherine;Stella, MassimoCo-ultimo
2024-01-01
Abstract
Word associations have been extensively used in psychology to study the rich structure of human conceptual knowledge. Recently, the study of word associations has been extended to investigating the knowledge encoded in LLMs. However, because of how the LLM word associations are accessed, existing approaches have been limited in the types of comparisons that can be made between humans and LLMs. To overcome this, we create LLM-generated word association norms mod- eled after the Small World of Words (SWOW) human-generated word association norms consisting of over 12,000 cue words. We prompt the language models with the same cues and participant profiles as those in the SWOW human-generated norms, and we conduct preliminary com- parative analyses between humans and LLMs that explore differences in response variability, biases, concreteness effects, and network properties. Our exploration provides insights into how LLM-generated word asso- ciations can be used to investigate similarities and differences in how humans and LLMs process information.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione