Word associations have been extensively used in psychology to study the rich structure of human conceptual knowledge. Recently, the study of word associations has been extended to investigating the knowledge encoded in LLMs. However, because of how the LLM word associations are accessed, existing approaches have been limited in the types of comparisons that can be made between humans and LLMs. To overcome this, we create LLM-generated word association norms mod- eled after the Small World of Words (SWOW) human-generated word association norms consisting of over 12,000 cue words. We prompt the language models with the same cues and participant profiles as those in the SWOW human-generated norms, and we conduct preliminary com- parative analyses between humans and LLMs that explore differences in response variability, biases, concreteness effects, and network properties. Our exploration provides insights into how LLM-generated word asso- ciations can be used to investigate similarities and differences in how humans and LLMs process information.

LLM-Generated Word Association Norms / Abramski, Katherine; Lavorati, Clara; Rossetti, Giulio; Stella, Massimo. - 386:(2024), pp. 3-12. (Intervento presentato al convegno HHAI tenutosi a Malmo nel 10, Giugno 2024) [10.3233/faia240177].

LLM-Generated Word Association Norms

Abramski, Katherine;Stella, Massimo
Co-ultimo
2024-01-01

Abstract

Word associations have been extensively used in psychology to study the rich structure of human conceptual knowledge. Recently, the study of word associations has been extended to investigating the knowledge encoded in LLMs. However, because of how the LLM word associations are accessed, existing approaches have been limited in the types of comparisons that can be made between humans and LLMs. To overcome this, we create LLM-generated word association norms mod- eled after the Small World of Words (SWOW) human-generated word association norms consisting of over 12,000 cue words. We prompt the language models with the same cues and participant profiles as those in the SWOW human-generated norms, and we conduct preliminary com- parative analyses between humans and LLMs that explore differences in response variability, biases, concreteness effects, and network properties. Our exploration provides insights into how LLM-generated word asso- ciations can be used to investigate similarities and differences in how humans and LLMs process information.
2024
HHAI 2024: Hybrid Human AI Systems for the Social Good
Amsterdam
IOS Press
9781643685229
Abramski, Katherine; Lavorati, Clara; Rossetti, Giulio; Stella, Massimo
LLM-Generated Word Association Norms / Abramski, Katherine; Lavorati, Clara; Rossetti, Giulio; Stella, Massimo. - 386:(2024), pp. 3-12. (Intervento presentato al convegno HHAI tenutosi a Malmo nel 10, Giugno 2024) [10.3233/faia240177].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/437360
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
  • OpenAlex ND
social impact