In this paper, we introduce a survey-based methodology to audit LLM-generated personas by simulating 200 US residents and collecting responses to socio-demographic questions in a zero-shot setting. We investigate whether LLMs default to standardized profiles, how these profiles differ across models, and how conditioning on specific attributes affects the resulting portrayals. Our findings reveal that LLMs often produce homogenized personas that underrepresent demographic diversity and that conditioning on attributes such as gender, ethnicity, or disability may trigger stereotypical shifts. These results highlight implicit biases in LLMs and underscore the need for systematic approaches to evaluate and mitigate fairness risks in model outputs.
In this paper, we introduce a survey-based methodology to audit LLM-generated personas by simulating 200 US residents and collecting responses to socio-demographic questions in a zero-shot setting. We investigate whether LLMs default to standardized profiles, how these profiles differ across models, and how conditioning on specific attributes affects the resulting portrayals. Our findings reveal that LLMs often produce homogenized personas that underrepresent demographic diversity and that conditioning on attributes such as gender, ethnicity, or disability may trigger stereotypical shifts. These results highlight implicit biases in LLMs and underscore the need for systematic approaches to evaluate and mitigate fairness risks in model outputs.
Mining Impersonification Bias in LLMs via Survey Filling / Bombieri, M.; Rospocher, M.. - In: INFORMATION. - ISSN 2078-2489. - 16:11(2025). [10.3390/info16110931]
Mining Impersonification Bias in LLMs via Survey Filling
Bombieri M.;
2025-01-01
Abstract
In this paper, we introduce a survey-based methodology to audit LLM-generated personas by simulating 200 US residents and collecting responses to socio-demographic questions in a zero-shot setting. We investigate whether LLMs default to standardized profiles, how these profiles differ across models, and how conditioning on specific attributes affects the resulting portrayals. Our findings reveal that LLMs often produce homogenized personas that underrepresent demographic diversity and that conditioning on attributes such as gender, ethnicity, or disability may trigger stereotypical shifts. These results highlight implicit biases in LLMs and underscore the need for systematic approaches to evaluate and mitigate fairness risks in model outputs.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione
