[10/10/2024] Text-Based Personality Assessment with LLM
Large Language Models Can Infer Personality from Free-Form User Interactions
date: May 2024
LLMs like GPT-4 can infer people's Big Five personality traits from free-form conversational interactions with moderate accuracy.
Accuracy was highest when the chatbot was prompted to elicit personality-relevant information from users, followed by a more naturalistic conversation condition.
Directly focusing on personality assessment did not negatively impact the user experience.
Can ChatGPT Assess Human Personalities? A General Evaluation Framework
date: Oct 2023
Proposes a framework for evaluating LLMs' ability to assess human personalities using MBTI tests
Key components include unbiased prompts, subject-replaced queries, and correctness-evaluated instructions
Introduces three evaluation metrics to measure consistency, robustness, and fairness of LLM personality assessments
Experiments show ChatGPT and other LLMs can independently assess human personalities
Large Language Models Can Infer Psychological Dispositions of Social Media Users
date: June 2024
LLMs like ChatGPT can infer the Big Five personality traits of social media users from their Facebook status updates with reasonable accuracy (average correlation of r = .29 between LLM-inferred and self-reported trait scores)
The accuracy of personality inferences varies across different traits, with Openness, Extraversion, and Agreeableness showing higher correlations than Conscientiousness and Neuroticism
The accuracy of personality inferences also varies across different age groups and gender categories, with more accurate predictions for women and younger individuals on several traits, suggesting potential biases in the underlying training data or differences in online self-expression
The ability of LLMs to infer psychological dispositions from user-generated text has both opportunities (e.g., facilitating large-scale research and personalized services) and challenges (e.g., raising ethical concerns regarding user privacy and self-determination)
ChatFive: Enhancing User Experience in Likert Scale Personality Test through Interactive Conversation with LLM Agents
date: July 2024
Rediscovering the Latent Dimensions of Personality with Large Language Models as Trait Descriptors
date: Sep 2024
LLMs implicitly encode notions of personality when modeling next-token responses
Applying SVD to the log-probabilities of trait-descriptive adjectives uncovers latent personality dimensions that align with the Big Five model
The top 5 principal components explain 74.3% of the variance in the latent space and correspond to the Big Five traits
Using the derived personality factors improves the accuracy of personality trait prediction compared to fine-tuned models and direct LLM-based scoring
PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents
Last updated