[05/04/2024] NLP and Chatbot on Personality Assessment (Tianjun)

Previous[05/06/2024] O*NET Reports and Documents Next[05/02/2024] Reads on Construct Validation

Last updated 1 year ago

[05/04/2024] NLP and Chatbot on Personality Assessment (Tianjun)

Development and validation of an artificial intelligence chatbot to assess personality

Abstract

Recent technological advances have allowed researchers to apply automated, language-based machine learning models as alternatives to self-reports for assessing personality. However, previous work has largely overlooked the multidimensional nature of personality and lacked indepth exploration of validity issues. In this paper, we examined novel methods for leveraging artificial intelligence (AI), natural language processing (NLP), machine learning, and automation to systematically glean personality-related information from textual data which offers rich information and reflects various aspects of personality but has been severely underutilized. We connected the five-factor (or Big Five) model (comprised of openness to new experiences, conscientiousness, extraversion, agreeableness, and neuroticism) with NLP from two angles: 1) a psychometric and construct validity perspective (i.e., the degree to which information extracted from textual data reflects personality constructs), and 2) an applicability perspective (i.e., the ability to elicit personality-relevant information from text in line with psychological and organizational principles). Innovatively, we built an interactive tool to automatically and adaptively prompt for, collect, and analyze personality-relevant topic-based (i.e., honoring the Big Five factorial structure) narrative data through conversations conducted by an AI chatbot. Results showed significant improvements in various validities of the new personality assessment tool compared to existing applications. Potential reasons for the improvement magnitudes, limitations of the current methods, and future directions are discussed.

A meta-analytic investigation of the psychometric evidence of languaged-based machine learning personality assessment

Abstract

This paper presents a meta-analytic review of the multidimensional psychometric evidence of language-based machine learning (ML) supported personality assessment, examining the reliability and construct validity, specifically convergent and discriminant validity, of the extracted scores for the big five personality domains derived from ML natural language processing (NLP) techniques. Moreover, factors that may potentially moderate the effect size correlations between traditional personality judgment using self-reports and machine-generated judgment from NLP algorithms, such as types of language data source, types of algorithms, and types of personality scales used. This study uncovered that personality scores derived from textual data using ML and NLP approaches are only partially consistent with those from traditional personality assessment, and that much psychometric evidence is lacking in existing language-based ML personality assessment applications.

Previous[05/06/2024] O*NET Reports and Documents Next[05/02/2024] Reads on Construct Validation

Last updated 1 year ago

Development and validation of an artificial intelligence chatbot to assess personality

Abstract

Recent technological advances have allowed researchers to apply automated, language-based machine learning models as alternatives to self-reports for assessing personality. However, previous work has largely overlooked the multidimensional nature of personality and lacked indepth exploration of validity issues. In this paper, we examined novel methods for leveraging artificial intelligence (AI), natural language processing (NLP), machine learning, and automation to systematically glean personality-related information from textual data which offers rich information and reflects various aspects of personality but has been severely underutilized. We connected the five-factor (or Big Five) model (comprised of openness to new experiences, conscientiousness, extraversion, agreeableness, and neuroticism) with NLP from two angles: 1) a psychometric and construct validity perspective (i.e., the degree to which information extracted from textual data reflects personality constructs), and 2) an applicability perspective (i.e., the ability to elicit personality-relevant information from text in line with psychological and organizational principles). Innovatively, we built an interactive tool to automatically and adaptively prompt for, collect, and analyze personality-relevant topic-based (i.e., honoring the Big Five factorial structure) narrative data through conversations conducted by an AI chatbot. Results showed significant improvements in various validities of the new personality assessment tool compared to existing applications. Potential reasons for the improvement magnitudes, limitations of the current methods, and future directions are discussed.

A meta-analytic investigation of the psychometric evidence of languaged-based machine learning personality assessment

Abstract

This paper presents a meta-analytic review of the multidimensional psychometric evidence of language-based machine learning (ML) supported personality assessment, examining the reliability and construct validity, specifically convergent and discriminant validity, of the extracted scores for the big five personality domains derived from ML natural language processing (NLP) techniques. Moreover, factors that may potentially moderate the effect size correlations between traditional personality judgment using self-reports and machine-generated judgment from NLP algorithms, such as types of language data source, types of algorithms, and types of personality scales used. This study uncovered that personality scores derived from textual data using ML and NLP approaches are only partially consistent with those from traditional personality assessment, and that much psychometric evidence is lacking in existing language-based ML personality assessment applications.