Artificial Analysis releases AA-PersonalityBench to evaluate model personality and expression style
Artificial Analysis has released AA-PersonalityBench, using 900 curated questions to evaluate differences among language models across 30 personality traits and 21 communication characteristics. Personality traits are scored by LLM judges through pairwise comparisons and aggregated into eight indices: expressiveness, organization, informativeness, transparency, agreeableness, assertiveness, imaginativeness, and Slop; communication characteristics are computed programmatically across four categories of metrics: length, structure, punctuation, and emphasis. Results show that Claude Fable 5.1 stands out on multiple indices including expressiveness, informativeness, transparency, agreeableness, and assertiveness, while Qwen3.8 27B averages about 1,030 words per response, nearly four times the 270 words of GPT-6 Luna.
Why read this
AI score 62
The benchmark measures the correctness of model answers separately from the user experience, and provides specific performance of each model on eight personality indices.
Topics
Body
Announcing AA-PersonalityBench, our new benchmark of the personality of language models across 30 traits and their communication style across 21 characteristics
Two models can give equally correct answers and still feel very different to use. AA-PersonalityBench measures those differences so people can understand which to use in different use-cases. Some use-cases like creative writing might demand imagination, while legal use-cases might demand directness. We evaluate models’ answers to 900 curated questions to judge subjective Personality Traits and programmatic means to calculate objective Communication Characteristics.
Personality Traits & Indexes. There are 30 personality traits (e.g., eloquent, pragmatic, sycophantic) which we roll up into eight aggregated Personality Indexes: Expressiveness, Organization, Informativeness, Transparency, Warmth, Conviction, Imagination and Slop. Each trait has a set of 30 dedicated questions and models’ answers are compared pairwise by a panel of LLM judges. We then z-score and percentile rank the trait and personality Elos.
Models with the strongest representation across the 8 Personality Indexes: ➤ Expressiveness: Claude Fable 5.1, GLM-5.3, GLM-5.3-Flash ➤ Organization: DeepSeek V4.1 Flash, Step 5 Preview, Inkling ➤ Informativeness: Muse Spark 1.3, Claude Fable 5.1, Muse Wonder, Claude Opus 5.5 ➤ Transparency: Claude Sonnet 5.5, Claude Fable 5.1, GPT-6 Astra ➤ Warmth: Muse Wonder, Muse Spark 1.3, Claude Fable 5.1 ➤ Conviction: Claude Fable 5.1, DeepSeek V4.1 Flash, Grok 4.7 ➤ Imagination: Muse Spark 1.3, Claude Opus 5.5, GLM-5.3 ➤ Slop: Llama 4 Maverick, GPT-4o, K2 Horizon
Communication Characteristics. There are 21 communication characteristics (e.g., average sentence length, emoji use, em dash usage) which we group into four categories: length, structure, punctuation, and emphasis. We compute these characteristics programmatically across all model responses. ➤ Length: Qwen3.8 27B averages about 1,030 words per response, nearly four times GPT-6 Luna’s 270 ➤ Structure: Nemotron 3 Ultra averages nearly 3 tables per response, while most models rarely use one ➤ Punctuation: GLM-5.3 averages 7 em dashes per response, while Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 almost never use one ➤ Emphasis: K2 Horizon averages over 2 emoji per response, while most models almost never use one