原始内容
Introducing six new Artificial Analysis Capability Indices for comparing model capabilities across key industry domains
The new industry indices cover Finance & Accounting, Legal, Healthcare & Medical, Strategy & Ops, Engineering, and Economics. We aim to capture the common capabilities required across knowledge work domains and evaluate how well current models meet those needs.
Each index is grounded in common tasks from O*NET occupational classifications. Tasks range from financial modeling, to legal research and contract review, to clinical decision support and patient documentation. We derive capabilities from each task, select the benchmarks that best represent the work, and weight by how often each capability appears across the domain. This means rethinking the Artificial Analysis benchmark suite for each domain and slicing evaluations to relevant domain tasks. Every component benchmark is run independently by Artificial Analysis.
The industry indices join the existing skill-based Agentic and Coding indices, which measure capabilities that cut across every domain.
Key Results
➤ Leading models: Claude Fable 5 (with Opus 4.8 fallback) leads all eight indices, with Claude Opus 4.8 (max) in second on six of eight Capability Indices and GPT-5.5 (xhigh) on two. Below the top two, rankings reshuffle substantially by domain between Gemini 3.5 Flash, Gemini 3.1 Pro Preview, GPT-5.5 (xhigh), Claude Sonnet 5 (max), and GLM-5.2 (max).
➤ Open weights leading models: Among open weights models, GLM-5.2 (max) leads on five of the six industry indices, ranking as high as fifth overall on the Artificial Analysis Engineering Index (53), within 2 points of Claude Sonnet 5 (max, 55) and GPT-5.5 (xhigh, 55). DeepSeek V4 Pro (max, 38) takes the open weights lead on Artificial Analysis Strategy & Ops Index.
➤ Cost efficiency: DeepSeek V4 Flash (max) completes tasks for <$0.04 across all six indices while scoring mid-pack, and GLM-5.2 (max) leads open weights score with a Cost per Task of $0.26 to $0.58. Frontier capability comes at a steep premium: on the Artificial Analysis Strategy & Ops Index, Claude Fable 5 (with Opus 4.8 fallback, $3.48) scores 12 points above DeepSeek V4 Pro (max, $0.03) at over 100x the Cost per Task.
➤ Time per Task: Time per Task spreads roughly 15x within each index, from 1.1 minutes for Nova 2.0 Pro Preview (medium) to 16.7 minutes for Claude Sonnet 5 (max). Speed shows a similar frontier to cost: on the Artificial Analysis Legal Index, Gemini 3.1 Pro Preview (0.8 minutes) completes tasks ~7x faster than Claude Fable 5 (with Opus 4.8 fallback, 5.4 minutes), while scoring within 11 points.
Artificial Analysis (@ArtificialAnlys): Explore all Capability Indices, with full weightings and component benchmarks, on the Artificial Analysis website: https://t.co/0jSq4HtZAr
Finance & Accounting: https://t.co/cu82nCqSOe
Strategy & Ops: https://t.co/Z8Y8NBIYvx
Legal: https://t.co/qbjxPHcaLs
Healthcare & Medical: https://t.co/mCWo4lRzlF
Engineering: https://t.co/wzjAfywmxG
Economics: https://t.co/qjCNODzuig
As always, our full methodology is public. See how evaluations are run, scored, and weighted on each index: https://t.co/edIbUixO8U