New exam, 13 points lost: 17 facts about AA Intelligence Index v5

On October 27, Artificial Analysis released Intelligence Index v5. AA's launch post drew 2 million views. Almost every model lost points the moment the new version went live, and Chinese models lost the most: the same models, given a new set of questions, scored 12.4 points lower on average.

Were Chinese models singled out, or were their old scores inflated? Here are 17 facts, all taken from AA's published scores and methodology.

1. A third of the exam was replaced, and most of it is hidden

25% of the v4.3.2 weight was removed and 30% of v5 is new; 60% of the weight now sits on private questions.

Removed: GDPval-AA, SciCode and the long-context test AA-LCR. Added: FrontierCode 2.0, a private coding benchmark from Cognition, and Terminal-Bench-Science, which runs research tasks in a terminal, at 15% each. HLE became the refined HLE-Diamond, CritPt became a 40-question private hard subset, and AA-Omniscience moved to v1.1.

In AA's own words, this is the biggest upgrade since v4, and the first time most of the index is built on private data.

2. Nearly every model lost points, not just Chinese ones

Of the 77 model entries scored on both versions, only 8 went up.

All 8 are top settings from Anthropic and OpenAI: Opus 5.5 max/xhigh, GPT-6 Astra max/xhigh/high/medium and GPT-6.1 Sol max/xhigh. Opus 5.5 max went from 57.6 to 58.4, Astra max from 52.7 to 56.9, Sol max from 51.8 to 54.5.

Everything else fell: Claude Sonnet 5.5 max by 1.8, Gemini 4 Argon by 2.1, Meta's Muse Wonder Full by 6.1.

3. Chinese models fell about twice as far

The 17 Chinese entries averaged -12.4; the other 60 averaged -6.8.

"Only Chinese models fell" is wrong. "Chinese models weren't hit any harder" is also wrong. The tide went out for everyone, and faster for Chinese models.

4. In the same score band, Chinese models still lost about 6 more points

Among models that scored 42–47 on v4.3.2: Chinese average -13.9, others -8.2.

Most Chinese models sat in the 42–47 band on the old index, and that band was hit hard across the board: all three settings of the US model Grok 4.7 lost 11–12 points. Compared within the band, Chinese models still lost 5.7 points more.

5. Xiaomi fell furthest

MiMo-V2.6-Pro: 46.3 → 29.9, down 16.4 points, from #27 to #45.

StepFun's Step 5 Preview fell 15.2 and Ant Group's Ling 3.1 Flash 1019 fell 14.4. Ling received its v4.3.2 score of 43.9 one day before v5 launched; the next day it was 29.5.

6. Kimi fell from #5 to #22 and is still the top open-weights model

Kimi K3.1 Pro: 55.1 → 46.0.

On the old index Kimi K3.1 Pro ranked 5th, ahead of GPT-6 Astra. On the new one it trails Opus 5.5 by 12.4 points. When AA says the gap between open weights and the closed frontier went from 3 points to 12, Kimi is the model it measures. Kimi K3.1 Pro's weights have not been released yet.

7. The best downloadable models now rank below Haiku

DeepSeek V4.1 Flash 1017: 32.5. GLM-5.3: 31.7. Claude Haiku 5.5: 36.2.

These two are the highest-scoring models with released weights. They sit 26 points behind the leader and below Anthropic's small model Haiku. That result set off the backlash on X: developer Alexey Fateev said he runs DeepSeek every day on four 3090s and doesn't "believe for a second that Haiku beats it." The post drew 2,895 likes and 170,000 views.

8. The largest single gap is on the new science-terminal test

Terminal-Bench-Science: Opus 5.5 59.0%, DeepSeek 12.9%, GLM 9.5%, MiMo 5.7%, Step 2.4%.

This test runs multi-step scientific computing tasks in a command-line environment and carries 15% of the weight. Median output tokens per v5 task rose from 35,000 to 59,000, and this test alone accounts for about a third of them. The median Chinese entry scores 4.1% here; other models score 13.6%.

Chinese models are not the only ones losing points here: Grok 4.7 scores 4.8% and Claude Haiku 5.5 only 20.0%.

9. Seven of the 26-point gap come from that one test

Recomputed from AA's weights and per-eval scores: on this test alone DeepSeek loses 6.9 index points to Opus, GLM 7.4, Qwen 7.1, MiMo 8.0.

These four models trail Opus 5.5 by roughly 25–28 points in total, and the science-terminal test accounts for about 27–30% of that. The second-largest gap is HLE-Diamond at 4–5 points; the third is FrontierCode 2.0 at about 3.

Kimi is the exception. Its gap is only 12.2 points, spread across tests: 3.7 on the science-terminal test, 3.1 on HLE-Diamond and 3.1 on FrontierCode.

10. HLE became HLE-Diamond: the top went up, Chinese models went down

Opus 5.5: 61.4% → 67.9%. GLM-5.3: 42.3% → 17.9%. MiMo: 49.4% → 15.2%.

Chinese models fell 18.4 percentage points on average on this test; other models fell 8.4.

Part of this is structural. The old HLE used 2,158 text-only questions. HLE-Diamond has 1,000, and 272 of them include images. Models without image input receive only the text but are still graded on all 1,000. For a text-only model, that penalty is fixed before the test starts.

11. The three removed tests were where Chinese models were closest

GDPval-AA: Opus 5.5 1866, Kimi 1858, DeepSeek 1746, GPT-6 Astra 1584.

The top three on the long-context test AA-LCR are all Chinese: Step 88.3%, Kimi and MiMo 86.3% each, against 84.7% for Opus 5.5. On SciCode, MiMo scores 60.9% and Kimi 60.8%, 6 points behind Opus but ahead of GPT-6 Astra.

Together these three made up 25% of the old index. Removing them took away ground where Chinese models held their own.

12. New coding test: the best open model is 18 points behind

FrontierCode 2.0: Opus 5.5 46%, DeepSeek V4.1 Flash 28%.

This is Cognition's private benchmark, graded on whether a repository maintainer would merge the agent's change. GPT-6 Astra, Terra and Sonnet all score 42%; Kimi 25.5%, GLM 21.6%, Step 12.0%.

13. The knowledge test changed too, and two Chinese models went negative

AA-Omniscience v1.1: MiniMax M3.1 Flash went from 16 to -6, MiMo V2.6 Pro from 8 to -5.

Scores on this test can be negative. v1.1 removed saturated and publicly available questions and added new ones. The average score fell only 1.9 points (median 0.6); both of the biggest drops AA named are Chinese models.

14. The new #1 is a model you can't use

Gemini 4 Carbon joined the index on October 28 at 59.4, one point above Opus 5.5.

It leads 4 of the 9 tests: HLE-Diamond, the science-terminal test, knowledge accuracy and AutomationBench. Its hallucination rate is 11%, against 50% for both Opus 5.5 and GPT-6 Astra. But in AA's words, Carbon "is being rolled out to selected users and is not yet publicly available," and the leaderboard tags it "No publicly available API."

15. Around 32 points, the price varies 17-fold

Cost per index task: DeepSeek V4.1 Flash $0.60, Qwen3.8 Max $10.02.

Their scores differ by just 1.1 points, and Qwen3.8 Max costs almost exactly what the #2 model, Opus 5.5 max, costs ($10.04). At the other end, GPT-6.1 Sol max scores only 4 points below Opus at one-eighth of the cost: $1.27 per task.

16. The same model can score 24 points apart depending on effort

Claude Opus 5.5: 34.5 on low ($0.98), 58.4 on max ($10.04).

GPT-6.1 Sol runs from 34.5 on low to 54.5 on max; its spread grew from 10 points on v4.3 to 20. v5 tasks are longer, so each extra step of reasoning budget buys a noticeably higher score. Which setting a vendor quotes now matters more than it did.

17. Half was froth, half is a real gap

Private questions squeeze out any benefit from memorised public questions, but the gaps on the science-terminal and new coding tests have nothing to do with memorisation.

The Chinese account "非典型 CFO" put it best: half of the drop is froth disappearing, half is a real gap showing through. The ruler got stricter, and the referee has even less oversight.

Nathan Lambert is split 50/50: private benchmarks often show open models further behind, but with a private benchmark there is no community to catch the bugs that will inevitably be there. Others think the new ranking matches hands-on experience. Japanese developer @ymg_aq says Opus 5.5 leads on performance, GPT-6.1 Sol on value, and Chinese open models are not as strong as the old benchmarks suggested.

The new #1 has no public API. Among models you can download, the best ranks 38th.

Sources: Artificial Analysis (v5 launch article, methodology page and leaderboard, fetched 2026-11-01); v4.3.2 scores from the Wayback snapshot of 2026-10-26 01:02 UTC; AA's Gemini 4 Carbon write-up (2026-10-28); AA's AA-Omniscience v1.1 thread; Interconnects. Ranks count each reasoning setting as its own entry.