DeepSeek V4 Pro Goes GA: Third-Party Benchmarks Don't Match the Leaked Scorecard

On August 13, DeepSeek quietly swapped deepseek-v4-pro in its API for DeepSeek-V4-Pro-0813. No launch event — just a version number update in the docs. The April preview that had been running for nearly four months finally went GA.

The same day, an agent benchmark comparison table started circulating on social media. The eye-catching line: Terminal Bench 2.1 — DeepSeek 87.9, Claude Opus 4.8 at 85.0. A Chinese model surpassing the best closed-source model on agent coding — the claim spread fast.

Also the same day, Artificial Analysis published their independent evaluation of 0813. I lined the two sets of numbers up side by side. The critical row doesn’t match.

Two Different Data Sets#

• The leaked agent benchmark table: granular per-item scores, but no verifiable source and no third-party reproduction.
• Artificial Analysis independent evaluation: run by a third party on the model explicitly identified as DeepSeek V4 Pro 0813.

Only one of these was actually run by someone else on the real model. So the right approach isn’t “show both and let readers decide” — it’s use the verifiable set to check the unverifiable one.

Independent Benchmark Position: 53 Points, Rank 23#

AA’s Intelligence Index combines 9 evaluations: Terminal-Bench 2.1, Humanity’s Last Exam, GPQA Diamond, SciCode, long-context reasoning, and more.

0813 scores 53 out of 605 models, ranking 23rd.

Ahead of it: Claude Opus 5 (63), Claude Fable 5 (62), GPT-5.6 Sol (61), Grok 4.6 (61), Kimi K3 (60), Qwen3.8 Max (58). Tied at 53: GLM-5.2, GPT-5.4, GPT-5.6 Terra.

Three facts you can read directly off the chart:

Real improvement over its own predecessor. The April V4-Pro (max) scored 44 on the same index; 0813 scores 53 — a 9-point gain measured by a third party.

Still a tier below the best. 53 vs. Opus 5’s 63 is a 10-point gap. On the same leaderboard, 53 roughly matches Claude Opus 5 at “low thinking intensity.” The “caught up with frontier closed-source” narrative doesn’t hold on aggregate metrics.

Not leading among Chinese models either. Kimi K3 at 60 and Qwen3.8 Max at 58 are both ahead. 0813 ties with GLM-5.2.

The Critical Row: 87.9 or 79%?#

Terminal-Bench v2.1 was independently tested by AA, so we can directly compare.

AA measured DeepSeek V4 Pro 0813 = 79%. The leaked table says 87.9.

That’s nearly a 9-point gap. More importantly, the ranking flips:

The leaked table’s most viral claim — “surpassed Opus 4.8” — is reversed in independent testing. On AA’s chart, 0813’s 79% ranks 7th from the bottom among 28 frontier models.

I initially assumed this might be a uniform offset from different test harnesses — different retry strategies can shift scores by a few points. But cross-checking the other models killed that explanation:

Competitors deviate by 0–3 points. Opus 4.8 matches exactly. Only DeepSeek’s own row is inflated by nearly 9 points — three times the deviation of any other model. This is not explainable by environment differences.

On HLE (Humanity’s Last Exam), AA also tested independently:

AA measures 0813 = 39%. The leaked table says 42.7. A 3.7-point gap, same direction — both show DeepSeek trailing. 0813’s 39% is second-to-last on the chart.

The pattern emerges: on benchmarks where DeepSeek is already behind, the self-reported numbers roughly match independent testing. On the benchmark where it claims to lead, the self-reported number is inflated just enough to turn a 6-point loss into a 2.9-point win.

The Rest of the Leaked Table#

The table includes eight or nine items that AA didn’t test independently — no way to cross-check.

I won’t treat these as conclusions, but the claimed improvement magnitudes are worth noting:

DeepSWE going from 12.8 to 62.7 — nearly a 5x jump. If real, it would indicate heavy post-training reinforcement on tool use and repository tasks.

But given the Terminal-Bench cross-check, I’d treat these as “unverified.” The composite score only improved by 9 points (44 → 53). If multiple agent sub-benchmarks truly improved by 20–50 points each, the composite wouldn’t move just 9.

Two Numbers That Are Easy to Miss#

Verbosity. AA’s page shows 130M tokens for Verbosity, against a peer median of 69M. To achieve its 53-point score, 0813 generated nearly twice the output tokens of comparable models. That means slower responses and — if billed per output token — higher real costs than the unit price suggests.

Cost per task. AA has a chart specifically for this — the true cost once inference overhead is factored in:

GPT-5.6 Luna $0.05, GLM-5.2 $0.32, Gemini 3.6 Flash $0.56, Kimi K3 $0.84, Claude Opus 5 $2.34, Claude Fable 5 $3.14.

But 0813’s entry is blank — AA hasn’t ingested its pricing yet. Input/Output Price both show $0.00, Speed shows N/A. Don’t mistake $0.00 for free; the data simply isn’t there.

One more detail: AA labels 0813 as Proprietary. The April MIT-licensed weights are a different build. The 0813 weights haven’t been released. If you want to self-host 0813, that option doesn’t exist yet.

Bottom Line#

Checking the verifiable parts:

0813 is a genuine improvement over the preview — a 9-point independent composite gain is not trivial. No question there.

But it hasn’t caught the frontier, and it doesn’t lead among Chinese models. Kimi K3 and Qwen3.8 Max are both ahead.

The leaked table can’t be cited at face value. Its numbers for competitors are reasonably accurate, but its own numbers on the critical benchmark are inflated by nearly 9 points — just enough to rewrite a 6-point loss as a 2.9-point win.

Benchmarks and real-world feel are different things. The real verdict will come from users putting 0813 to work in production.

Check it yourself. The benchmarks and prices used above sit side by side on the model comparison page, and the LLM cost calculator converts them to your own token volume. A number computed from what you actually send is closer to your invoice than any vendor table — which is the whole reason not to take the leaked one at face value.