Fiction Eval leaderboard · report

The July 2026 Fiction Model Report

8 frontier models, 10 fresh genre scenes, three tries each, 819 blind matchups, judged by a different AI family, every score with a likely range.

Every model-launch cycle produces the same question in writing communities — "which model actually writes fiction best?" — and the same answers with no receipts. This report is the receipt: every ranking below comes from blind head-to-head matchups on scenes written fresh for this run (so they can't be in anyone's training data), graded by a model that never shares a family with either contestant, with a likely range on every score.

The board

#	Model	Score	Record	Slop
1	Claude Opus 4.8	1927 (likely 1888–1976)	150–14–43	14.88
2	Claude Sonnet 4.6	1914 (likely 1874–1961)	146–15–46	18.57
3	Grok 4.3	1856 (likely 1815–1899)	137–28–42	9.43
4	GLM 5.2	1776 (likely 1732–1822)	117–42–42	17.53
5	Gemini 3.1 Pro	1471 (likely 1423–1519)	78–104–13	13.45
6	DeepSeek V3.2	1251 (likely 1201–1302)	52–144–11	23.24
7	Mistral Large	1118 (likely 1054–1167)	35–162–10	17.79
8	Llama 4 Maverick	686 (likely 627–741)	0–206–1	22.92

What stands out

Claude Opus 4.8 and Claude Sonnet 4.6 are tied at the top. They're close enough that it's a coin flip — re-run the numbers and the two swap places about 33% of the time. Anyone telling you one of these two is definitively better at fiction is reading noise.
Grok 4.3 is genuinely third — with three tries per scene it sits clearly below the top pair, not tied with them.
Grok 4.3 writes the cleanest prose in the field (9.43 slop per 1,000 words) — but clean and good aren't the same thing, which is exactly why the board scores them separately.
GLM 5.2 debuts at #4, above Gemini 3.1 Pro, in its first appearance on the board.
Llama 4 Maverick lost every decisive matchup it played (0–206–1).

Method, briefly

Ten genre scenes (dungeon-core LitRPG, cozy-mystery cold open, enemies-to-lovers turn, air-traffic-control thriller, xianxia sect betrayal…), three passages per model per scene. Every pair of models is judged on the same scene twice — once in each order, so being shown first can't help — by a judge from a different AI family. All the wins and losses become one strength score (shown like a chess rating), and re-running the numbers many times gives the likely range next to each. A separate fixed checklist counts AI tells — the "it wasn't X, it was Y" crutch, essay words, stock phrases, filler — per 1,000 words.

Full excerpts, the blind-vote widget, the paste-your-prose AI-tell checker, and the FAQ are on the board. Genre-level boards: cozy mystery · fantasy · LitRPG · progression fantasy · romance · science fiction · thriller · cultivation / xianxia.

How it works: every pair of models is judged blind on the same scene, with the passages' order flipped so being shown first can't sway it, by GPT-5.4 — a family that isn't on the board, so nobody scores their own side. Each score carries a likely range; overlapping ranges are called a tie. Slop is scored separately by a fixed checklist, not an AI. Full board, every prompt, and the FAQ: thebookfactoryai.com/board. Get each new board by email on the model-drop list.

Writing a book of your own? Book Factory runs the same craft checks on full manuscripts.