Why a Single AI Grading Its Own Work Can't Be Fully Trusted

In one test case from a 2025 study, a user asked two AI assistants the same simple question: "Please list the blue items." GPT-3.5-Turbo just listed them — jeans, blueberries, a backpack, ten items in all. GPT-4 opened with a disclaimer ("As a language model AI, I can't see the physical objects around you") before listing its own ten items and asking the user to clarify. Both lists were reasonable. Humans preferred the direct one. GPT-3.5-Turbo, judging the pair itself, also preferred the direct one.

GPT-4, judging the same pair, preferred its own answer.

A name for the pattern: self-preference bias

This is called self-preference bias: the tendency of a model, when asked to judge which of two answers is better, to rate its own output more favorably than an independent evaluator would. It matters well beyond academic benchmarks — it shows up any time a single AI is put in the position of grading its own output, which is precisely what happens when you ask a model to double-check its own reasoning, critique its own draft, or simply take its own confidence at face value.

The study

Researchers at SB Intuitions — Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri — published Self-Preference Bias in LLM-as-a-Judge in 2025, addressing a gap they identified directly:

"There is a lack of established methods to measure the self-preference bias quantitatively, and its underlying causes are poorly understood."

They borrowed a concept from algorithmic fairness research — Equal Opportunity, normally used to check whether a classifier treats different demographic groups consistently — and repurposed it to compare how a model treats its own outputs versus everyone else's. Using the Chatbot Arena dataset (33,000 real conversations, each with a pair of AI responses and a human judgment of which one was better), they had eight different LLMs act as judges on the exact same pairs humans had already scored, then measured how far each model's self-ratings drifted from human judgment.

What they found

The self-preference bias score for each model (0 = no bias, closer to 1 = strong bias toward itself):

• GPT-4 — 0.52
• Vicuna-13B — 0.25
• Koala-13B — 0.15
• GPT-3.5-Turbo — 0.03
• Vicuna-7B — 0.05
• oasst-pythia-12B, dolly-v2-12B, stablelm-tuned-alpha-7B — slightly negative (these models, if anything, rated their own answers a little too harshly)

GPT-4's confusion matrix tells the concrete story: when a human rated GPT-4's answer as the better one, GPT-4 agreed 94.5% of the time. When a human rated GPT-4's answer as the worse one, GPT-4 still sided with itself 42.5% of the time — nearly a coin flip on cases where an independent judge had already called it wrong.

The cause isn't ego — it's familiarity

The more useful finding is why. The researchers hypothesized that models favor text with lower perplexity — text that reads as more predictable, more "fluent" to the model itself — and tested it directly. Across nearly every model studied, evaluation scores tracked perplexity far more tightly than they tracked whether the text was self-generated:

"LLMs assign significantly higher evaluations to outputs with lower perplexity than human evaluators, regardless of whether the outputs were self-generated. […] the self-preference bias exists because LLMs prefer texts more familiar to them."

In other words, a model isn't (consciously) rooting for itself. It's rooting for whatever sounds like something it would say — and its own writing sounds like that by definition. The paper is careful to note this bias tracks style, not factual accuracy: in the blue-items example above, neither answer was wrong. GPT-4 simply rated its own habit of hedging and asking for clarification as the higher-quality choice, a policy preference dressed up as a quality judgment.

What this actually means for how you use AI

The paper's own proposed fix is the part worth sitting with:

"One possible approach is ensemble evaluation using multiple models […] this method is expected to provide a more equitable evaluation by avoiding reliance on a single model."

That's not a call for a smarter model. It's a call for a genuinely different one — evaluating the same thing without sharing the same blind spot. A single AI checking its own work is grading against its own habits, not against an outside standard. It can't see its own hedge, its own stylistic tic, its own comfortable phrasing as a choice at all; to that model, it just looks like the obviously right answer.

This is the whole premise behind putting multiple independent models in the same conversation instead of asking one model to mark its own homework. See our Best Practices guide for how to actually set that up — which providers to mix, and where a single model's self-assessment is the weakest signal you have (Research Rooms' self-reported reliability score is a direct real-world instance of exactly this bias, which is why it's treated as the least trustworthy number on the page).

Sources

• Wataoka, K., Takahashi, T., & Ri, R. (2025). Self-Preference Bias in LLM-as-a-Judge. arXiv:2410.21819.
• Zheng, L. et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — source of the Chatbot Arena dataset used in the study above.