How can you tell a benchmark is contaminated?

a suspiciously high score with weakness on fresh/paraphrased tasks of the same type; checking n-grams against the training corpus.

What biases does an LLM judge have and how do you catch them?

position (rerun with A/B swapped), length (correlation of the verdict with length), self-preference (compare judges from different model families).

How do you evaluate an LLM feature in production?

a golden set of real queries, rubrics, an LLM judge calibrated against humans, regression runs for every release.

Model A beat B in 6 out of 10 comparisons

is it better?” — no, the sample is tiny: the confidence interval covers 50/50.

Home · LLMs: Training and Inference · Updated October 26, 2026

Evaluating LLMs: Benchmarks and LLM-as-Judge

The task “write a letter to a client” has no single correct answer — so evaluating an LLM is harder than evaluating a classifier. We cover benchmarks and their diseases, Elo arenas, LLM judges with their biases, and how to build an eval for your own product.

Intuition: why this is hard

In classical ML there is an exact answer: a class label, a number. A generative model has infinitely many “correct” answers, and “goodness” is multidimensional: factual accuracy, completeness, tone, safety, format adherence. You cannot compute accuracy head-on, and the industry has built a whole ladder of trade-offs between cheapness and trustworthiness of evaluation.

Automatic benchmarks and their diseases

The first floor is benchmarks with checkable answers: MMLU (multiple-choice tests across 57 subjects), GSM8K (school math problems with a numeric answer), HumanEval (code generation checked by unit tests). Cheap, reproducible, convenient to compare. But benchmarks have two chronic diseases:

Contamination — tasks leaked into the training data (they sit on the open internet), and the model “remembers” answers instead of solving. A pretty number says nothing about ability.
Saturation — top models score 90%+, and the benchmark stops distinguishing them from each other.

The second floor is leaderboards based on human preferences: LMArena shows the user two anonymous answers, they vote for the better one, and from millions of pairwise comparisons an Elo rating is computed (as in chess). It is more robust to contamination, but it measures “what people like in chat”, not quality on your task.

LLM-as-judge and its biases

Humans are expensive and slow, so answers are increasingly judged by a strong LLM: the judge (LLM-as-judge) receives a question, a pair of answers and a rubric, and outputs a verdict. Cheap and scalable, but the judge is systematically biased:

Position bias — a tendency to pick the answer that comes first (or last), regardless of content.
Length bias — a love of long, elaborate answers, even when they are watery.
Self-preference — the judge inflates scores for answers that resemble its own style (especially answers from its own model).

So the judge must be calibrated: compare its verdicts with human ones on a labeled set, run every pair twice with A and B swapped (if the verdict flips — that is position bias), check the correlation of the verdict with answer length.

💡 Key idea An LLM judge is a model too, and you must evaluate it like the main one: measure agreement with humans on a golden set and always check with an A/B swap. An uncalibrated judge measures its own biases, not your product's quality.

How to build an eval for your product

Public benchmarks will not tell you whether your bot answers your users well. A working recipe:

Golden set — 100–500 real product queries with reference answers or human ratings. This is your main asset.
Rubrics — explicit criteria for the judge (“factually correct”, “fits the format”, “did not invent links”) instead of a vague “rate from 1 to 10”.
Pairwise comparisons — “which of the two answers is better” is more reliable from both humans and judges than absolute scores.
Regression runs — every prompt release or model change is run on the golden set; metrics are compared with the previous ones. Eval is CI for quality.

Human evaluation remains the gold standard — it calibrates the judge and settles disputed calls, but living entirely on it is expensive. And everywhere you need statistical honesty: on 20 examples a “71% vs 68%” difference is noise; compute confidence intervals or at least keep sets in the hundreds of examples.

⚠️ Caution The most common mistake is trusting an uncalibrated judge. The team celebrates “+5% quality”, but in reality the new prompt just makes answers longer, and the judge's length bias counts that as an improvement. Minimal defense: measure judge–human agreement on the golden set and a control run with A/B swapped.

🎤 Interview

“How can you tell a benchmark is contaminated?” — a suspiciously high score with weakness on fresh/paraphrased tasks of the same type; checking n-grams against the training corpus.
“What biases does an LLM judge have and how do you catch them?” — position (rerun with A/B swapped), length (correlation of the verdict with length), self-preference (compare judges from different model families).
“How do you evaluate an LLM feature in production?” — a golden set of real queries, rubrics, an LLM judge calibrated against humans, regression runs for every release.
“Model A beat B in 6 out of 10 comparisons — is it better?” — no, the sample is tiny: the confidence interval covers 50/50.

All interview questions in this section: LLMs: Training and Inference — interview questions

Evaluating LLMs: Benchmarks and LLM-as-Judge

Intuition: why this is hard

Automatic benchmarks and their diseases

LLM-as-judge and its biases

How to build an eval for your product

Related topics