Who judges? And how?

There are three mainstream approaches to scoring today:

- Code-based Evaluators

- LLM-as-a-Judge

- Human Evaluation

Code-based Evaluators

Code-based Evaluators are suited for tasks with hard rules, explicit states, or computable answers.

This type of evaluation is fast, low-cost, and produces stable results — ideal for CI/CD pipelines as regression tests after every change to the model, prompt, tools, or workflow.

For task execution specifically, evaluation should care about the final result or system state, not just what the model said. For tasks with clearly defined answers, you can directly check whether the model output matches the expected result.

During R&D, we often need to quickly identify an optimization direction — choosing a model, modifying a prompt, or tuning hyperparameters. This process requires running a large number of experiments. If every round depends on human annotation and scoring, the evaluation cycle, labor cost, and subjective variance all become confounding factors, and experiments become hard to reproduce reliably.

This is where Code-based Evaluators come in: scoring through automated, standardized, reproducible, and quantifiable methods.

At its core, this involves two parts: datasets and evaluation metrics. Datasets were covered in the previous article, so this one focuses on metrics.

How to Choose Metrics

Metric selection is fundamentally a modeling exercise — translating business objectives into quantifiable questions. Different task types call for different evaluation metrics.

For classification tasks, Accuracy is a common baseline metric. But in scenarios with class imbalance — medical diagnosis, for instance — relying on Accuracy alone can be highly misleading. You may need to additionally track Recall, and depending on the actual problem, choose metrics like F1-Score, MCC (Matthews Correlation Coefficient), or ROC curves.

For tasks with definitive answers — a final numerical value in a math problem, a classification label, or a fixed identifier — evaluation is relatively straightforward. Exact Match can directly check whether outputs are consistent; pass@k measures the probability that the model succeeds at least once across k samples.

Evaluating LLM-generated responses, by contrast, is more challenging.

Exact Match works for scenarios where answer expression is tightly constrained, but has clear limitations for open-domain QA. N-gram metrics like BLEU and ROUGE focus on lexical overlap and struggle to capture full semantic relationships. Methods like BERTScore compute semantic similarity via embeddings, but introduce a new model variable — and similarity scores are usually relative values that still require calibration and benchmarking.

These open-text tasks typically call for LLM-as-a-Judge.

As model capabilities grow, evaluation targets are also expanding beyond accuracy to dimensions like safety, trustworthiness, stability, and cost.

Don’t Let Metrics Replace the Task Objective

BLEU, ROUGE, and Embedding Similarity can all be computed automatically by code — but “can be computed automatically” does not mean they effectively reflect business quality.

A response with high semantic similarity can still miss a critical constraint; a response with low lexical overlap can still be entirely correct. Metrics are merely measurement proxies for business objectives. Once a proxy decouples from the real objective, optimizing the metric may actually push the system away from the problem you genuinely need to solve.

LLM-as-a-Judge

Many tasks have no single correct answer.

For example: determining whether a response is complete, whether it faithfully adheres to reference materials, whether it overlooks risks, or whether an analysis genuinely addresses the user’s question. These criteria can be described in words, but are exceedingly difficult to encode as an exhaustive set of rules.

This is where LLM-as-a-Judge comes in. Common approaches include:

  • Classification: pass/fail judgment, or categorizing the type of error.

  • Scoring: assigning a score of 1–5 against an explicit rubric.

  • Pairwise comparison: comparing version A and version B, judging which better meets the criteria.

  • (and more…)

Among these, pairwise comparison is generally more stable than assigning absolute scores. When presented with two concrete answers, humans also tend to find it easier to judge which is better, rather than precisely articulating “why this is a 3 and not a 4.”

The value of an LLM Judge lies in scale. It can perform initial screening and repeat evaluation across large volumes of open text, and can output error categories and scoring rationales, helping teams pinpoint issues.

But a Judge’s score is still a model output — a scalable approximation of judgment, one that itself needs to be evaluated and calibrated.

What to Watch Out for with LLM Judges

Judge one clear dimension at a time

“Please comprehensively evaluate the quality of this response” rarely yields stable results. Break it down into factual correctness, completeness, relevance, and safety — and write observable scoring descriptors for each. The results become far more interpretable.

Make the rubric executable

“Professional content” and “good user experience” are both too vague. A more effective rubric spells out which facts must be included, which errors count as critical failures, and what performance each score level corresponds to.

This is also why the previous article recommended breaking Golden Answers into Statements and Rubrics: having a Judge verify a clear set of decision points is far more stable than comparing whether two long passages are “similar.”

Require evidence and reasoning

Keeping only a score makes it hard to detect where the Judge went wrong. Have the Judge cite evidence from the evaluated response and output structured reasoning — this makes human spot-checking and downstream analysis feasible.

Address known biases

Judges may favor longer answers, more familiar formats, options appearing earlier in the list, and content that matches their own stylistic patterns.

When doing pairwise comparisons, randomly swap answer order to reduce position bias. For critical cases, repeat evaluation and check for consistency via self-consistency. When necessary, aggregate results from multiple Judges.

For content that is “partially correct but severely wrong in specific areas,” Judges can still miss failures. Knowledge boundary issues and hallucinations in specialized domains also require human review.

Pin and record versions

The Judge model, prompt, temperature, and rubric are all part of the evaluator. Change any one of them, and the evaluation scale may drift. Experiment logs should preserve the full version and final scores.

Calibrate with human-annotated samples

Before large-scale execution, prepare a small set of samples judged by domain experts and check the Judge’s alignment with human conclusions. For high-risk or specialized domains, ongoing spot-checking is also necessary.

Human Evaluation

Human evaluation is the most expensive and slowest approach, but it shoulders work that the other two methods cannot accomplish independently.

Human capacity for complex semantic understanding and value judgment gives Human Evaluation an irreplaceable position — and also accounts for its higher cost.

Systematic Human Evaluation, depending on the completeness of available evaluation resources, can take several forms:

  • When no ready-made dataset exists: provide complete task descriptions and structured scoring guidelines. For example, when evaluating a dialogue system, clearly define dimensions like “coherence,” “relevance,” and “safety,” and require annotators to provide scoring rationales.

  • When a dataset already exists: provide annotators with model inputs, outputs, and scoring guidelines together.

  • When both a dataset and automated scoring results exist: have human annotators review the results through Error Annotation.

Human evaluation cannot be reduced to “grab a few people and have them try it out based on gut feeling.” Systematic human evaluation requires, at minimum: task descriptions, model inputs and outputs, scoring dimensions, score definitions, and examples. When necessary, annotators should also record error categories, severity, and scoring rationales.

Compared to vague scoring, Error Annotation is often far more useful. Having domain experts point out where the error occurred, what type it belongs to, and how severe it is — the resulting information feeds directly into error analysis and eval set iteration.

What to Watch Out for with Human Evaluation

Choose people who genuinely possess judgment ability

Fluency can be left to general annotators; whether a professional conclusion is reliable requires domain experts. Expert time is scarce — concentrate it on the parts that truly demand professional judgment.

Reduce the cognitive burden of annotation

The previous article introduced a principle: favor true/false questions over multiple choice, and multiple choice over open-ended questions. Breaking complex tasks into explicit decision points lowers costs and reduces interpretation variance across annotators.

Run a small-scale pilot first

Before scaling up annotation, test a small batch of samples to check whether the instructions are clear and whether edge cases produce disagreement. If different annotators consistently fail to reach agreement, the problem likely lies in the rubric itself.

Document disagreements

Disagreement is not just noise. It may signal ambiguity in the task definition, or that the business side has yet to converge on a unified standard. Further judgment is needed on whether the disagreement stems from insufficient knowledge, unclear instructions, or genuinely different value judgments.

Evaluators Need Controlled Variables Too

Regardless of which scoring method you use, evaluation results are affected by experimental setup.

Prompts affect results. A sound prompt should include a clear task description, the context needed to complete the task, and specific, unambiguous questions.

Repeat experiments to reduce random error. Design multiple prompt variants for the same task, vary temperature, shuffle order, and so on.

Structured Output is also an experimental variable. Structured output helps improve evaluation controllability and reduces extraneous content, but can also alter the model’s output distribution. When your actual deployment uses JSON, Schema, or Tool Calling, evaluation should ideally run under the same interface conditions.

Summary

Code-based Evaluators are suited for tasks that can be directly verified through hard rules, definitive answers, or system state.

LLM-as-a-Judge is suited for tasks that require open-ended semantic judgment at scale — but it carries its own randomness, biases, and scale drift.

Human Evaluation is responsible for defining standards, calibrating automated evaluation results, and handling samples that are high-stakes, highly specialized, or contentious.