How do you test a model?

Start with the decision

An evaluation is useful only when it supports a real decision. Define the intended use, affected users, model version, input conditions, acceptable behaviour, unacceptable failures, baselines, resource budgets, and who owns the decision.

“Is this model good?” is not testable. “Does this model improve English and Polish support-ticket classification without increasing privacy failures, p95 latency beyond the service objective, or cost beyond the declared budget?” is closer to an evaluation contract.

Build cases from the claim

Representative evaluation needs more than random prompts. Include:

• ordinary examples weighted like intended traffic;
• boundary and ambiguous cases;
• important languages, user groups, formats, and lengths;
• rare but severe risks;
• adversarial and misuse cases derived from a threat model;
• operational failures such as truncation, malformed output, and unavailable dependencies.

Protect held-out evidence from repeated prompt and model tuning. Record dataset versions, exclusions, expected outputs, and source permissions.

flowchart LR
  D[Release decision] --> C[Claims and hard gates]
  C --> S[Representative cases and slices]
  S --> R[Versioned runs]
  R --> G[Deterministic, human, and model graders]
  G --> E[Evidence with counts and limits]
  E --> D

Choose graders deliberately

Use deterministic code for exact schemas, labels, calculations, prohibited strings, and invariants. Use humans for nuanced domain judgement when observable rubrics can be applied. Model judges can scale subjective review, but they are measurement instruments that require calibration against independent human labels and tests for position, verbosity, and self-preference bias.

Never allow one model-judge score to override a deterministic privacy or authorization failure.

Report variation and slices

Generative behaviour can vary across identical trials. Report sample counts, failures, variation, named slices, and severe events rather than one average. Compare systems on the same cases. Keep latency and cost beside quality and safety rather than hiding everything inside one composite score.

One score cannot describe a model

A benchmark score is not evidence that a product is ready. It may use different users, data, prompts, tools, risks, or success criteria. A demo is not an evaluation. A pass rate without the number of trials, excluded cases, version manifest, and failure categories is not reproducible evidence.

Model testing also cannot prove that the surrounding retrieval, harness, UI, permissions, or tools behave correctly. Those components require system-level tests.

Build a release decision from slices and failures

For a defensible model evaluation:

1. Freeze the model, tokenizer, prompt or adaptation, decoding settings, and environment.
2. Version the case set and expected evidence.
3. Dry-run the pipeline with known passes, failures, abstentions, and malformed outputs.
4. Compare a candidate with a relevant baseline on paired cases.
5. Repeat stochastic trials according to a declared schedule.
6. Inspect disagreement between graders and manually reproduce selected verdicts.
7. Publish raw counts, slices, limitations, and every critical failure outside averages.

The result should say what may proceed, for whom, under which conditions, and what new evidence would change the decision.