Why evals matter

Ask a team that has just shipped an AI feature whether it is good and watch what happens. Someone will say it feels much better than last month. Someone else will produce a benchmark score for the underlying model. A third person will mention a demo that went badly. None of those is an answer, and the gap between them is where most enterprise AI programs quietly stall.

The public instruments have stopped helping. Stanford HAI's 2026 AI Index found that nearly half of the sixty most-cited language model benchmarks are now saturated. Frontier models clear 99 percent on MMLU and separate from each other by two or three points, which is noise rather than signal. This is not a crisis so much as a category error being corrected in public: a leaderboard was always a rough report on what the frontier can do, and it was never going to tell you whether this workflow, on your cases, produces work your people accept at a cost you can defend.

The instrument that can answer that is one you build. This piece is about why that instrument exists, why the public ones stopped working, what the enterprise version actually contains, and the two claims we would make that go slightly further than the consensus: that the evaluation set is an owned asset rather than engineering hygiene, and that nobody can honestly claim to use the best model for each task without one.

What an eval is for

Software has a compiler and a test suite. Run the tests, see green, ship. A language model has neither. The same input produces different output on Tuesday than it did on Monday, a prompt change that fixes one case breaks another silently, and a model upgrade that improves every published benchmark can degrade the one thing your team depends on.

An evaluation is the instrument that converts judgment into a number you can act on. It does three jobs, and the three are worth separating because teams usually build for one and then discover they needed the others.

Choosing. Which model, which prompt, which decomposition of the task, which retrieval strategy. Without measurement this is taste, and taste does not survive contact with a second opinion or a procurement process.

Gating. May this ship. May this workflow widen from read-only to write access. May this agent be given a second action class. A gate is an eval with a threshold and an owner attached.

Detecting. Has anything moved. Providers ship silent updates, retrieval corpora drift, an upstream schema changes, and the first signal is usually a complaint. Scheduled evaluation turns that into a number that drops before anyone notices by feel.

Why the public instruments stopped working

Four things went wrong at once, and it is worth being precise about them because each has a different implication.

Saturation. When the top ten models score within noise of each other on a test, the test has stopped discriminating. Nearly half the most-cited benchmarks are in that condition. A benchmark that everyone passes is a historical artifact, not a measurement.

Contamination. Benchmarks are public, and models are trained on the public internet. Benchmark items end up in training data, which inflates scores without improving capability, and contamination has been documented in widely used models including GPT-4 and Llama 2. As of early 2026 there is still no industry standard for detecting it, no agreed threshold, and no enforcement, so scores from different labs are not strictly comparable even when they are honest.

Optimization against the metric. In April 2025 Meta submitted a variant of Llama 4 Maverick to Chatbot Arena that had been tuned for the way humans vote in blind comparisons, verbose and heavily formatted, rather than the model it released. That is the clearest public example of a general pattern: when a number becomes the thing being sold, the number is what gets optimized. Goodhart's law is not a moral failing, it is what happens to every metric with commercial weight on it.

The construct gap. This is the deepest problem and the one least discussed. Benchmarks measure what is cheap to grade: short answers, multiple choice, problems with a key. Enterprise work is long, multi-step, judged against a standard that lives in a reviewer's head, and correct only if it also touched the right records by an allowed path. A model that tops a reasoning benchmark tells you almost nothing about whether it will follow your approval rules.

Put together: public benchmarks remain a useful weather report about the frontier, and they are not a decision instrument for your workload. Treating them as one is the actual mistake, and it predates the saturation.

The judge is an instrument too

The standard way to scale evaluation past what humans can read is to have a model grade the output. This works, and it has known defects that have to be managed rather than assumed away. Model judges show position bias, favoring whichever answer appears first; verbosity bias, rewarding length; style bias, rewarding formatting that resembles their own; and self-preference, favoring output from their own model family. In agentic settings, where a judgment on one step feeds the next, early misjudgments compound rather than cancel.

None of that makes automated grading useless. It makes it an instrument with a calibration requirement. You measure the judge against human labels on a sample, you check agreement, and you re-check it when you change judges. A judge nobody has calibrated is a number generator.

What an enterprise is actually asking

The enterprise question is never which model is best. It is whether this workflow produces work our people accept, on our cases, at a cost we can defend, without doing anything it was not allowed to do. That is a different instrument with five different properties.

	Public benchmark	Enterprise eval
Data	A public corpus everyone shares	Your cases, from your work
Standard	An answer key	What your reviewers accept
Scope	The model alone	The whole system: retrieval, tools, prompts, approval rules
Decides	Which model is roughly ahead	Whether to ship, widen, route, or stop
Decay	Saturates and gets trained on	Overfits if you never change it

The cases are yours. The set is drawn from real production traffic, with permission and stripped of what should not be in it, plus authored cases for the failure modes that actually hurt: a permission violation, a fabricated citation, a silently truncated document, a tool call nobody asked for. The authored half matters more than teams expect, because production traffic under-represents exactly the failures you most need to catch.

The standard is your reviewers'. Not a key, a rubric that encodes what an accepted piece of work looks like here. This is where the ontology earns its keep a second time: it gives the rubric a vocabulary. Was the right object touched. Was the action one this role may take. Did the state end where it should. Those are checkable without a judge at all.

Agents are scored on the path as well as the answer. A right answer reached by an unauthorized retrieval is a failure, and an end-to-end score alone will call it a success. Step-level scoring against the action boundary is what catches it.

The denominator is cost per accepted outcome. Not tokens, not latency in isolation. Accepted means a person or a downstream check signed off, and the cost includes the retries and the human correction time. That is the only number that lets you compare a frontier model, a cheaper one, and a post-trained one on the same axis.

It is a living thing. Rerun on every model change, prompt change, connector change, and scope widening, with a threshold that blocks the change rather than merely reporting on it.

The eval set is the asset

Here is where we would go further than the usual advice. Most writing treats evaluation as engineering hygiene, a thing responsible teams do, like tests. We think the set belongs on the balance sheet next to the ontology and the decision model, for the same reason.

It is created by your own work and by nobody else's. It encodes a standard that exists nowhere else in writing, because the rubric is what your best reviewers know and the labels are the corrections they made. It is the thing that stays valid when everything underneath it changes: the model gets replaced, the prompt gets rewritten, the retrieval strategy is thrown away, and the eval set is still the definition of done. And it compounds, because every workflow you add contributes cases and the next decision gets cheaper to make.

That also makes it the same kind of sovereignty question as everything else in our operating playbook. An evaluation harness that lives only inside a vendor's console, whose cases cannot be exported and whose judge cannot be pointed at a different model, is a dependency dressed as a feature. The test is the same one we apply to context: move it to a second environment, run it against a different model, and see whether the numbers still mean anything.

Model-agnosticism is unearned without it

The second claim is sharper, and it is aimed at a phrase that has become nearly universal in enterprise AI marketing, including in places where it is not backed by anything.

A great many platforms now say they use the best model for each task. That sentence is only meaningful if the speaker can produce the measurement behind it: which task classes they define, what the success bar is for each, which model currently clears it at the lowest cost per accepted outcome, and when they last checked. Without that, "best model for the task" is a preference dressed as an architecture, and in practice it usually means whichever model the team is most familiar with, or whichever contract was signed first.

We hold ourselves to the same standard. Fig's routing sends each task class to a model chosen on measured results for that class, which is a claim that only survives because the measurement exists and gets re-run when the frontier moves. The same instrument decides the question in our post-training piece: the condition for training a model on a customer's data is that the general model demonstrably fails, and demonstrably is an eval word. It is also what the four proofs in the playbook are, under a different name, and what makes the four outcome metrics in a transformation auditable rather than asserted.

Evaluation is not one workstream among several. It is the thing that makes the others decidable.

How we build it with a customer

Serious enterprise platforms treat this as core rather than optional. Palantir ships AIP Evals with ontology-defined labels, model comparison, and variance across repeated runs, which is the right shape for the problem. What matters is less the tooling than where the set comes from and who ends up holding it.

In a Fig engagement the harness is built during the work rather than sold alongside it.

The audit that opens an engagement produces the baseline: how the workflow runs today, how long it takes, what fraction of output gets reworked. That is the number every later measurement is read against, and a program that skips it can never prove anything afterward.

The eval set comes out of the workflow itself. The first cases are real ones the team is already doing. The authored adversarial cases come from the workshop where the people who own the process explain what would be unacceptable. And once the workflow is running, the labeled data arrives on its own: every correction a reviewer makes to an agent's output is a labeled example, recorded in Command Center as a side effect of the work. This is the same asset that makes post-training possible, which is not a coincidence.

The judge, where one is used, is calibrated against those same reviewers before anyone trusts its numbers, and re-calibrated when it changes.

The gates are real. Nothing widens scope without a rerun. Approval controls stay on the irreversible steps regardless of the score.

And the customer owns it. The cases, the rubric, the thresholds, and the results are theirs, exportable, and readable by their own engineers. When the engagement ends, the harness is what lets their team make the next decision without us.

Where evals stop

Three honest limits.

They measure, they do not guarantee. A workflow that scores well is a workflow that scored well on the cases you thought to include. Evaluation reduces uncertainty; it does not eliminate it, and it is not a substitute for approval gates on anything irreversible.

Your own set decays too. Everything said above about saturation and Goodhart applies to an internal set the moment a team starts optimizing against it. A set that never changes becomes a benchmark you overfit. Rotate cases, add new failures as they appear, keep a holdout you do not tune against, and check occasionally that the set still resembles the work.

There is a phase where this is overhead. The case against heavy evaluation is that it slows early exploration, and in the exploratory phase that is true: when you are trying to find out whether something is possible at all, taste and fast iteration beat a harness. The line is not a stage of maturity, it is a property of the work. The moment an action becomes irreversible, or a task starts repeating at volume, measurement stops being optional, because at that point the cost of being wrong is paid by someone who did not run the demo.

Key takeaways

• Public benchmarks are saturated, contaminated, and optimized against, and they were never built to answer an enterprise's question. Read them as a weather report, not a decision.
• An eval does three jobs: choosing, gating, and detecting drift. Most teams build for the first and get caught by the third.
• The enterprise instrument differs in kind: your cases, your reviewers' standard, the whole system rather than the model, scored on the path as well as the answer, with cost per accepted outcome as the denominator.
• The eval set is an owned asset. It outlives every model underneath it, and a claim to use the best model for each task is unverifiable without one.

You cannot manage what you cannot measure is a cliché because it is load-bearing. In enterprise AI it is the difference between a program that can defend its decisions and one that is running on the confidence of whoever spoke last.