What is Evaluability? The Missing Ingredient in AI-Assisted Development
Short answer: Evaluability is the property of a requirement that makes it possible to determine — definitively — whether it has been met. A requirement is evaluable if you can test it. Most AI-assisted development fails not because the AI is wrong, but because the requirements aren't evaluable.
The simplest definition
A requirement is evaluable if you can answer "yes" or "no" to the question: does this output meet this requirement?
Most requirements written for AI agents cannot pass this test. They describe intent ("the user should have a good experience"), direction ("make the checkout flow faster"), or features ("add a dashboard"), but they don't define success. When there's no definition of success, the AI's output is always plausible and never verifiable.
Evaluability closes this gap. It converts intent into criteria.
Why evaluability matters for AI agents specifically
Human developers can ask clarifying questions. They accumulate context. They notice when something feels wrong even if they can't articulate why. AI agents do none of these things — they optimise toward the nearest plausible interpretation of the instruction they were given.
This means the quality of AI-generated output is directly proportional to the evaluability of the intent it was given. Give an agent a fuzzy instruction and you get plausible output of uncertain correctness. Give an agent an evaluable instruction and you get output that either passes or fails a definable standard.
There is no middle ground in the long run. As agents become more capable, the bottleneck shifts entirely to the quality of the intent they're given.
What makes something evaluable?
An evaluable requirement has three components:
1. A subject Who is doing something? (A user, an admin, the system)
2. A behaviour What do they do differently once this succeeds?
3. A condition or constraint Under what circumstances? With what limitation?
Non-evaluable: "The onboarding should be smooth."
Evaluable: "A new user can complete onboarding and create their first project without contacting support — and without leaving the onboarding flow."
The evaluable version can be tested. You can watch a real user attempt it, pass or fail the criterion, and know whether the requirement is met. The non-evaluable version produces endless debate about what "smooth" means.
The difference between a spec and an intent document
Traditional specifications describe what to build: features, screens, interactions, data models.
Intent documents define what correct looks like: user behaviours, acceptance criteria, the difference between a feature existing and a feature working. This is the thinking behind the INTENT.md format and intent-driven development.
| Traditional spec | Intent document | |
|---|---|---|
| Describes | What to build | What success looks like |
| Written in | Feature language | User-outcome language |
| Testable | Often not | By definition |
| Changes when | Requirements change | When the outcome changes |
| AI can use it as | A description to approximate | A standard to optimise toward |
The distinction is not academic. AI agents given a traditional spec will produce code that looks like what was described. AI agents given an intent document will produce code that can be verified against what was specified as correct.
Evaluability in practice: the one question that reveals it
Before writing any requirement, ask:
What does a user do differently after this succeeds?
If you can answer that question concretely, you have an evaluable requirement. If you can't — if your answer is another description of the feature rather than a change in user behaviour — you don't have a requirement yet. You have a direction.
The question is useful precisely because it's hard to fake. "The user feels more confident" is not an answer. "The user completes checkout without abandoning the payment step" is.
Frequently asked questions
Is evaluability just acceptance criteria by another name? Acceptance criteria are one implementation of evaluability. Evaluability is the underlying property — the capacity of a requirement to be tested against a definable outcome. Not all acceptance criteria are evaluable (many describe features rather than outcomes); and evaluability can be expressed in formats other than formal criteria.
Can non-technical founders write evaluable requirements? Yes. Evaluability is a product skill, not a technical one. The question "what does a user do differently?" is answerable by anyone who understands their user. The formalisation of that answer can be tool-assisted.
Does evaluability slow you down? Defining criteria before building takes time. Rebuilding features because you didn't know what correct looked like takes more time. The return on evaluability is front-loaded and compounds: the clearer the criteria before the build, the fewer the rebuilds after.
What's the minimum viable version of an evaluable requirement? One sentence: "A [user type] can [do something] without [unacceptable condition]." That's sufficient to give an AI agent something to optimise toward and you something to verify against.
IntentDocs helps non-technical founders write evaluable intent before they build — so their AI agents have a standard to work from, not a description to approximate. intentdocs.com
Ready to build what you intended?
Turn your idea into a structured spec and hand it to Claude Code or Cursor over MCP — with an audit trail of built versus planned.
Start free →7-day free trial · no card required