Delivery · Evolved from Product Requirements Document
PRD + Eval Specification
Traditional PRD structure paired with a formal evaluation specification when the feature is model-driven: eval sets, ground truth, guardrails, and post-deployment monitoring criteria.
Executive Guidance
How to hold this framework as a leader
A PRD is a contract between product intent and engineering delivery. For deterministic features, prose acceptance criteria are sufficient. For AI features they are not: prose cannot be tested and cannot detect regression. Any AI-affecting PRD must ship with an evaluation specification that acts as the enforceable contract.
The eval spec is not a QA artifact; it is the definition of correctness for behavior that no engineer directly wrote. Treat it with the same authority as the product requirements themselves.
When to Apply
- Any capability powered by an LLM, model, or autonomous agent
- Regulated products requiring documented acceptance criteria
- Cross-functional work spanning product, engineering, and AI/ML
AI-Era Notes
Acceptance criteria for AI features must be expressed as eval sets, not prose. The eval spec becomes the contract between product intent and model behavior.
Key Trade-off
Prose acceptance criteria are unenforceable against models; eval specs are non-negotiable for AI features.
Operating Sequence
The order in which to install the framework
- 01Write the PRD with problem, users, requirements, and non-goals as usual.
- 02For AI-affecting requirements, replace prose acceptance criteria with an eval spec.
- 03Define the eval set: representative inputs, expected behavior, and pass thresholds.
- 04Document guardrails, monitoring, and rollback authority before release.
- 05Version-control the PRD and eval spec together; treat updates with the same rigor as code review.
Key Artifacts
The documents this framework produces
PRD
Problem, users, requirements, non-goals, and open questions.
Owner: Product
Eval Specification
Formal definition of correctness for AI-affecting requirements.
Owner: Product + AI/ML
Release Runbook
Guardrails, monitoring, and rollback authority for launch.
Owner: Product + Engineering
Operating Checklist
What "good" looks like when installed
PRD
- Problem statement is grounded in discovery evidence.
- Non-goals are explicit.
- Every AI-affecting requirement points to a corresponding eval spec.
Eval Spec
- Eval set covers success, safety, adversarial, and regression cases.
- Pass thresholds are numeric and pre-committed.
- Ownership and review cadence are defined.
Common Antipatterns
- AI PRDs with prose acceptance criteria and no measurable evals
- Eval specs written after the model is already in production
Boardroom Questions
- Can we point to the eval set that defines correctness for this AI feature?
- Who owns the eval set after launch?
- What is our published rollback threshold?