How to Write Kill Criteria Before You Write the AI Prompt

Why stopping conditions belong in the first document

Teams usually define what success looks like before they build a feature. They are less disciplined about defining failure. The result is a pilot that keeps moving because the team has invested engineering time, stakeholders expect progress or the latest prompt adjustment appears promising. None of those reasons establishes that the feature deserves more resources.

Kill criteria solve this problem by making the exit conditions part of the original product decision. They also protect a useful distinction: a pilot can fail its commercial hypothesis while demonstrating technical feasibility, or meet a technical target while creating an unacceptable user experience. The decision rules should reflect both dimensions instead of compressing everything into one success score.

• Prevent open-ended pilot extensions.
• Protect the original budget and scope.
• Make negative evidence actionable.
• Separate technical progress from user value.

Separate immediate stops from performance failures

Not every failure should wait for a weekly review. Some conditions require an immediate pause because continuing could expose users or the company to unacceptable risk. Examples include an unexpected disclosure of private information, unauthorized data access, a serious security issue or behavior that breaches an approved product restriction. The responsible security, privacy or compliance professional should define the applicable requirements and response process.

Performance failures follow a different process. A low completion rate, frequent corrections or weak repeat usage may justify a pause, a narrower experiment or a stop after the agreed observation period. Define these separately in the charter. Otherwise, a team may accidentally treat an urgent operational problem as an ordinary product metric that can wait until Friday.

• Immediate pause: critical privacy or security issue.
• Immediate pause: behavior outside approved product boundaries.
• Performance stop: primary metric below its minimum threshold.
• Commercial stop: cost above the approved ceiling.
• Scope stop: essential dependencies cannot be delivered in time.

Write thresholds that can be measured

A usable kill criterion contains four parts: a metric, a threshold, an observation window and a required action. Consider a short-video app that generates captions. A vague condition would be poor caption quality. A measurable condition might be: if fewer than 75 percent of sampled caption segments meet the predefined acceptable-quality rubric in two consecutive evaluation batches, pause expansion and review the feature. The 75 percent threshold is illustrative and must be set against the product's requirements.

Define the denominator and measurement method. Does caption quality refer to words transcribed correctly, correct speaker attribution, timing alignment or the percentage of complete clips that require no substantial correction? These are different measures. A threshold that does not specify its unit can become an argument about definitions instead of a decision about performance.

• Metric: the exact behavior or outcome.
• Threshold: the minimum or maximum acceptable value.
• Window: the period or number of observations.
• Action: pause, narrow, change or stop.

Cover the five common failure categories

A compact kill-criteria set should cover user value, output quality, economics, reliability and operating risk. User value asks whether people actually complete the intended task. Quality asks whether the output is usable. Economics covers model, infrastructure and support costs. Reliability covers latency, availability and failures across supported conditions. Operating risk covers data handling, abuse, accessibility and the product's ability to recover when the AI feature fails.

For example, a pilot might require at least 70 percent task completion, cap median response time at four seconds and keep variable cost below a predefined amount per successful task. These are proposed thresholds, not universal standards. A team building an occasional creative tool may accept different latency and cost limits from a team adding AI to a frequent, time-sensitive workflow.

• User value: minimum successful task completion.
• Quality: maximum acceptable correction or error rate.
• Economics: maximum cost per completed task.
• Reliability: latency and availability limits.
• Risk: mandatory pause conditions and escalation owners.

Avoid criteria that can be negotiated away

A weak criterion often uses language such as good engagement, acceptable accuracy or reasonable cost. Each phrase invites a different interpretation after results become available. Replace subjective language with a defined measure and an agreed calculation. Also distinguish the minimum sample required for a decision from the performance threshold itself. If a pilot collects too little evidence, the outcome may be inconclusive rather than a demonstrated failure.

Set a rule for inconclusive evidence. For example, if the pilot needs 200 completed tasks but reaches only 90 by the end of week five, the team could allow one limited extension only if recruitment remains feasible and the budget is available. That extension should have a fixed deadline. Repeated extensions should require a new charter and a new resource decision.

• Avoid undefined words such as good and acceptable.
• Specify the sample and denominator.
• State how missing or incomplete data is handled.
• Set a maximum extension period.
• Require approval for any changed threshold.

A go-no-go decision template

The final decision should show which criteria passed, which failed and which remain unknown. A weighted score can help organize supporting evidence, but it should never override a mandatory stop condition. If a privacy condition is breached, a high engagement score does not cancel the need to pause. Likewise, a pilot that misses its primary threshold should not be labeled successful because several secondary metrics improved.

Use three outcomes. Go means the feature can proceed to the next explicitly approved development stage. Change means the central hypothesis still deserves investigation, but the current implementation or audience needs adjustment. No-go means the feature stops in its current form. Record the evidence and the next owner so the decision has operational consequences.

• Pilot: [feature and version].
• Primary hypothesis: [expected user outcome].
• Mandatory stop conditions: [list].
• Performance thresholds: [metric, threshold and window].
• Evidence status: [passed, failed or inconclusive].
• Decision: [go, change or no-go].
• Owner and next action: [name or role and deadline].

Questions readers ask