Voice Agent Prompt Evolution Loop

0. Purpose, Scope, and What This Is Not

Purpose

This document establishes the WHAT and WHY of the Voice Agent Prompt Evolution Loop - the closed-loop process by which an inbound voice agent's system prompt iteratively improves week over week, on real production call data, with minimal human intervention.

It is the application of the canonical SalesBlaster Darwinian Loop doctrine to a specific class of artifact: the LLM system prompt. Where the marketing Darwinian Loop treats a stimulus (cold email, ad creative) as the genotype, this loop treats the system prompt itself as the genotype, with production calls as the population it produces phenotypes across, and the scoring rubric as the fitness function that selects against weakness.

Scope

• The end-to-end loop architecture from V_n in production to V_{n+1} shipped to staging or production.
• The prompt-as-genotype decomposition into testable modules.
• The fitness function design and calibration discipline.
• The constants that must hold across cycles for evolution (not noise) to occur.
• The questions every cycle must answer.
• The engineering first principles that govern the architecture.
• Anti-patterns specific to prompt-evolution loops (distinct from the marketing-stimulus anti-patterns in the parent doctrine).

What This Is Not

• It is not a Vapi/ElevenLabs configuration manual. Provider specifics live in implementation specs.
• It is not a copy or scenario library. Scenarios are produced under this doctrine, not enumerated by it.
• It is not a replacement for the parent Darwinian Loop doctrine. It is the specialization of that doctrine to one artifact class.

Reading Order

Sections 1–4 are the core. If you read nothing else, read those. Sections 5–7 are the diagnostic discipline. Sections 8–11 are reference. The worked example is in §12.

---

1. Executive Summary - The Loop in One Page

A voice agent is its prompt. The prompt produces a probability distribution over agent behavior; production calls are samples drawn from that distribution. To improve the agent is to shift the distribution - and you shift the distribution by treating the prompt as a stack of needle-mover variables and applying the scientific method as a closed loop, with calls as the population and a versioned scoring rubric as the fitness function.

The four operating laws inherited from the parent doctrine, restated for prompts:

1. Polaris is sacred. Sub-rules are not. A voice agent's Polaris is composite: business-cost-weighted rubric score plus downstream customer outcome (callback completion, retention, revenue). Optimizing rule-compliance scores while customer outcomes degrade is Goodhart's Law in action.
2. Ceteris paribus or the cycle is wasted. Iterate one prompt module per cycle. If V_{n+1} changes the persona AND the close sequence AND the scenario routing and scores higher than V_n, you produced no information about which change mattered.
3. Stimuli decay; rubrics drift; bottlenecks migrate. What the agent fails at this cycle is rarely what it failed at last cycle. The rubric must update deliberately to follow the bottleneck - but never silently mid-cycle.
4. Cycle time is the whole game. A team running a weekly loop on 200 calls beats a team running a quarterly loop on 5,000. The infrastructure investment that earns its keep is the one that compresses the time from "production failure observed" to "patched prompt deployed and verified."

The unique additions for prompts as a genotype class:

• The fitness function is the system. A misweighted rubric produces a Goodhart artifact. Weight by business cost per failure type, not by failure frequency.
• The rubric is a control-theory set point. It must be versioned and immutable per cycle. A rubric that changes mid-cycle invalidates cross-cycle comparisons.
• LLM-evaluates-LLM compounds bias. Anchor periodically to ground truth (human review, real customer outcomes, downstream conversion data).
• Holdout calls = regression suite. Without one, V_{n+1} can fix the new failure mode while silently breaking what V_n handled correctly.
• Modularity beats monolith. Persona, hard rules, scenario routing, outputs, escape hatch - each is a separate genotype variable iterable in isolation.

Everything else in this document is elaboration of these points.

---

2. First Principles - The Prompt as Genotype

2.1 The Reframe

A system prompt is not source code. Source code is deterministic: same input produces same output. A prompt produces a probability distribution over outputs, and production calls are samples drawn from that distribution. You do not "fix bugs" in a prompt the way you fix bugs in code - you select against bad samples and reinforce good ones across a population of variants.

This is the same shift biologists make from individual organism to population: you don't make a species more fit by debugging one individual; you select across the population. The mental model that follows:

2.2 Why This Reframe Earns Its Keep

Three immediate consequences. First, the same Darwinian discipline you already enforce on cold email genotypes applies - same bottleneck diagnosis, same ceteris paribus rule, same cycle-time obsession. You don't invent a new framework, you instantiate an existing one. Second, the genotype/phenotype separation forces clarity on what's being changed (genotype: prompt) versus what's being measured (phenotype: call behavior). Most prompt-engineering teams confuse these, then can't explain why their prompt got worse. Third, it makes the loop's failure modes legible - they're the same failure modes the marketing loop has, with names that already exist in your doctrine (premature iteration, sub-metric optimization, ceteris paribus violation).

2.3 The Six Sub-Systems for Voice Agents

The marketing doctrine's six sub-systems (latent → attention → interest → appointment → show → primed → closed) map onto a voice agent's call lifecycle as five sub-systems. This is the prospect-state axis of the call.

A voice agent failing at sub-system 5 (the pilot's V1: 75% close-sequence violation rate) is not the same failure as failing at sub-system 1. They have different needle-mover variables, different fitness criteria, and require different prompt modifications. Iterating the wrong sub-system is the voice-agent equivalent of optimizing a sub-metric in isolation: you lose cycles producing no signal.

---

3. The Voice Agent Prompt Genotype - DNA Decomposition

This is the most consequential design choice in the loop. The prompt is not one thing. It is a stack of modules, each with its own 2–4 needle-mover variables. Iteration must happen at the module level for ceteris paribus to hold and for attribution to be possible.

3.1 The Five Canonical Modules

3.2 The Output Schema is a Sixth, Special Module

The post-call structured output (issue type, captured fields, urgency tag, outcome classification) is a code-level contract. It is not a hot-swappable variable. Changes to the output schema require deployment because downstream consumers depend on the shape. The prompt module that populates the schema is iterable; the schema itself is versioned in code.

This boundary matters: a synthesizer LLM that proposes "add a new field to the output" is proposing a code change, not a prompt change. The loop must distinguish these and route them differently - prompt changes go through the weekly cycle; schema changes go through a deployment.

3.3 Why Modules and Not Monolith

The reason ceteris paribus is hard for prompts: changing one sentence in one section can ripple through the model's behavior in non-obvious ways. The mitigation is to constrain which sentence in which section changes per cycle. If V_{n+1} changes only the close-sequence module, attribution is clean: any score change against the close-related rubric items is attributable to that change; any score change against scenario-routing rubric items is noise.

Operators (and synthesizer agents) violate this constantly - a recommendation that "we should improve the close AND tighten the persona AND add a new scenario" is a three-cycle plan, not a one-cycle plan. The loop infrastructure must enforce single-module iteration as a structural constraint, not a guideline.

---

4. The Evolution Loop - Ten Phases

The pilot workflow had seven implicit steps. This section names ten phases with explicit inputs, outputs, and constants. Phases 7, 8, and 10 are the gaps in the current workflow - the loop is open, not closed, without them.

Phase 1 - Production Telemetry (always-on)

Every call is recorded with transcript, structured output, duration, status, and provider metadata. Downstream outcome data (customer callback completion, scheduled service completed, NPS, retention signal) is captured asynchronously and joined to call records.

Output: A growing call corpus, persistent and queryable. Constant: Telemetry schema. If telemetry shape changes, historical comparability breaks - version the schema and migrate explicitly.

Phase 2 - Cycle Trigger

Time-based (weekly) or volume-based (every N calls). Pick one and stick with it; mixing is a constants violation.

Recommended default: Weekly for production deployments with steady volume, volume-based (every 200 calls) during initial deployment when volume is variable.

Output: Cycle ID, locked corpus boundary (the call IDs that will be evaluated this cycle). Constant: Trigger criterion across cycles.

Phase 3 - Evaluation

The evaluation prompt scores every call in the locked corpus against the versioned rubric. Per-call scorecards plus aggregate analysis report.

Inputs: Locked corpus, versioned rubric, versioned evaluator prompt. Outputs: Scorecards (one per call), aggregate report, flagged-calls file, dropped-calls file. Constants: Rubric version, evaluator prompt version, evaluator model and temperature, scoring formula. All four must be frozen for the duration of the cycle. A rubric edit mid-cycle invalidates the data.

Phase 4 - Synthesis

A synthesizer LLM ingests the aggregate report, the per-call scorecards, the OCP organizational wiki for the tenant, and the current production prompt. It produces an Improvement Spec - a structured artifact naming: which prompt module is the bottleneck, what the proposed change is, what the predicted score impact is, and what success on the next cycle looks like.

Inputs: Aggregate report, scorecards, OCP wiki for tenant, current prompt (V_n), parent doctrine references. Output: Improvement Spec (artifact_type: note, role: improvement_spec). Constants: Synthesizer prompt and model. The synthesizer is itself a prompt that iterates - but on a slower clock than the agent prompt (quarterly, not weekly). Critical: The Improvement Spec must name one module to iterate. A spec that proposes changes to three modules must be split into three specs and queued - only one runs this cycle.

Phase 5 - Variation

A generation LLM produces 3 candidate prompt variants from the Improvement Spec. The variants differ in their interpretation of the spec, not in scope - all three modify the same module, but with different angles (slight / moderate / complete modification per the parent doctrine §4.5).

Inputs: Improvement Spec, current prompt (V_n). Outputs: V_{n+1}.A, V_{n+1}.B, V_{n+1}.C (three candidate prompts). Constants: Variation prompt and model. Why three: Not because three is optimal - because three lets you compare angles cheaply before committing. One variant is conservative (slight mod, high affinity to V_n); one is moderate; one is exploratory (complete reset of the targeted module). This gives the convergence step real choice.

Phase 6 - Convergence

A final synthesis LLM merges the best traits from the three variants into a single V_{n+1} candidate. It is given the original Improvement Spec, the three variants, the OCP wiki, and a checklist of internal-consistency checks (no contradictions between modules, examples match abstract rules, no schema drift).

Inputs: Three variants, Improvement Spec, OCP wiki, consistency checklist. Output: V_{n+1} candidate prompt. Failure mode: The merged prompt accidentally combines incompatible traits and is worse than any of the three inputs. The validation phase catches this.

Phase 7 - Validation (the first gap in the current workflow)

V_{n+1} is scored against a frozen holdout corpus - a curated subset of historical calls representing the full distribution of scenarios, urgencies, and edge cases. The holdout is not the cycle's evaluation corpus. It is a stable regression suite that does not change cycle to cycle except through deliberate, infrequent updates.

V_{n+1} must:

1. Score better than V_n on the holdout's aggregate.
2. Not regress materially on any rubric category (no category drops more than X points; X is policy, default 10).
3. Pass a human-in-the-loop spot check on the 5 lowest-scoring calls (the human is checking for "did the rubric miss something the model is doing badly").

Inputs: V_{n+1} candidate, frozen holdout corpus, current rubric, human reviewer. Output: Validation report (pass / conditional / fail). If fail: Return to Phase 4 with the validation report appended to the inputs. Do not deploy. Critical: This is the regression suite for prompts. Without it, the loop fixes new failures while breaking old successes, and you only discover this in production.

Phase 8 - Deployment (the second gap)

V_{n+1} ships to canary first - a fraction of production traffic (e.g., 20%) for a defined window (24–72 hours). During canary, two metrics gate full rollout:

1. No P0 regressions: specific named failure modes that are unacceptable at any rate (promise violations, "send an email" to frustrated callers, missed hard-stops on harmful content).
2. Aggregate score within tolerance: rubric score on canary calls is within X% of holdout score (X = policy, default 5%).

If both gates pass for the canary window, V_{n+1} promotes to 100%. If either fails, automatic rollback to V_n.

Inputs: V_{n+1} candidate, traffic split configuration, P0 alarm thresholds. Output: V_{n+1} live in production OR rollback executed. Constant: Canary fraction and duration. Changing canary policy mid-cycle is a constants violation.

Phase 9 - Observation

The first full cycle on V_{n+1} is the observation phase. Calls are collected, evaluated against the (still frozen) rubric, and the cycle's aggregate is compared to V_n's aggregate from the previous cycle. This is the actual fitness measurement that decides whether the change worked.

Critical discipline: Do not iterate on V_{n+1} during its observation cycle. The parent doctrine's Step 2 ("do nothing") applies. Premature iteration during the observation window is the most expensive error in this loop. The infrastructure must not allow mid-cycle prompt changes without explicit override and logged justification.

Phase 10 - Decision

After the observation cycle:

• V_{n+1} hit Polaris and improved without regressions: ship V_{n+2} as a continued iteration of the next-priority module (bottleneck has migrated).
• V_{n+1} failed to improve or regressed: rollback to V_n, mark the Improvement Spec as falsified, return to Phase 4 with the lesson appended.
• V_{n+1} mixed (improved on target, regressed elsewhere): this is the most informative outcome. It tells you the modules are not as independent as you thought. Update the genotype decomposition accordingly.

Output: Cycle retrospective (artifact_type: report, role: cycle_retrospective). Captures what changed, what the prediction was, what actually happened, and what the loop learned about the genotype.

---

5. The Fitness Function - Designing the Rubric

5.1 The Most Consequential Design Choice in the Whole Loop

Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." The rubric IS the system. Optimizing against it produces an agent that maximizes rubric score, which is only useful insofar as the rubric correlates with what you actually want.

The pilot's V1 rubric weighted "got it" violations identically to promise violations. This implies they're equally important. They are not - by orders of magnitude. A "got it" violation is a brand-tone leak. A promise violation is a commitment the company cannot keep, generating angry callbacks, damage to customer relationships, and operational scramble. Weight by business cost per failure, not by failure frequency.

5.2 Three Layers of Fitness

A robust voice-agent rubric has three layers, each anchored to different ground truth:

Layer 1 - Rule compliance (LLM-scorable, fast feedback). Did the agent follow the hard rules? Use one question per turn? Avoid banned words? Hit the close sequence? This is what an LLM evaluator can score reliably and cheaply across hundreds of calls.

Layer 2 - Outcome quality (joinable to downstream data, slower feedback). Did the call resolve? Did the captured action actually execute (truck dispatched, ticket created, callback completed)? Did the customer call back angry? This requires joining call records to operational outcomes - usually a 24-72 hour delay before signal is available.

Layer 3 - Customer outcome (slowest feedback, hardest to attribute). Did the customer churn? Did NPS move? Did revenue change? This is monthly or quarterly signal, but it is the only ground truth that anchors the other two against drift.

The composite Polaris is a weighted function of all three. The weight on Layer 1 should be high enough to drive weekly iteration but low enough that gaming Layer 1 alone cannot produce a good Polaris score. A Polaris that can be hit by Layer 1 alone is a misdesigned Polaris.

5.3 Rubric Versioning and Drift Discipline

The rubric is a versioned artifact. Per-cycle, its version is frozen. Updates happen between cycles, deliberately, with a documented reason ("we are removing 'got it' from the banned-words list because the client's brand voice now wants warmth"; "we are adding 'access notes captured' as a mandatory field for pickup scenarios because dispatch failures correlate with missing access notes").

Rubric drift is the single largest cause of cross-cycle data being uninterpretable. The mitigation is treating the rubric like a database schema: versioned, migrated, never silently edited.

---

6. Constants - What Must Hold Across Cycles

The parent doctrine's ceteris paribus law (§4.5) requires that everything not under test is held constant. For the prompt-evolution loop, the constants are:

If any of the above changes during a cycle, that cycle's data is contaminated. The cycle does not produce information; it produces noise dressed as information, which is worse than no data at all because it produces false confidence.

The corollary is that the scorecards-cycle-retrospective trail must record the version of every constant for every cycle. When a regression appears six cycles later, the audit trail tells you whether a constant silently changed and explains it.

---

7. The Questions Framework - Fifteen Questions Every Cycle Must Answer

This mirrors the companion measurement discipline of questions every metric must answer. Any cycle of the loop must answer at least one question from each of the three groups below. A cycle where you cannot answer the questions is a cycle that produced no learning.

Pre-Cycle Questions (Loop Architecture)

1. Genotype: What are the modules of this prompt, and what is the 2–4 needle-mover variable set within each? If you cannot enumerate them, you are iterating a black box.
2. Fitness function: What is the dollar cost of each failure mode, and is the rubric weighted accordingly? "Got it" usage and promise violations cannot have the same weight.
3. Cycle time: What is the loop latency from production deployment → next batch evaluable → V_{n+1} ready for canary? Where is the bottleneck - call volume, evaluation cost, human review, or deployment friction?
4. Ground truth anchor: Where does the rubric connect to real customer outcomes (Layer 2 and Layer 3)? A rubric that is purely Layer 1 will optimize itself into a corner.
5. Constants list: What is held fixed across cycles, and where is the version audit log?
6. Reversibility: Can V_{n+1} be rolled back in one click? If not, the loop is unsafe to run autonomously.

Per-Cycle Questions (During Execution)

7. Sample adequacy: Is the call corpus large enough and representative enough to detect the effect we are testing for? What is the minimum sample for the Polaris-level effect size that matters?
8. Evaluator calibration: When a human reviews the same call as the LLM evaluator, do they score it the same? What is our inter-rater reliability? When did we last spot-check?
9. Variation meaningfulness: Are the three variants meaningfully different (different angles), or are they paraphrases of the same change? Paraphrase variation is no variation.
10. Convergence criteria: What rule decides which trait survives the merge? Human pick, LLM pick, or score against the holdout? Different rules produce different prompts.
11. Validation gates: Did V_{n+1} score better than V_n on the holdout? Did any rubric category regress materially? Did a human spot-check the worst calls under V_{n+1}?
12. Deployment safety: What is the canary fraction, duration, and rollback trigger? What named P0 conditions force immediate rollback regardless of aggregate score?

Cross-Cycle Questions (Compounding Learning)

13. Bottleneck migration: After this cycle's improvement, where is the next bottleneck? If it is the same module again, did we actually fix anything?
14. Rubric drift: Has the rubric drifted from customer outcomes since it was last calibrated? When do we recalibrate against Layer 2 and Layer 3?
15. Module independence: Are the prompt modules we are iterating still genuinely independent, or has a recent change introduced cross-module coupling that breaks ceteris paribus?

---

8. Engineering First Principles - Ten Laws

These are the deeper "why" beneath the architecture. A team that internalizes them does not need the loop diagram - they will reconstruct it from first principles whenever they need it.

Law 1 - The prompt is a genotype, not source code. Source code is deterministic; prompts produce probability distributions. You don't fix bugs in a prompt - you shift the distribution. Mindset shift: from "find the bug" to "select against bad outputs across a population of variants."

Law 2 - The fitness function is the system. The single most consequential design choice in any optimization loop is the objective function. A misspecified rubric produces a Goodhart artifact: the system optimizes for the metric, not the intent. Weight by business cost per failure type, anchor to downstream outcomes, version it, never edit it silently.

Law 3 - Closed-loop control needs a stable set point. From control theory: if the set point moves while you are trying to converge, you never converge. The rubric is the set point. Frozen per cycle is non-negotiable; updates are deliberate, between cycles, and logged.

Law 4 - One change per cycle or no information. Ceteris paribus, applied to prompts: if V_{n+1} changes five things and is better than V_n, you don't know which of the five mattered. Worse, you might keep something that was actually neutral or harmful while dropping the actual win.

Law 5 - Holdout sets are the regression suite. Standard ML hygiene applied to prompts. Without a frozen holdout, you cannot verify that V_{n+1} fixed the new failure mode without breaking what V_n handled correctly. The cost of maintaining a holdout (curation, periodic refresh) is dwarfed by the cost of catastrophic forgetting.

Law 6 - LLM-evaluates-LLM compounds bias. The evaluator has biases. The synthesizer reads the evaluator's output and proposes recommendations consistent with those biases. The variation generator implements them. The new prompt produces calls that get evaluated by the same biased evaluator. Compounding. Mitigation: periodic human spot-checks at known-bias points and periodic recalibration against ground truth (Layer 2 and 3 outcomes).

Law 7 - Production data is not ground truth. A 100% rule-compliant call that loses the customer is a failed call. The rubric measures rule compliance; customer churn measures outcome. They can diverge. The loop must eventually anchor to outcomes (callbacks completed, retention, revenue) - not just rubric scores.

Law 8 - Modularity at the prompt layer. A prompt has parts: persona, rules, scenarios, outputs, escape. Iterate one part at a time. Module independence makes attribution clean. It also enables module-level reuse across tenants - a "hard rules" module proven on one client is a starting V1 for the next.

Law 9 - Specs over hope. Each cycle produces a spec for what is changing (which module, why, with prediction), an implementation (V_{n+1}), a validation (regression score + canary), and a decision (ship, rollback, iterate). This mirrors tracer-bullet and spec-driven development. Hope-driven prompt iteration is how teams ship four "improvements" in a row that all measurably degrade the agent.

Law 10 - Reversibility is cheap; irreversibility is expensive. Versioned prompt history. Blue-green or canary deployment. One-click rollback. The cost of being able to undo is low; the cost of not being able to undo is unbounded. Build for reversibility from V1, not from V5.

---

9. Multi-Tenant Generalization - From One Client to N Clients

The loop runs per-tenant: each inbound voice client has its own corpus, its own prompt versions, its own rubric, its own holdout, its own cycle history. The infrastructure that runs the loop is shared; the data the loop operates on is tenant-isolated.

Three things are shared across tenants and properly belong at the platform layer:

1. The loop machinery itself - orchestration, evaluator pipeline, synthesizer pipeline, deployment infrastructure. Reusable across all voice agent tenants. This is the SalesBlaster moat: a self-improvement system that improves with every client we onboard.
2. The base rubric - a starter rubric covering universal voice-agent quality (turn shape, banned-words category, close-sequence existence, hard-stop categories). Each tenant inherits and extends with client-specific rules.
3. The base genotype decomposition - the five-module pattern (persona, hard rules, scenario routing, intake, close) is universal. Each tenant's modules are populated with client-specific content.

Three things are tenant-specific and must not be shared:

1. The scenario library - a mobile-services company's pickup/cancellation/billing scenarios are not transferable to a chiropractor or a law firm. Scenario routing is the most tenant-specific module.
2. The approved-language phrases and brand voice - these come from the tenant's OCP wiki (/orgs/{tenant}/notes/company-knowledge-base.md, /orgs/{tenant}/adrs/). Knowledge belongs in the org, not in the prompt.
3. The Layer 2 and Layer 3 outcome data - each tenant's downstream outcome signal is tenant-private and tenant-specific.

The synthesizer prompt (§4 Phase 4) takes the tenant's OCP wiki as input precisely so that recommendations are grounded in tenant-specific knowledge while the loop architecture itself remains universal.

---

10. Anti-Patterns and Failure Modes

These compound the parent doctrine's anti-patterns (§11 of the parent). Every one is specific to prompt-evolution loops.

10.1 Rubric drift mid-cycle. Editing the rubric during a cycle to "improve scoring" silently makes V_n and V_{n+1} incomparable. The cycle produced no information. The patch is process: rubric edits go between cycles, never during.

10.2 Evaluator-synthesizer feedback compounding. The evaluator and synthesizer are both LLMs. Their biases align (same model family, similar training). The synthesizer agrees with the evaluator, the variation generator implements it, and the new prompt is rewarded for the same biases on the next cycle. Symptoms: rubric scores climb cycle over cycle while customer outcomes flatten or decline. Mitigation: periodic human spot-check of the bottom 5% and top 5% scoring calls; periodic recalibration of the rubric against Layer 2/3 ground truth.

10.3 Catastrophic forgetting. V_{n+1} fixes the new failure mode but breaks something V_n handled correctly. Discovered in production, not in validation. Mitigation: the frozen holdout corpus (§4 Phase 7) is the only structural defense.

10.4 Overfit to the evaluation corpus. The synthesizer recommends changes specific to patterns in the 222 evaluation calls that do not generalize. V_{n+1} performs better on next cycle's evaluation but is no better in production. Mitigation: validate on the holdout, not on the evaluation corpus; resist the urge to "tune" against this cycle's specific calls.

10.5 Multi-module changes per cycle. The synthesizer proposes "improve close AND tighten persona AND add scenario X." All three ship in V_{n+1}. The cycle's signal is uninterpretable. Mitigation: the Improvement Spec must name one module; multi-module specs are split and queued.

10.6 No rollback capability. V_{n+1} ships, regresses, and there is no clean path back to V_n. The team patches forward instead, accumulating tech debt and brand damage in production. Mitigation: prompt versions are immutable artifacts; deployment is parameterized; rollback is one command.

10.7 Optimizing the rubric instead of the agent. When V_{n+1} fails to improve, the team edits the rubric to make V_{n+1} score better. This is the prompt-loop equivalent of moving the goalposts. It is intellectually obvious to spot in others and almost invisible to spot in oneself. Mitigation: rubric edits require a justification anchored to Layer 2 or Layer 3 outcome data, not to making this cycle look better.

10.8 No P0 deployment gates. V_{n+1} ships to canary, accumulates promise violations during the canary window, and is auto-promoted because aggregate score was within tolerance. Aggregate scores hide rare-but-catastrophic regressions. Mitigation: explicit P0 conditions (e.g., "any promise violation," "any 'send an email' to a flagged-frustrated caller") force rollback regardless of aggregate.

10.9 Silent constant drift. The Vapi voice config gets bumped, or the model temperature changes, or the canary fraction is altered between cycles. The next cycle's data is contaminated. Symptoms: unexplained score swings cycle over cycle. Mitigation: every cycle's retrospective records the version of every constant, including infrastructure constants.

10.10 Loop runs unsupervised for too long. Six weeks of autonomous cycles, no human review, and the agent has drifted somewhere subtle but bad. Mitigation: weekly human spot-check of the lowest-scoring 5 calls and the highest-scoring 5 calls - the highest matter as much as the lowest because they reveal what the rubric is rewarding that humans might not.

---

11. Mapping to an OCP-Conformant Platform

11.1 OCP-Conformant Artifacts

The loop produces and consumes OCP artifacts:

11.2 Bounded Context Placement

The loop's runtime infrastructure lives alongside the voice agents it improves. The orchestration of the cycles can be a durable workflow: Phase 1 telemetry feeds into a per-tenant cycle workflow that fires on the trigger condition and walks Phases 2–10. Each phase is a step with idempotency keys, retries, and durable state.

The synthesizer and evaluator are LLM use cases accessed via a prompt-registry port. Their prompts are themselves OCP artifacts - but they are platform-tenant artifacts, iterated on a quarterly clock, not a weekly clock.

11.3 Tracer Bullet First, Then Extract

The first instance of this loop should ship as a Tracer Bullet - single-file, single-tenant, production-quality but focused. All architectural layers present (evaluator port, synthesizer port, validation port, deployment port) but each implemented in a single file with explicit // Future home: comments marking extraction paths. The extraction trigger is the second voice agent client. Build for the second tenant by making the first tenant's infrastructure clean; do not preemptively generalize across tenants you don't have.

This is the canonical discipline for new platform capabilities and applies here in full.

---

12. The Worked Example - Pilot Deployment

This section grounds the framework in the actual workflow the first deployment ran.

The two phases that are gaps in the current workflow (7 and 8) are the highest-priority infrastructure investments to close the loop. Without them, V2 ships, succeeds or fails in production, and the team learns only by lagging customer-outcome signal - far too slow a feedback signal to drive weekly iteration.

The pilot cycle's most informative finding for the doctrine: the rubric weighted "got it" violations (41 instances) on similar terms to promise violations (8 instances). The synthesizer naturally pointed at the close sequence (75% violation, ~166 instances) as priority because raw count dominated. A business-cost-weighted rubric would have surfaced promise violations as Priority 0 from the start. This is the empirical case for §5.1.

---

13. Operating Principles - The Twelve Commandments

Compressed for memorization. Every one is derived from sections above.

1. The prompt is a genotype. Decompose into modules before iterating.
2. The fitness function is the system. Weight by business cost, anchor to outcomes.
3. The rubric is a set point. Frozen per cycle; updated deliberately, between cycles.
4. One module per cycle. Ceteris paribus or no information.
5. Holdout is the regression suite. No validation without it.
6. LLM-evaluates-LLM compounds bias. Spot-check humans periodically.
7. Production data is not ground truth. Anchor to Layer 2 and 3.
8. Reversibility is cheap; irreversibility is expensive. One-click rollback or it's not safe.
9. Specs over hope. Improvement spec → variation → validation → decision; documented at each step.
10. Do nothing during observation. Premature iteration kills working systems.
11. P0 gates are non-negotiable. Aggregate scores hide rare catastrophes.
12. Cycle time is the whole game. Compress the loop; everything else is scaffolding.

---

14. Glossary

• Evaluation Corpus - The frozen subset of calls scored within a single cycle. Bounded at the trigger phase, immutable for the cycle's duration.
• Fitness Function - The scoring rubric. The objective function the loop optimizes against. Versioned, frozen per cycle.
• Genotype (Prompt) - The decomposition of the system prompt into modules: persona, hard rules, scenario routing, intake/action, close, output schema.
• Goodhart Artifact - An agent that maximizes the rubric while degrading on the underlying intent the rubric is meant to measure.
• Holdout Corpus - The frozen regression suite of historical calls used to validate V_{n+1} before deployment. Distinct from the per-cycle evaluation corpus.
• Improvement Spec - The structured artifact produced by the synthesizer naming the bottleneck module, the proposed change, the predicted impact, and the success criteria for V_{n+1}.
• Layer 1/2/3 Fitness - Three layers of rubric ground truth: rule compliance (LLM-scorable), operational outcomes (joined to ops data), customer outcomes (churn, NPS, revenue).
• Module (Prompt) - One of the five canonical genotype variables: persona, hard rules, scenario routing, intake/action, close. Iteration target.
• P0 Condition - A named failure mode that forces immediate rollback regardless of aggregate score (promise violation, hard-stop miss, abuse-handling failure).
• Polaris (Voice Agent) - The composite metric: business-cost-weighted rubric score plus downstream customer outcome.
• Phase 7 Gap / Phase 8 Gap - The two missing phases in the pilot workflow (validation against holdout; canary deployment with auto-rollback).
• Rubric Drift - Silent, mid-cycle changes to the scoring rubric that invalidate cross-cycle comparability.
• Tracer Bullet - The discipline of shipping the first instance of a capability as a single-file, production-quality, tenant-specific implementation, with extraction paths marked but not built.
• V_n - The system prompt version currently in production.
• V_{n+1} - The candidate replacement prompt produced by the cycle. Validated, then deployed via canary.

---

15. Final Note on This Document's Status

This is v1.0. It is a seed. The expectation is that it will be extended as the loop produces real cross-cycle data and as the second voice agent tenant exposes which parts of this framework are universal versus pilot-specific. Section headers are stable; content beneath extends. Future versions add detail; they do not restructure.

The most dangerous version of this document would be one that became more elaborate without the loop at its center actually shipping V_{n+1} prompts that demonstrably outperform V_n on real customer outcomes. If that happens, this document has become over-engineering, and a future maintainer should aggressively cut it back to §1 (the four operating laws) and §13 (the twelve commandments) and rebuild from there.

The loop is the point. Everything else is scaffolding.

• End of Document v1.0