# Prompt Management and Cost Showback Central prompt registry with versioning, review, and AI cost attribution by team and product. ## Prompt Registry | Concept | Description | |---|---| | **Prompt Template** | Versioned prompt with variables (e.g. `{{question}}`, `{{context}}`) | | **Prompt Version** | Immutable snapshot of template + variables + guardrails + eval config | | **Prompt Stage** | `Draft`, `Review`, `Production`, `Deprecated` | | **Prompt Alias** | Human-readable pointer (`champion`, `challenger`) | | **Guardrails** | Input/output filters attached to each prompt version | | **Eval Config** | Metrics and thresholds for evaluating prompt quality | ## Prompt Registry Schema ```yaml # prompts/registry.yaml prompts: - id: support-faq-v2 name: Customer Support FAQ version: "2" owner: support-team stage: Production aliases: - champion template: | You are a helpful customer support agent. Answer the user's question using only the provided context. If you don't know, say so. Context: {{context}} Question: {{question}} Answer: variables: - name: context type: string required: true - name: question type: string required: true guardrails: - id: pii-detection type: input - id: toxicity-check type: output - id: hallucination-check type: output eval: judge_model: gpt-4o-mini sample_rate: 0.1 metrics: - faithfulness - relevance thresholds: faithfulness: 0.90 relevance: 0.85 model: default: meta-llama/Llama-3.1-8B-Instruct fallback: meta-llama/Llama-3.1-1B-Instruct created_at: "2026-01-10T09:00:00Z" updated_at: "2026-01-15T14:30:00Z" updated_by: prompt-engineering-team changelog: - version: "1" change: "Initial prompt" date: "2026-01-10" - version: "2" change: "Added hallucination guardrail, tightened context instruction" date: "2026-01-15" - id: code-review-v1 name: Code Review Assistant version: "1" owner: platform-team stage: Review aliases: - challenger template: | Review the following code diff for bugs, security issues, and style violations. Diff: {{diff}} Language: {{language}} variables: - name: diff type: string required: true - name: language type: string required: true guardrails: - id: pii-detection type: input eval: judge_model: gpt-4o-mini metrics: - accuracy - usefulness ``` ## Prompt Lifecycle ```mermaid stateDiagram-v2 [*] --> Draft: create prompt Draft --> Review: submit for review Review --> Production: approval + pass eval Production --> Deprecated: superseded Deprecated --> [*]: archive Production --> Draft: rollback / revision note right of Review: Peer review + eval gate required note right of Production: Immutable once deployed ``` ## Prompt Management Implementation ```python from dataclasses import dataclass, field from typing import Optional, List from datetime import datetime import hashlib import json @dataclass class PromptVersion: id: str version: str template: str variables: List[str] guardrails: List[str] model: str created_at: datetime created_by: str def content_hash(self) -> str: content = json.dumps({ "template": self.template, "variables": self.variables, "guardrails": self.guardrails, "model": self.model, }, sort_keys=True) return hashlib.sha256(content.encode()).hexdigest()[:12] @dataclass class PromptRegistry: prompts: dict = field(default_factory=dict) def register(self, prompt: PromptVersion): key = f"{prompt.id}:{prompt.version}" self.prompts[key] = prompt def get(self, prompt_id: str, version: Optional[str] = None) -> PromptVersion: if version: return self.prompts[f"{prompt_id}:{version}"] # Return latest Production version candidates = [ p for p in self.prompts.values() if p.id == prompt_id and p.stage == "Production" ] if not candidates: raise ValueError(f"No Production version for {prompt_id}") return max(candidates, key=lambda p: p.created_at) def render(self, prompt: PromptVersion, **variables) -> str: # Validate required variables for var in prompt.variables: if var not in variables: raise ValueError(f"Missing required variable: {var}") template = prompt.template for var, value in variables.items(): template = template.replace(f"{{{{{var}}}}}", str(value)) return template # Usage registry = PromptRegistry() prompt = PromptVersion( id="support-faq", version="2", template="Context: {{context}}\nQuestion: {{question}}\nAnswer:", variables=["context", "question"], guardrails=["pii-detection", "toxicity-check"], model="meta-llama/Llama-3.1-8B-Instruct", created_at=datetime.now(), created_by="prompt-team", ) registry.register(prompt) rendered = registry.get("support-faq").render( context="Our refund policy allows returns within 30 days.", question="Can I return this item?", ) print(rendered) ``` ## Cost Showback Attributing AI inference and training costs to teams and products. ### Cost Attribution Model | Cost Component | Attribution Method | Granularity | |---|---|---| | Inference tokens | Tag by `team` + `product` per request | Per request, aggregated monthly | | Training compute | Tag training jobs by team | Per training run | | Model hosting | Allocate by model usage % | Monthly per team | | Embedding/storage | Tag by vector store namespace | Monthly per team | | Platform overhead | Split evenly or by headcount | Monthly per team | ### Cost Showback SQL ```sql -- Monthly AI cost by team and model SELECT team, product, model, SUM(prompt_tokens) as total_prompt_tokens, SUM(completion_tokens) as total_completion_tokens, SUM(cost_usd) as total_cost_usd, COUNT(*) as request_count, SUM(cost_usd) / NULLIF(COUNT(*), 0) as avg_cost_per_request_usd FROM ai_inference_events WHERE event_date >= DATE_TRUNC('month', CURRENT_DATE) AND event_date < DATE_TRUNC('month', CURRENT_DATE) + INTERVAL '1 month' GROUP BY team, product, model ORDER BY total_cost_usd DESC; -- Budget alert: teams approaching monthly limit SELECT team, SUM(cost_usd) as spend_this_month, monthly_budget_usd, SUM(cost_usd) / monthly_budget_usd as budget_pct FROM ai_inference_events JOIN team_budgets USING (team) WHERE event_date >= DATE_TRUNC('month', CURRENT_DATE) GROUP BY team, monthly_budget_usd HAVING SUM(cost_usd) / monthly_budget_usd > 0.8; ``` ### Cost Showback Dashboard | Team | Model | Requests | Cost (USD) | Budget | % of Budget | Trend | |---|---|---|---|---|---|---| | Support | Llama-3.1-8B | 450,000 | $1,240 | $2,000 | 62% | ↗ | | Platform | GPT-4o-mini | 120,000 | $890 | $1,500 | 59% | → | | Research | Llama-3.1-70B | 15,000 | $2,100 | $3,000 | 70% | ↗↗ | | Marketing | Claude-3-Haiku | 80,000 | $650 | $1,000 | 65% | ↘ | ## Decision Guidance | Situation | Recommended Approach | |---|---| | Many AI teams, one platform | Central prompt registry + versioning + review | | Prompt sprawl | Require all prompts in registry; deprecate hardcoded prompts | | No cost visibility | Tag all AI spend by team; monthly showback reports | | Budget overruns | Budget alerts at 80%; circuit breaker at 100% | | Prompt quality regression | Eval gate before promoting to Production stage | ## Anti-Patterns | Anti-Pattern | Fix | |---|---| | Prompts hardcoded in applications | Central prompt registry with versioning | | No cost attribution | Tag all AI spend; showback to teams | | No prompt review process | Peer review + eval gate before Production | | No budget alerts | Alert at 80% of monthly budget; hard stop at 100% | | No prompt rollback | Keep previous prompt version; rollback = alias change | ## Related Notes - [[Platform/Internal-Developer-Platform.md]] — IDP, golden paths, model serving - [[MLOps/Model-Registry-And-Lineage.md]] — Model versions, stages, aliases - [[Observability/Eval-In-Production.md]] — Continuous evaluation, LLM-as-judge - [[AgenticEngineering/Production/Cost-Optimization.md]] — Caching, model selection, token budgets ## Sources * PromptLayer, LangSmith, Humanloop, Weights & Biases Prompts, Internal Developer Platform patterns, FinOps for AI, Cloud cost showback, Team Topologies, platform engineering community.*