Prompt Management and Cost Showback

Central prompt registry with versioning, review, and AI cost attribution by team and product.

Prompt Registry

Prompt Registry Schema

# prompts/registry.yaml
prompts:
  - id: support-faq-v2
    name: Customer Support FAQ
    version: "2"
    owner: support-team
    stage: Production
    aliases:
      - champion

    template: |
      You are a helpful customer support agent. Answer the user's question
      using only the provided context. If you don't know, say so.

      Context: {{context}}
      Question: {{question}}

      Answer:

    variables:
      - name: context
        type: string
        required: true
      - name: question
        type: string
        required: true

    guardrails:
      - id: pii-detection
        type: input
      - id: toxicity-check
        type: output
      - id: hallucination-check
        type: output

    eval:
      judge_model: gpt-4o-mini
      sample_rate: 0.1
      metrics:
        - faithfulness
        - relevance
      thresholds:
        faithfulness: 0.90
        relevance: 0.85

    model:
      default: meta-llama/Llama-3.1-8B-Instruct
      fallback: meta-llama/Llama-3.1-1B-Instruct

    created_at: "2026-01-10T09:00:00Z"
    updated_at: "2026-01-15T14:30:00Z"
    updated_by: prompt-engineering-team

    changelog:
      - version: "1"
        change: "Initial prompt"
        date: "2026-01-10"
      - version: "2"
        change: "Added hallucination guardrail, tightened context instruction"
        date: "2026-01-15"

  - id: code-review-v1
    name: Code Review Assistant
    version: "1"
    owner: platform-team
    stage: Review
    aliases:
      - challenger

    template: |
      Review the following code diff for bugs, security issues, and style violations.

      Diff: {{diff}}
      Language: {{language}}

    variables:
      - name: diff
        type: string
        required: true
      - name: language
        type: string
        required: true

    guardrails:
      - id: pii-detection
        type: input

    eval:
      judge_model: gpt-4o-mini
      metrics:
        - accuracy
        - usefulness

Prompt Lifecycle

stateDiagram-v2
    [*] --> Draft: create prompt
    Draft --> Review: submit for review
    Review --> Production: approval + pass eval
    Production --> Deprecated: superseded
    Deprecated --> [*]: archive
    Production --> Draft: rollback / revision

    note right of Review: Peer review + eval gate required
    note right of Production: Immutable once deployed

Prompt Management Implementation

from dataclasses import dataclass, field
from typing import Optional, List
from datetime import datetime
import hashlib
import json

@dataclass
class PromptVersion:
    id: str
    version: str
    template: str
    variables: List[str]
    guardrails: List[str]
    model: str
    created_at: datetime
    created_by: str

    def content_hash(self) -> str:
        content = json.dumps({
            "template": self.template,
            "variables": self.variables,
            "guardrails": self.guardrails,
            "model": self.model,
        }, sort_keys=True)
        return hashlib.sha256(content.encode()).hexdigest()[:12]

@dataclass
class PromptRegistry:
    prompts: dict = field(default_factory=dict)

    def register(self, prompt: PromptVersion):
        key = f"{prompt.id}:{prompt.version}"
        self.prompts[key] = prompt

    def get(self, prompt_id: str, version: Optional[str] = None) -> PromptVersion:
        if version:
            return self.prompts[f"{prompt_id}:{version}"]
        # Return latest Production version
        candidates = [
            p for p in self.prompts.values()
            if p.id == prompt_id and p.stage == "Production"
        ]
        if not candidates:
            raise ValueError(f"No Production version for {prompt_id}")
        return max(candidates, key=lambda p: p.created_at)

    def render(self, prompt: PromptVersion, **variables) -> str:
        # Validate required variables
        for var in prompt.variables:
            if var not in variables:
                raise ValueError(f"Missing required variable: {var}")

        template = prompt.template
        for var, value in variables.items():
            template = template.replace(f"{{{{{var}}}}}", str(value))
        return template

# Usage
registry = PromptRegistry()

prompt = PromptVersion(
    id="support-faq",
    version="2",
    template="Context: {{context}}\nQuestion: {{question}}\nAnswer:",
    variables=["context", "question"],
    guardrails=["pii-detection", "toxicity-check"],
    model="meta-llama/Llama-3.1-8B-Instruct",
    created_at=datetime.now(),
    created_by="prompt-team",
)

registry.register(prompt)
rendered = registry.get("support-faq").render(
    context="Our refund policy allows returns within 30 days.",
    question="Can I return this item?",
)
print(rendered)

Cost Showback

Attributing AI inference and training costs to teams and products.

Cost Attribution Model

Cost Showback SQL

-- Monthly AI cost by team and model
SELECT
  team,
  product,
  model,
  SUM(prompt_tokens) as total_prompt_tokens,
  SUM(completion_tokens) as total_completion_tokens,
  SUM(cost_usd) as total_cost_usd,
  COUNT(*) as request_count,
  SUM(cost_usd) / NULLIF(COUNT(*), 0) as avg_cost_per_request_usd
FROM ai_inference_events
WHERE event_date >= DATE_TRUNC('month', CURRENT_DATE)
  AND event_date < DATE_TRUNC('month', CURRENT_DATE) + INTERVAL '1 month'
GROUP BY team, product, model
ORDER BY total_cost_usd DESC;

-- Budget alert: teams approaching monthly limit
SELECT
  team,
  SUM(cost_usd) as spend_this_month,
  monthly_budget_usd,
  SUM(cost_usd) / monthly_budget_usd as budget_pct
FROM ai_inference_events
JOIN team_budgets USING (team)
WHERE event_date >= DATE_TRUNC('month', CURRENT_DATE)
GROUP BY team, monthly_budget_usd
HAVING SUM(cost_usd) / monthly_budget_usd > 0.8;

Cost Showback Dashboard

Decision Guidance

Anti-Patterns

Related Notes

• Internal Developer Platform — IDP, golden paths, model serving
• Model Registry And Lineage — Model versions, stages, aliases
• Eval In Production — Continuous evaluation, LLM-as-judge
• Cost Optimization — Caching, model selection, token budgets

Sources

• PromptLayer, LangSmith, Humanloop, Weights & Biases Prompts, Internal Developer Platform patterns, FinOps for AI, Cloud cost showback, Team Topologies, platform engineering community.*