# Guardrails and Safety

Input and output guardrails, failover patterns, and incident response for AI systems operating in production.

## Guardrail Types

| Type | Position | Purpose | Examples |
|---|---|---|---|
| **Input Guardrail** | Before model | Block unsafe prompts, PII, jailbreak attempts | Profanity filter, PII detector, prompt injection guard |
| **Output Guardrail** | After model | Block unsafe or off-topic responses | Toxicity filter, hallucination check, format validator |
| **Semantic Guardrail** | Before/after | Detect off-topic or malicious intent | Embedding similarity to allowed topics |
| **Rate Limit Guardrail** | Before model | Prevent abuse and cost spikes | Requests per user per minute, token budget per day |

## Guardrail Architecture

```mermaid
graph TD
    A[User Input] --> B[Input Guardrails]
    B -->|Blocked| C[Reject / Sanitize]
    B -->|Pass| D[Model Inference]
    D --> E[Output Guardrails]
    E -->|Blocked| F[Fallback Response]
    E -->|Pass| G[User Response]
    
    B --> B1[PII Detection]
    B --> B2[Jailbreak Detection]
    B --> B3[Topic Filter]
    
    E --> E1[Toxicity Check]
    E --> E2[Hallucination Detection]
    E --> E3[Format Validation]
    
    style C fill:#ffcdd2
    style F fill:#fff3e0
    style G fill:#e8f5e9
```

## Decision Guidance

| Situation | Recommended Approach |
|---|---|
| User-facing chatbot | Input guardrails (PII, jailbreak) + output guardrails (toxicity, hallucination) |
| Safety-critical system | Fallback model + human-in-the-loop for high-stakes outputs |
| High cost per request | Circuit breaker on expensive models; route to cheaper fallback |
| Model degradation | Error budget triggers rollback to previous model version |

## Anti-Patterns

| Anti-Pattern | Fix |
|---|---|
| No guardrails on user input | Add PII and jailbreak detection before the model sees input |
| No fallback when model fails | Route to a smaller model or cached response on failure |
| Retrying forever on bad outputs | Circuit breaker; stop calling a consistently failing model |
| Deploying without an incident playbook | Write AI-specific runbooks: bad output, data leak, cost spike |
| Silent guardrail failures | Log every guardrail decision; alert on block rates |

## Related Notes

- [[Reliability/Circuit-Breakers-And-Fallbacks.md]] — Circuit breakers, fallback models, error budgets
- [[AgenticEngineering/Production/Reliability.md]] — Retries, fallbacks, circuit breakers, SLOs
- [[AgenticEngineering/Production/Security.md]] — Auth, authorization, data privacy, PII
- [[Observability/]] — Monitoring and alerting for reliability
- [[MLOps/]] — Rollback and deployment safety

## Guardrail Example: NVIDIA NeMo Guardrails

```python
from nemoguardrails import LLMRails
from nemoguardrails.actions import action

# Configure guardrails
config = {
    "models": [
        {"type": "main", "engine": "openai", "model": "gpt-4o-mini"},
    ],
    "rails": {
        "input": [
            {"definition": "block user messages containing PII"},
            {"definition": "block jailbreak attempts"},
        ],
        "output": [
            {"definition": "block responses with toxic language"},
            {"definition": "block responses containing unverified medical advice"},
        ],
    },
}

rails = LLMRails(config)

# Register custom action for PII detection
@action(is_system_action=True)
async def detect_pii(text: str):
    # PII detection logic here
    return {"contains_pii": False, "entities": []}

# Generate with guardrails
response = await rails.generate_async(
    messages=[{"role": "user", "content": "Hello, how can you help me today?"}]
)
print(response["content"])
```

## Incident Response Playbook

| Incident | Detection | Response | Owner |
|---|---|---|---|
| Hallucinated output | Output guardrail + user report | Trigger fallback model, alert on-call | ML Engineer |
| Data leak in response | Input/output PII guardrail | Block response, revoke tokens, investigate | Security + ML |
| Cost spike | Cost per token alert | Circuit breaker on expensive model, route to cheaper | Platform Eng |
| Model unresponsive | Latency SLO breach | Failover to secondary model/region | SRE |

## Sources

* NVIDIA NeMo Guardrails, LlamaGuard (Meta), Microsoft Guidance, AWS Bedrock Guardrails, Google SRE, AI incident response playbooks, Purple Llama.*