Reliability
Guardrails, failover, safety, and incident response for AI systems in production.
Core Concepts
| Concept |
Description |
| Guardrails |
Input/output filters that block unsafe or off-topic content |
| Circuit Breaker |
Temporarily stop calling a failing model or dependency |
| Fallback Model |
A smaller/cheaper model that takes over when the primary fails |
| Retry with Backoff |
Exponential backoff for transient inference failures |
| Error Budget |
Allowed failures before triggering remediation |
| Incident Response |
Playbooks for AI-specific incidents: bad outputs, data leaks, cost spikes |
Decision Guidance
| Situation |
Recommended Approach |
| User-facing chatbot |
Input guardrails (PII, jailbreak) + output guardrails (toxicity, hallucination) |
| Safety-critical system |
Fallback model + human-in-the-loop for high-stakes outputs |
| High cost per request |
Circuit breaker on expensive models; route to cheaper fallback |
| Model degradation |
Error budget triggers rollback to previous model version |
Anti-Patterns
| Anti-Pattern |
Fix |
| No guardrails on user input |
Add PII and jailbreak detection before the model sees input |
| No fallback when model fails |
Route to a smaller model or cached response on failure |
| Retrying forever on bad outputs |
Circuit breaker; stop calling a consistently failing model |
| Deploying without an incident playbook |
Write AI-specific runbooks: bad output, data leak, cost spike |
- Reliability — Retries, fallbacks, circuit breakers, SLOs
- Security — Auth, authorization, data privacy, PII
- Observability — Monitoring and alerting for reliability
- MLOps — Rollback and deployment safety
Sources
- Google SRE, NVIDIA NeMo Guardrails, LlamaGuard, Meta Purple Llama, AWS Bedrock Guardrails, Microsoft Guidance, AI incident response playbooks.*
