# MLOps

Deployment, training pipelines, LLMOps, rollout, and lifecycle management for AI systems.

## Core Concepts

| Concept | Description |
|---|---|
| **CI/CD for ML** | Pipelines that test data, models, and code before deployment |
| **Model Registry** | Versioned store of trained models with metadata and lineage |
| **Continuous Training** | Retraining on new data on a schedule or trigger |
| **Canary Deployment** | Routing a fraction of traffic to a new model before full rollout |
| **Rollback** | Reverting to a previous model version on regression |
| **Data Versioning** | Versioning datasets alongside model versions for reproducibility |
| **LLMOps** | MLOps practices specific to large language models: prompt versioning, eval gates |

## Decision Guidance

| Situation | Recommended Approach |
|---|---|
| First model deployment | Simple pipeline: train → eval → register → deploy to staging |
| Frequent model updates | Continuous training + canary rollout with automated eval gate |
| Multi-model serving | Model registry + A/B testing framework |
| Regulated environment | Full lineage: data version → model version → deployment record |

## Anti-Patterns

| Anti-Pattern | Fix |
|---|---|
| Deploying models without eval | Add an eval gate before any production rollout |
| No model versioning | Use a model registry; track data + code + config per version |
| Rollback requiring redeploy | Keep previous model version hot; rollback is a config change |
| Training on unversioned data | Version datasets; record data hash with each model |

## Related Notes

- [[AgenticEngineering/Production/Deployment-Architecture.md]] — Serverless, containers, edge, hybrid deployment
- [[AgenticEngineering/Production/Cost-Optimization.md]] — Caching, model selection, token budgets
- [[Inference/]] — Model serving and inference engines
- [[Observability/]] — Monitoring deployed models
- [[Platform/]] — Internal tooling for ML teams

## Sources

* MLflow, Weights & Biases, Kubeflow, SageMaker Pipelines, Google Vertex AI, ML Engineering book, Chip Huyen, MLOps community, LLMOps papers.*