AI-Systems-Engineering
This section bridges traditional Systems Engineering with the AI infrastructure content already in the wiki. Treat AI systems (models, agents, platforms) as engineered systems requiring the same rigor: needs analysis, architecture, trade studies, V&V, and operational excellence.
Philosophy
AI systems are software-intensive, data-centric, and often non-deterministic. Apply SE discipline while adapting for: - Non-determinism - Evaluation over exact assertion - Rapid evolution - Models, prompts, and tools change frequently - Data dependency - Quality of data often matters more than code - Cost sensitivity - Inference costs are real operating expenses - Safety concerns - Guardrails are requirements, not features
Contents
| Topic | File(s) | Purpose |
|---|---|---|
| Inference | Link to ../Inference/ |
Model serving, engines, batching, optimization |
| MLOps/LLMOps | Link to ../MLOps/ |
Deployment, registry, rollout, lifecycle |
| Observability | Link to ../Observability/ |
Tracing, metrics, evals in production |
| Reliability | Link to ../Reliability/ |
Guardrails, circuit breakers, fallbacks |
| EdgeAI | Link to ../EdgeAI/ |
On-device, quantization, local inference |
| Platform | Link to ../Platform/ |
IDP, prompt management, cost showback |
Applying SE to AI Systems
| SE Phase | AI Application |
|---|---|
| Needs | "Reduce support costs by X%" not "Build an LLM app" |
| Requirements | Include eval thresholds (faithfulness > 0.9), latency SLOs, cost limits |
| Trade Studies | Model selection, hosting (API vs self-host), framework choice |
| Architecture | Agent topology, tool design, memory, retrieval |
| V&V | Eval suites, red teaming, human evaluation, A/B tests |
| Risk | Hallucination, PII, prompt injection, cost overruns |
| Operations | Continuous eval, drift detection, model updates |
Related Notes
- Systems Thinking — Holistic thinking
- Needs Vs Requirements — Start from needs
- Trade Study Methodology — Model selection trade studies
- V Model And VV Planning — Eval-driven V&V
- Resilience Engineering — AI system resilience
- Production — Production patterns
- Evaluation — Evaluation frameworks
Sources
- ML Engineering by Chip Huyen
- Designing Machine Learning Systems by Chip Huyen
- LLM Engineer's Handbook
- OpenAI Cookbook, Anthropic docs
- Eugene Yan's blog (eugeneyan.com)
- vLLM/SGLang documentation