Case Studies: Software-Intensive Systems
Bridging Systems Engineering with Software and Agentic Engineering through practical examples.
Case Study 1: Building a RAG-Powered Knowledge Assistant
Context
A company wants to help employees find answers from internal documentation. Business need: reduce time spent searching for information.
Applying Systems Engineering
| SE Activity | Application |
|---|---|
| Stakeholder Needs | "Find answers quickly", "Trust the sources", "Don't hallucinate" |
| System Requirements | Response time < 3s, citations required, grounded in approved docs only |
| Trade Study | Vector DB choice (pgvector vs Weaviate vs Pinecone) - see Trade Study Methodology |
| Architecture | Ingestion → Chunking → Embedding → Vector Store → Retrieval → Reranking → Generation |
| Interfaces | API contract, MCP tool for other agents, chunking interface |
| V&V | Retrieval eval (recall@k), faithfulness eval (LLM-as-judge), latency tests |
| Risks | Hallucination, stale docs, PII leakage, cost overruns |
| Operations | Ingestion pipeline monitoring, evals on live traffic, cost tracking |
Key Lessons
- Separation of concerns: Retrieval and generation are separate - can optimize independently
- Eval is requirements: "Grounded in sources" needs measurable criteria
- Guardrails matter: "Don't answer outside knowledge base" needs enforcement
- Traceability: Link eval failures back to chunks/retrieval issues
Case Study 2: Multi-Agent Customer Support System
Context
Automate tier-1 support with multiple specialized agents (triage, knowledge, escalation) while maintaining human oversight.
Applying Systems Engineering
| SE Activity | Application |
|---|---|
| Needs Analysis | Reduce response time, maintain CSAT, escalate complex issues |
| Stakeholder Requirements | 24/7 availability for simple issues, human escalation path, audit trail |
| System Requirements | Confidence threshold for auto-response, PII redaction, conversation logging |
| Architecture | Supervisor pattern with specialized agents, shared memory, tool access |
| Trade Study | Orchestration framework (LangGraph vs CrewAI vs custom) |
| V&V | Per-agent evals + end-to-end conversation evals + red teaming |
| Safety | Input/output guardrails, rate limiting, escalation rules |
| Resilience | Fallback to human, circuit breakers, graceful degradation |
Key Lessons
- Handoffs are interfaces: Agent-to-agent handoffs need clear contracts
- Observability is critical: Need traces across multiple agents
- Validation is hard: Conversation quality is subjective - need rubrics
- Human-in-the-loop: Design escalation as first-class feature, not afterthought
Case Study 3: Platform Engineering for AI - Internal Model Serving
Context
Build an internal developer platform for AI teams to self-serve model deployment with governance.
Applying Systems Engineering
| SE Activity | Application |
|---|---|
| Stakeholder Needs | "Easy to deploy models", "Control costs", "Stay secure/compliant" |
| System Requirements | Self-service API, cost attribution, RBAC, audit logs, eval gates |
| Architecture | Control plane + data plane, model registry, routing layer |
| Trade Study | Build vs buy (vLLM on K8s vs managed service) |
| Interfaces | Platform API, CLI, golden path templates |
| V&V | Platform reliability SLOs, security review, developer UX testing |
| Risk Management | GPU availability, cost runaway, model abuse |
| Lifecycle | Onboarding, deprecation, capacity planning |
Key Lessons
- Platform as product: Treat internal users as customers
- Golden paths reduce complexity: Paved roads with guardrails
- Cost is a constraint: Need showback/chargeback from day one
- Standards matter: OpenAI-compatible APIs reduce friction
Cross-Cutting Lessons
| Lesson | How SE Helps |
|---|---|
| "Build vs buy" ambiguity | Trade studies make it objective |
| Scope creep | Requirements traceability keeps focus |
| Production surprises | V&V + resilience engineering |
| Team misalignment | Clear needs/requirements prevent this |
| Technical debt | Lifecycle thinking includes maintenance costs |
| AI non-determinism | Eval-driven validation instead of exact-match tests |
Connecting to Other Wikis
- Software Engineering: Architecture patterns, testing strategies apply
- Agentic Engineering: Agent patterns, tool use, evaluation frameworks
- Network Engineering: Connectivity, load balancing, security for distributed systems
Exercises
- RAG System: Write stakeholder needs, derive 3-5 system requirements, create a simple trade study
- Agent System: Design interfaces between agents, plan V&V including evals
- Platform: Do a build vs buy trade study with real criteria and weights
Sources
- Real-world patterns from production AI systems
- INCOSE case studies
- AWS Well-Architected Framework
- Team Topologies
- Platform Engineering patterns