# Case Studies: Software-Intensive Systems

Bridging Systems Engineering with Software and Agentic Engineering through practical examples.

## Case Study 1: Building a RAG-Powered Knowledge Assistant

### Context
A company wants to help employees find answers from internal documentation. Business need: reduce time spent searching for information.

### Applying Systems Engineering

| SE Activity | Application |
|---|---|
| **Stakeholder Needs** | "Find answers quickly", "Trust the sources", "Don't hallucinate" |
| **System Requirements** | Response time < 3s, citations required, grounded in approved docs only |
| **Trade Study** | Vector DB choice (pgvector vs Weaviate vs Pinecone) - see [[../Trade-Studies/Trade-Study-Methodology]] |
| **Architecture** | Ingestion → Chunking → Embedding → Vector Store → Retrieval → Reranking → Generation |
| **Interfaces** | API contract, MCP tool for other agents, chunking interface |
| **V&V** | Retrieval eval (recall@k), faithfulness eval (LLM-as-judge), latency tests |
| **Risks** | Hallucination, stale docs, PII leakage, cost overruns |
| **Operations** | Ingestion pipeline monitoring, evals on live traffic, cost tracking |

### Key Lessons
- **Separation of concerns**: Retrieval and generation are separate - can optimize independently
- **Eval is requirements**: "Grounded in sources" needs measurable criteria
- **Guardrails matter**: "Don't answer outside knowledge base" needs enforcement
- **Traceability**: Link eval failures back to chunks/retrieval issues

## Case Study 2: Multi-Agent Customer Support System

### Context
Automate tier-1 support with multiple specialized agents (triage, knowledge, escalation) while maintaining human oversight.

### Applying Systems Engineering

| SE Activity | Application |
|---|---|
| **Needs Analysis** | Reduce response time, maintain CSAT, escalate complex issues |
| **Stakeholder Requirements** | 24/7 availability for simple issues, human escalation path, audit trail |
| **System Requirements** | Confidence threshold for auto-response, PII redaction, conversation logging |
| **Architecture** | Supervisor pattern with specialized agents, shared memory, tool access |
| **Trade Study** | Orchestration framework (LangGraph vs CrewAI vs custom) |
| **V&V** | Per-agent evals + end-to-end conversation evals + red teaming |
| **Safety** | Input/output guardrails, rate limiting, escalation rules |
| **Resilience** | Fallback to human, circuit breakers, graceful degradation |

### Key Lessons
- **Handoffs are interfaces**: Agent-to-agent handoffs need clear contracts
- **Observability is critical**: Need traces across multiple agents
- **Validation is hard**: Conversation quality is subjective - need rubrics
- **Human-in-the-loop**: Design escalation as first-class feature, not afterthought

## Case Study 3: Platform Engineering for AI - Internal Model Serving

### Context
Build an internal developer platform for AI teams to self-serve model deployment with governance.

### Applying Systems Engineering

| SE Activity | Application |
|---|---|
| **Stakeholder Needs** | "Easy to deploy models", "Control costs", "Stay secure/compliant" |
| **System Requirements** | Self-service API, cost attribution, RBAC, audit logs, eval gates |
| **Architecture** | Control plane + data plane, model registry, routing layer |
| **Trade Study** | Build vs buy (vLLM on K8s vs managed service) |
| **Interfaces** | Platform API, CLI, golden path templates |
| **V&V** | Platform reliability SLOs, security review, developer UX testing |
| **Risk Management** | GPU availability, cost runaway, model abuse |
| **Lifecycle** | Onboarding, deprecation, capacity planning |

### Key Lessons
- **Platform as product**: Treat internal users as customers
- **Golden paths reduce complexity**: Paved roads with guardrails
- **Cost is a constraint**: Need showback/chargeback from day one
- **Standards matter**: OpenAI-compatible APIs reduce friction

## Cross-Cutting Lessons

| Lesson | How SE Helps |
|---|---|
| **"Build vs buy" ambiguity** | Trade studies make it objective |
| **Scope creep** | Requirements traceability keeps focus |
| **Production surprises** | V&V + resilience engineering |
| **Team misalignment** | Clear needs/requirements prevent this |
| **Technical debt** | Lifecycle thinking includes maintenance costs |
| **AI non-determinism** | Eval-driven validation instead of exact-match tests |

## Connecting to Other Wikis

- **Software Engineering**: Architecture patterns, testing strategies apply
- **Agentic Engineering**: Agent patterns, tool use, evaluation frameworks
- **Network Engineering**: Connectivity, load balancing, security for distributed systems

## Exercises

1. **RAG System**: Write stakeholder needs, derive 3-5 system requirements, create a simple trade study
2. **Agent System**: Design interfaces between agents, plan V&V including evals
3. **Platform**: Do a build vs buy trade study with real criteria and weights

## Sources

* Real-world patterns from production AI systems  
* INCOSE case studies  
* AWS Well-Architected Framework  
* Team Topologies  
* Platform Engineering patterns