Platform
Developer experience, self-service, and internal tooling for AI platform teams.
Core Concepts
| Concept |
Description |
| Internal Developer Platform (IDP) |
Self-service platform for AI teams to deploy models and apps |
| Model Serving as a Service |
Managed inference endpoints abstracting infrastructure |
| Prompt Management |
Versioned, reviewed prompt templates shared across teams |
| Cost Showback |
Attributing AI inference/training costs to teams and products |
| Golden Paths |
Paved-road templates for common AI workloads (RAG, fine-tuning, eval) |
| Self-Service Evals |
Teams run their own evals without platform team involvement |
Decision Guidance
| Situation |
Recommended Approach |
| Many AI teams, one platform |
Build an IDP with golden paths for RAG and fine-tuning |
| Cost visibility needed |
Tag inference/training spend by team; monthly showback |
| Prompt sprawl |
Central prompt registry with versioning and review |
| Repeated infra setup |
Paved-road templates (Terraform/Helm) for common workloads |
Anti-Patterns
| Anti-Pattern |
Fix |
| Every team reinvents model serving |
Golden path: one supported serving stack |
| No cost visibility |
Tag all AI spend; showback to teams |
| Prompts hardcoded in apps |
Central prompt registry with versioning |
| Platform team becomes bottleneck |
Self-service evals and deployment; platform enables, doesn't gate |
- Deployment Architecture — Serverless, containers, edge, hybrid
- Cost Optimization — Caching, model selection, token budgets
- MLOps — Deployment and training pipelines
- Inference — Model serving and inference engines
- Observability — Monitoring AI systems
Sources
- Backstage, Humanitec, AWS Proton, Google Cloud AI Platform, Azure ML, Internal Developer Platform patterns, Team Topologies, platform engineering community.*
