Model Registry and Lineage

Versioned model store with metadata, lineage, and reproducibility for AI systems: tracking data, code, config, and deployment history per model version.

Core Concepts

Concept	Description
Model Registry	Central store of versioned models with metadata (framework, metrics, creator, stage)
Model Version	Immutable snapshot: weights + config + code version + data version + metrics
Model Stage	Lifecycle state: `None`, `Staging`, `Production`, `Archived`
Lineage	End-to-end trace: dataset → preprocessing → training run → model artifact → deployment
Data Version	Immutable dataset snapshot with hash, schema, and provenance
Model Signature	Schema of model inputs/outputs for type-safe serving
Model Alias	Human-readable pointer to a version (e.g. `champion`, `challenger`)

Registry Schema Example

models:
  - name: customer-support-llm
    versions:
      - version: "3"
        stage: Production
        created_at: "2026-01-15T10:30:00Z"
        created_by: ml-platform-team
        model_uri: s3://ml-models/customer-support-llm/v3/
        framework: transformers
        framework_version: "4.45.0"

        # Lineage
        data_version: support-tickets-v7
        training_run_id: trn-9f2a1b
        code_commit: a1b2c3d

        # Metrics
        metrics:
          eval_accuracy: 0.94
          cost_per_1k_tokens_usd: 0.0012
          p99_latency_ms: 1450
          hallucination_rate: 0.02

        # Signature
        signature:
          inputs:
            - name: prompt
              type: string
          outputs:
            - name: response
              type: string

        aliases:
          - champion

      - version: "4"
        stage: Staging
        created_at: "2026-01-20T14:00:00Z"
        aliases:
          - challenger

Decision Guidance

Situation	Recommended Approach
First model deployment	Simple registry: train → eval → register → deploy to staging
Frequent model updates	Continuous training + automated eval gate before Production stage
Multi-model serving	Model registry + A/B testing framework (champion vs challenger)
Regulated environment	Full lineage: data version → model version → deployment record, audit trail

Anti-Patterns

Anti-Pattern	Fix
Deploying models without eval gate	Add evaluation step before any Production stage promotion
No model versioning	Use a model registry; track data + code + config per version
Rollback requiring redeploy	Keep previous model version in Production; rollback = stage change
Training on unversioned data	Version datasets; record data hash with each model artifact
No model signature	Enforce input/output schema for type-safe serving

Canary And Rollback — Canary deployment and rollback strategies
Deployment Architecture — Serverless, containers, edge, hybrid deployment
Cost Optimization — Caching, model selection, token budgets
Inference — Model serving and inference engines
Observability — Monitoring deployed models
Platform — Internal tooling for ML teams

Model Promotion Pipeline

stateDiagram-v2
    [*] --> Staging: register + pass eval
    Staging --> Production: champion alias + approval
    Production --> Archived: superseded
    Production --> Staging: rollback / demote
    Staging --> Archived: failed eval

    note right of Staging: Automated eval gate required
    note right of Production: Human approval for regulated envs

Sources

MLflow Model Registry, Weights & Biases Model Registry, Kubeflow, SageMaker Model Registry, Google Vertex AI Model Registry, ML Engineering (Chip Huyen), MLOps community, LLMOps papers.*

From the Systems Engineering research vault · CC BY 4.0 · all notes · raw markdown