Test Pyramid & Testing Strategies

A strategic model for software testing emphasizing an optimized approach across different test levels.

The Test Pyramid

                    ┌─────────────┐
                    │   E2E /     │  ← 10% — Critical user journeys
                    │   UI Tests  │     Slow, brittle, expensive
                    ├─────────────┤
                    │ Integration │  ← 20% — Component interactions
                    │  Tests      │     Real dependencies, moderate speed
                    ├─────────────┤
                    │   Unit      │  ← 70% — Single units in isolation
                    │   Tests     │     Fast, deterministic, cheap
                    └─────────────┘

Core Principle: Prefer fast, deterministic tests you fully control. Lower-level tests are cheaper to write, faster to run, and more reliable.

Test Levels Defined

Unit Tests (Base — ~70%)

• Scope: Single class/function/module in isolation
• Dependencies: All external dependencies mocked/stubbed
• Speed: Milliseconds
• Determinism: High
• Goal: Verify logic, edge cases, error handling

Best practices: - Test behavior, not implementation (black box) - Use test doubles for external dependencies - Validate test doubles with contract tests - Keep under 10 minutes for full suite - Delete tests that never fail (maintenance cost, no benefit)

Integration Tests (Middle — ~20%)

• Scope: Interactions between components
• Dependencies: One real external dependency at a time (narrow integration)
• Speed: Seconds
• Determinism: Medium
• Goal: Verify wiring, contracts, data flow

Narrow vs Broad Integration: | Narrow (Good) | Broad (Avoid) | |---|---| | One real DB, mocked HTTP | Real DB + real HTTP + real queue | | Run locally/ephemerally | Require full test environment | | Fast, deterministic | Slow, flaky |

End-to-End Tests (Top — ~10%)

• Scope: Full user journeys through deployed system
• Dependencies: All real (production-like)
• Speed: Minutes
• Determinism: Low (flaky)
• Goal: Validate critical happy paths, system wiring

E2E Strategy: - Automate only critical user flows - Limit scope to high-risk areas - Run in dedicated environment with versioned test data - Monitor, don't gate (run on schedule, not every commit)

Contract Testing

Consumer-Driven Contracts: Consumers write tests defining what they need from an interface; providers verify they satisfy those contracts.

Consumer                    Provider
   │                          │
   ├─ Contract Test ────────► │ (generates mock)
   │                          │
   │ (uses mock)              │
   │                          │
   │                          ├─ Provider Verification ────► (runs against real impl)
   │                          │

Benefits: - Decouples deployability — consumer and provider deploy independently - Catches breaking changes early - Replaces broad integration tests for API boundaries - Enables independent team autonomy

Tools: Pact, Spring Cloud Contract, GraphQL schema validation

Test-Driven Development (TDD)

Cycle: Red → Green → Refactor

1. RED:   Write a failing test for the desired behavior
2. GREEN: Write minimal code to make it pass
3. REFACTOR: Improve design while keeping tests green

Three Objectives: 1. Prevent forgotten tests — Testing first makes it hard to postpone 2. Encourage testable design — Designing from the consumer perspective 3. Design practice — First user of your code is its test

TDD is not just testing — it's a design practice that yields: - >90% test coverage naturally - Simpler interfaces (you're the first consumer) - Code that's easier to change

Modern Test Strategy: Beyond the Pyramid

Optimize for Feedback, Not Taxonomy

Layer names vary (unit/service/component/contract/E2E). What matters operationally: 1. Scope — What is included? 2. Speed — Time-to-signal 3. Determinism — Flake rate 4. Diagnosability — How localized the failure is

If a suite is "correctly named" but slow, flaky, and untrusted, it is functionally broken.

Risk-Based Allocation

1. List critical scenarios and invariants (domain + NFR)
2. Identify smallest scope that can falsify each claim
3. Allocate evidence to lowest level possible
4. Add higher-level tests only for: - Wiring/infra semantics not provable lower - Contract compatibility - Minimal E2E journeys

The "Push Down" Rule (Anti-Duplication Discipline)

When a high-level test fails: - If the failure cannot be reproduced with a lower-level test → add the missing lower-level test and reduce reliance on the high-level one - Keeps the pyramid from inverting over time

Time Budgets (Governance)

Without budgets, scope creeps upward and the pyramid collapses.

Flake Budget

• Adopt a clear policy: flaky tests are blocked, not tolerated
• Quarantine flaky tests immediately
• Fix or delete — don't let them erode trust

Test Code Quality

Treat test code as production code: - Same care, review, refactoring attention - DRY applies (but to knowledge, not text) - Clear naming: should_return_error_when_input_invalid - Arrange-Act-Assert (or Given-When-Then) structure

Accessibility Testing

• WCAG compliance scans are fast, deterministic
• Run on every commit
• Automate detectable rules; reserve manual for subjective judgment

Testing in CI/CD Pipeline

Common Anti-Patterns

Related Notes

• Test Driven Development — TDD cycle and practices
• Testing Strategies — Property-based, mutation, chaos testing
• Test Organization — Structure, naming, fixtures
• CI CD Best Practices — Pipeline integration
• Resilience Patterns — Testing resilience (chaos, fault injection)

Sources: Martin Fowler (Practical Test Pyramid), MinimumCD Practice Guide, GitLab Testing Guide, Home Office Engineering Standards, Software Engineering: A Modern Approach (Chapter 8), Pactflow, Google Testing Blog