Starphix
Network Engineering / Sources
The Lab / Sources / Network Engineering / Automation-Observability/NetDevOps-Fundamentals.md

NetDevOps Fundamentals

NetDevOps applies software practice to network operations: configurations in version control, changes through a pipeline, tests before deployment, and telemetry after.

Why it exists

Manual network change is slow, unaudited, and inconsistent. Two devices configured by two engineers in two sittings are two different devices. NetDevOps makes the network declarative — you describe the intended state, and a tool converges reality to it.

Manual NetDevOps
SSH, paste commands Commit a change, pipeline applies it
No history Git history, review, blame
Drift discovered during an outage Drift detected continuously
Testing in production Pre-change validation, canary, rollback
Knowledge in one engineer's head Intent in a repository

The stack

Layer Tools What it does
Source of truth Git, NetBox, Infrahub Desired state and inventory
Templating Jinja2, Go templates Config from data, not copy-paste
Transport SSH/CLI, NETCONF (830), RESTCONF (443), gNMI (gRPC) How config reaches the device
Models YANG, OpenConfig Schema for the data
Orchestration Ansible, Nornir, Salt, Terraform Run the work across devices
Validation Batfish, pyATS, containerlab, SuzieQ Prove the change before and after
CI/CD GitHub Actions, GitLab CI, Argo The pipeline

Two philosophies

Imperative Declarative
Shape "Run these commands" "This is the state; make it so"
Tool Ansible CLI modules, Nornir Terraform, NETCONF edit-config, gNMI Replace
Drift Invisible until you look Detected by design
Risk Re-running is not idempotent Re-running is safe

Idempotence is the property that makes automation safe: running the same play twice must not change anything the second time.

Minimal viable pipeline

# .github/workflows/network.yml (shape, not a finished file)
on: [pull_request]
jobs:
  validate:
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt
      - run: python render.py --check        # templates render, no syntax errors
      - run: python validate.py --lint       # schema + policy checks
      - run: python test_batfish.py          # reachability assertions
  deploy:
    if: github.ref == 'refs/heads/main'
    steps:
      - run: python deploy.py --canary 1     # one device first
      - run: python verify.py                # post-change assertions
      - run: python deploy.py --rest        # the fleet

The five properties worth having on day one: render, lint, test reachability, deploy to one device, verify, then continue.

What to automate first

  1. Backups and config collection — read-only, instantly useful, zero risk.
  2. Compliance checks — does every device have NTP, syslog, an ACL, a banner?
  3. Inventory — the source of truth every later step depends on.
  4. Low-risk changes — VLANs, descriptions, monitoring config.
  5. Full change management — last, and only after the first four are boring.

Automating the riskiest change first is how automation projects fail.

Where it goes wrong

  • No source of truth. Two Git repos and a spreadsheet disagree, and the pipeline deploys whichever it read.
  • Automating a bad process. If the manual process is wrong, the pipeline makes it wrong faster and at scale.
  • No rollback. A canary without a defined rollback is a hope.
  • Templates with hidden logic. Config that depends on three nested conditionals is unreadable and untestable.
  • Secrets in the repo. Credentials in Git, even briefly, are compromised.
  • Partial failure. Device 40 of 100 fails; is the estate half-configured, or rolled back? Decide in advance.
  • No drift detection. Automation without verification just means the drift happens faster.

From the Network Engineering research vault · CC BY 4.0 · all notes · raw markdown