Pin Terraform Modules or Accept Silent Drift

Fourteen roots, six modules, one silent prod change

A platform team maintains fourteen Terraform root modules for dev, staging, and production slices across networking, Kubernetes foundations, and shared data services. Six reusable modules encapsulate VPC patterns, EKS baselines, RDS wrappers, and IAM role factories. Staging applied a module bump on a Tuesday; production still referenced the same semantic version tag but the Git ref was a branch head that moved. Friday’s plan wanted to replace a load balancer listener because a provider default changed upstream. Nobody pinned the module source. The arithmetic is simple: eighty-four pin surfaces, one drift class per week without gates, and an outage budget spent on infrastructure that should have been immutable.

Structural map: env roots and shared modules

Think in three layers: root modules (state per environment), shared service modules (versioned tags), and provider pins (locked in .terraform.lock.hcl). Each root calls shared modules via source with an explicit version or Git ref. Promotion flows dev → staging → prod with plan-only artefacts, not re-run apply from laptop history.

flowchart LR
  subgraph roots
    DEV[dev/root]
    STG[staging/root]
    PRD[prod/root]
  end
  subgraph modules
    NET[networking v3.2.1]
    EKS[eks-base v1.8.0]
    RDS[rds-wrapper v2.0.4]
  end
  DEV --> NET
  DEV --> EKS
  STG --> NET
  STG --> EKS
  PRD --> NET
  PRD --> EKS
  PRD --> RDS

Drift appears when live cloud resources diverge from the last applied Terraform state, or when state matches but code moved because an unpinned module source fetched new defaults.

Integration tax of unpinned modules

Unpinned infrastructure costs engineering time in predictable line items:

• Plan review meetings that compare unexplained resource replacements instead of intentional changes
• Rollback drills that fail because nobody recorded which module Git SHA shipped to prod
• Cross-team tickets when staging and prod disagree on security group rules after a silent patch
• Provider upgrade thrash when lock files are not committed per root
• Audit findings on change control because apply ran from engineer laptops without signed plans
• On-call pages when autoscaling tags drift from the tagging module’s latest default

Each item is a few hours to a few days. Multiplied across quarters, the tax exceeds the one-time cost of pinning, drift jobs, and promotion gates.

Boundary rules for module ownership

Platform teams own modules that encode org policy: CIDR allocation, mandatory tags, logging buckets, KMS keys. Application teams own roots that compose modules with service-specific variables. Anti-corruption boundaries matter: a module should not expose raw provider resources without opinionated defaults; roots should not fork module internals with copy-paste blocks. What stays internal to the module is implementation detail (subnet calculation, IAM policy documents). What crosses the boundary are inputs, outputs, and semver guarantees documented in CHANGELOG.

HashiCorp’s module development guidance on versioning and release constraints recommends semver for registry modules and explicit version = "~> 3.2" constraints in callers. Git-sourced modules need immutable refs: tags or commit SHA, never floating branches in production roots.

Decision table: pin strategy by environment

Environment	Module source ref	Provider lock	Drift detection	Apply gate
Dev sandbox	Branch allowed with TTL	Committed lock, weekly refresh	Optional nightly	Engineer apply OK
Staging	Semver tag only	Same lock as prod target	Required daily plan-only	PR + plan artefact
Production	Tag or SHA pinned in code	Identical to tested staging lock	Hourly plan-only + alert	Signed plan + second approver
Disaster recovery	Same tag as prod	Frozen until DR exercise completes	Weekly	Change window only

Acceptance protocol: prove pins and drift gates work

Run these steps in a sandbox subscription before claiming the programme is mature:

1. Pin all module sources in staging root to explicit tags; commit .terraform.lock.hcl.
2. Execute a deliberate out-of-band change in the cloud console (add a tag manually to a module-managed security group).
3. Run plan-only in Terraform Cloud or CI; confirm non-empty plan and failing policy check.
4. Revert manual change via apply; confirm plan returns empty.
5. Bump a module minor version in dev; run full test suite and plan diff review; promote tag to staging only after automated tests pass.
6. Block prod apply unless CI uploads the plan JSON from staging run with matching module versions and provider lock checksum.

Terraform Cloud plan-only runs support drift detection workflows without apply permissions on production workspaces. Pair with OPA or Sentinel policies that reject plans containing forces replacement on load balancer resources unless a break-glass label is present.

Provider pins and lock file hygiene

Module pins without provider locks invite a second drift class: the same module version plans differently after a provider patch. Commit .terraform.lock.hcl in every root; run terraform providers lock in CI when upgrading. Staging and production roots that target the same cloud estate should share identical provider checksums in the lock file at promotion time. Diff locks in the PR that bumps a module tag; if the provider block changed, rerun integration tests even when module semver looked patch-only.

Remote state backends need their own access boundaries. Drift detection reads state and compares to live APIs; production workspaces should use read-only tokens for plan-only automation and separate credentials for apply. A drift job that can apply accidentally is a future outage.

When custom module libraries still beat a monolithic IDP

A paved path does not require a single internal developer portal UI. Small platform teams often ship faster with a versioned module monorepo, documented root templates, and CI gates than with a bespoke IDP that duplicates Terraform Cloud features. Custom work makes sense when regulatory partitions require separate state backends per region, when module composition varies materially between business units, or when drift detection must integrate with existing SIEM tickets rather than a greenfield portal.

Composable platform engineering is about promotion discipline and module contracts, not about naming every pipeline “golden path.” Teams standardising Terraform across dozens of roots benefit from an explicit module map, pin policy, and drift acceptance tests before expanding scope.

Our write-up on composable platform engineering and paved paths covers how to scope module libraries without building a monolithic IDP. If your drift incidents outnumber feature releases, start with pins and plan-only gates; the portal can wait.