The Terralith Trap: Why Scaling Terraform Demands a Shift to Decentralized Infrastructure Delivery
San Francisco, CA — In the early stages of an organization’s cloud journey, managing infrastructure is a straightforward affair. A single systems engineer or a small DevOps team oversees a handful of repositories, a single state file, one deployment pipeline, and a unified set of cloud credentials. There is no coordination overhead because there is no one to coordinate with.
However, industry data reveals that this centralized model rapidly collapses the moment a second, third, or fourth team—spanning networking, platform engineering, application development, and security—attempts to deploy infrastructure simultaneously. What once was an agile development process transforms into a system plagued by pipeline gridlocks, security vulnerabilities, and organizational bottlenecks.
As enterprises scale, the traditional "monolithic Terraform" approach—increasingly referred to in the engineering community as a "terralith"—actively slows down product delivery. This investigative report examines the structural failures of centralized Infrastructure as Code (IaC), traces the historical evolution of scaling workarounds, presents community-driven architectural solutions, and outlines the implications of moving toward a decentralized, declarative delivery model.
1. Main Facts: The Structural Failures of Centralized IaC
When multiple engineering teams attempt to operate within a shared Terraform environment, they encounter a series of systemic bottlenecks. These friction points are not merely operational inconveniences; they are fundamental limitations of how state-based configuration engines manage cloud resources.
+-------------------------------------------------------------+
| THE TERRALITH TRAP |
| A single, monolithic state file leads to systemic failures: |
+-------------------------------------------------------------+
|
+----------------------+----------------------+
| | |
v v v
+---------------+ +---------------+ +---------------+
| State Lock | | Massive Blast | | Credential |
| Contention | | Radius | | Sprawl |
| Pipeline grid | | Accidental | | Shared keys |
| lock across | | deletion of | | fail audits |
| teams. | | core VPCs. | | in CI/CD. |
+---------------+ +---------------+ +---------------+State Lock Contention
Terraform utilizes state locking to prevent concurrent writes to its state file, ensuring that two processes do not attempt to modify the same resources simultaneously. However, state locking is applied at the state-file level, not the resource level.
If a platform engineer runs a terraform plan to adjust a minor security group rule, the entire deployment pipeline is locked. During this window, an application team attempting to deploy a new microservice is blocked—even though their resources have no logical connection to the security group being modified. As the number of teams increases, developers spend more time waiting for locks to release than they do deploying code.
Blast Radius Vulnerabilities
In a monolithic configuration, all resources exist within the same state boundaries. If application-level microservices and core networking resources (such as VPCs, transit gateways, and route tables) share a state file, they also share a blast radius.
A configuration error or a misplaced variable in a developer’s pull request can trigger an unintended destruction of critical shared infrastructure. While manual code reviews are designed to catch these anomalies, human error remains an inevitable variable. In a shared state model, a single terraform apply has the implicit authority to take down an entire corporate network.
Credential Sprawl and Compliance Failures
To execute a centralized pipeline, the underlying CI/CD runner must possess highly privileged credentials. It requires write access to the networking team’s cloud accounts, the application team’s database clusters, and the security team’s audit logs.
This centralization of secrets creates a severe security risk. Any developer with the ability to trigger a CI run can potentially extract these high-value secrets or execute unauthorized operations across different cloud environments. For organizations subject to SOC2, ISO 27001, or PCI-DSS compliance, this lack of privilege isolation is an automatic audit failure.
Knowledge Boundaries and Cognitive Load
Modern cloud architectures are highly specialized. Networking teams understand BGP peering, routing tables, and firewalls; application teams specialize in container orchestration, caching layers, and database schemas.
Forcing these distinct domains into a single Terraform codebase requires developers to understand the entire infrastructure graph. Application engineers must navigate complex networking code to avoid breaking dependencies, while platform engineers are forced to review application-specific resource allocations. This high cognitive load slows down onboarding and increases the likelihood of misconfigurations.
2. Chronology: The Evolution of IaC Governance
The industry’s struggle to scale IaC has played out over the last decade through a series of evolutionary phases, each attempting to solve the limitations of the previous era.
2014-2016: The Monolith Era (The "Terralith")
Single repository, shared state file. Simple but highly fragile.
│
▼
2017-2019: The Workspace & Wrapper Era
Introduction of Terraform Workspaces and Terragrunt to split state.
│
▼
2020-2022: The Platform Bottleneck
Creation of dedicated platform teams acting as gatekeepers.
│
▼
2023-Present: Modern Declarative Orchestration
Decoupled, event-driven IaC pipelines with granular access controls.Phase 1: The Monolith Era (2014–2016)
In the early days of Terraform, organizations managed their entire infrastructure in a single repository. State files were stored locally or in shared S3 buckets without locking. As teams grew from two to ten engineers, frequent state corruption and overwritten changes forced the adoption of backend state locking (introduced in Terraform 0.9 in 2017).
Phase 2: The Workspace and Wrapper Era (2017–2019)
To combat monolithic state files, organizations turned to Terraform Workspaces. However, workspaces were designed to manage multiple instances of the same infrastructure (such as development, staging, and production environments), not to divide ownership of different infrastructure components within the same environment.
To bridge this gap, open-source wrappers like Terragrunt emerged. Terragrunt allowed teams to define dependencies between different state files and keep their code dry. While successful, it introduced a new layer of abstraction, requiring teams to master terragrunt.hcl configurations, manage custom dependency blocks, and debug issues across two separate tools.
Phase 3: The Platform Bottleneck (2020–2022)
As organizations embraced "Platform Engineering," many established a central platform team tasked with owning all IaC assets. To deploy infrastructure, application teams were required to submit tickets or fill out simplified YAML forms.
While this model protected core infrastructure, it transformed the platform team into a human queue. Application developers faced multi-day delays for minor database provisioning or DNS updates, defeating the core promise of cloud agility.
Phase 4: Modern Declarative Orchestration (2023–Present)
Today, the industry is shifting toward a decentralized, event-driven model. Organizations are seeking frameworks that allow individual teams to independently own their respective infrastructure slices while programmatically sharing outputs (such as subnet IDs or database connection strings) through secure, declarative APIs.
3. Supporting Data: The Technical Friction Points
The demand for a scalable, multi-root-module architecture is validated by longstanding open-source community discussions and technical bottlenecks.
For years, users have raised critical issues on HashiCorp’s public repositories regarding the platform’s architectural constraints:
- Implicit Access Control (GitHub Issue #23874): Terraform lacks native, granular access controls within backends. Providing a user or pipeline access to a specific backend state file often implicitly grants access to all other states hosted within that same storage container or workspace.
- Dynamic Backend Configurations (GitHub Issue #16627 & #13022): The inability to use variables within Terraform’s
backendconfiguration block has forced teams to maintain duplicate code or rely on external bash scripts to dynamically inject backend keys. - The "Terralith" Reform Movement: Within the OpenTofu (the open-source Linux Foundation fork of Terraform) community, Issue #2860, which proposes native mechanisms to deprecate monolithic "terraliths" in favor of decoupled multi-root modules, has received significant community backing.
+-----------------------------------------------------------------------------+
| ORGANIZATIONAL BOUNDARY MAPPING (CONWAY'S LAW) |
+-----------------------------------------------------------------------------+
|
+------------------------------+------------------------------+
| | |
v v v
+---------------+ +---------------+ +---------------+
| Platform Team | | App Team A | | Security Team |
| Namespace | | Namespace | | Namespace |
+---------------+ +---------------+ +---------------+
| | |
+------------+ | |
| | |
v v |
+------------+ +------------+ |
| Networking | | Database | |
| Module | | Module | |
+------------+ +------------+ |
| ^ |
| vpc_id | |
+~~~~~~~~~~~~~~~~~+ v
(Auto-wired Dependency) +---------------+
| IAM Policy |
| Module |
+---------------+4. Community & Industry Responses: Modern Solutions
In response to these systemic limitations, modern infrastructure continuous delivery (CD) platforms have introduced native abstractions designed to decouple team workflows without sacrificing central governance.
Platforms like Snap CD have introduced architectural patterns that map directly to enterprise organizational structures.
Structural Hierarchies: Stacks, Namespaces, and Modules
Rather than forcing all configurations into a single repository, modern architectures break infrastructure down into logical administrative units:
# Defining a global production environment
resource "snapcd_stack" "prod"
name = "prod"
# Establishing a secure boundary for the core platform team
resource "snapcd_namespace" "platform"
name = "platform"
stack_id = snapcd_stack.prod.id
# Establishing a secure boundary for an application engineering team
resource "snapcd_namespace" "app_a"
name = "app-a"
stack_id = snapcd_stack.prod.id
# The platform team's networking module, run on a dedicated network runner
resource "snapcd_module" "networking"
name = "networking"
namespace_id = snapcd_namespace.platform.id
source_url = "https://github.com/myorg/infra-networking.git"
source_revision = "main"
runner_id = data.snapcd_runner.platform.id
# The application team's database module, isolated from core network code
resource "snapcd_module" "app_a_database"
name = "database"
namespace_id = snapcd_namespace.app_a.id
source_url = "https://github.com/myorg/app-a-database.git"
source_revision = "main"
runner_id = data.snapcd_runner.app_a.id
Granular Role-Based Access Control (RBAC)
By decoupling configurations into namespaces, organizations can enforce strict Role-Based Access Control (RBAC) that mirrors their org chart. Teams are granted autonomy over their own domains while remaining restricted from modifying upstream resources:
# Platform team receives full ownership over the platform namespace
resource "snapcd_namespace_role_assignment" "platform_team"
principal_id = snapcd_group.platform_team.id
principal_discriminator = "Group"
role_name = "Owner"
namespace_id = snapcd_namespace.platform.id
# Application Team A receives full ownership over their app namespace
resource "snapcd_namespace_role_assignment" "app_a_team"
principal_id = snapcd_group.app_a_team.id
principal_discriminator = "Group"
role_name = "Owner"
namespace_id = snapcd_namespace.app_a.id
# Application Team A is granted read-only access to view platform outputs
resource "snapcd_namespace_role_assignment" "app_a_reads_platform"
principal_id = snapcd_group.app_a_team.id
principal_discriminator = "Group"
role_name = "Reader"
namespace_id = snapcd_namespace.platform.id
5. Implications: The Blueprint for Decentralized Infrastructure
The transition from monolithic IaC to a decentralized model changes how organizations manage risk, speed, and governance. To successfully implement this structure, enterprise architects must adopt several key operating principles.
| Team | Namespace | Managed Modules | Scoped Runner & IAM Permissions |
|---|---|---|---|
| Platform | prod/platform |
Networking, DNS, Shared Services | runner-platform (Azure Subscription Owner / Network Contributor) |
| App Team A | prod/app-a |
API Databases, Redis Cache, S3 Storage | runner-app-a (AWS IAM Role scoped strictly to App-A Resource Group) |
| App Team B | prod/app-b |
SQS Queues, Lambda Functions | runner-app-b (AWS IAM Role scoped strictly to App-B Account) |
1. Enforce Hard Security Boundaries with Isolated Runners
Organizations must move away from shared, all-powerful CI/CD runners. Each team should operate their own dedicated runner with scoped, least-privilege cloud credentials.
The platform team’s runner should only possess the IAM permissions required to manipulate core networking and global services. An application team’s runner should only be authorized to provision resources within their designated resource groups or AWS sub-accounts. If an application team’s pipeline is compromised, the blast radius is restricted to their specific application namespace.
2. Implement Declarative, Event-Driven Dependency Chains
In a decoupled architecture, teams must still share data. The traditional method—hardcoding IDs or dynamically querying active resources via data sources—creates fragile, uncoordinated execution paths.
Modern architectures address this by wiring dependencies declaratively. When an upstream dependency changes, downstream modules are programmatically notified and queued for execution:
# Automatically pipe the platform team's VPC ID output into App A's database input
resource "snapcd_module_input_from_output" "vpc_id"
module_id = snapcd_module.app_a_database.id
input_kind = "Param"
name = "vpc_id"
output_module_id = snapcd_module.networking.id
output_name = "vpc_id"
Under this model, if the platform team modifies a subnet or updates a VPC configuration, the system automatically triggers a terraform plan for all downstream application modules that consume that VPC ID.
This workflow eliminates manual coordination, prevents broken references, and ensures that downstreams are updated safely and transparently through their respective approval channels.
3. Shift from Monolithic Code Reviews to Domain-Specific Approvals
By aligning IaC modules with team boundaries, organizations can eliminate the centralized gatekeeper bottleneck.
The networking team retains sole approval authority over pull requests affecting prod/platform. Application developers can confidently approve and merge changes to their own databases in prod/app-a without waiting for platform review. This trust-but-verify model accelerates delivery while preserving compliance boundaries.
6. Conclusion: Navigating Conway’s Law in IaC
Conway’s Law states that organizations design systems that mirror their own communication structures. Monolithic Terraform is a reflection of a centralized command-and-control IT structure—a model that is increasingly incompatible with modern, fast-paced product engineering.
By breaking down the "terralith" into sovereign, team-owned modules governed by declarative dependencies and isolated permissions, enterprises can unlock the true value of cloud infrastructure: security, speed, and scalability.
